ArcPyandArcGIS–GeospatialAnalysiswithPython

TableofContents

ArcPyandArcGIS–GeospatialAnalysiswithPython

Credits

AbouttheAuthor

AbouttheReviewers

www.PacktPub.com

Supportfiles,eBooks,discountoffers,andmore

Whysubscribe?

FreeaccessforPacktaccountholders

Preface

Whatthisbookcovers

Whatyouneedforthisbook

Whothisbookisfor

Conventions

Readerfeedback

Customersupport

Downloadingtheexamplecode

Downloadingthecolorimagesofthisbook

Errata

Piracy

Questions

1.IntroductiontoPythonforArcGIS

OverviewofPython

Pythonasaprogramminglanguage

Interpretedlanguage

Standard(built-in)library

Thegluelanguage

Wrappermodules

ThebasicsofPython

Importstatements

Variables

Forloops

If/Elif/Elsestatements

Whilestatements

Comments

Datatypes

Strings

Integers

Floats

Lists

Tuples

Dictionaries

Iterabledatatypes

Otherimportantconcepts

Indentation

Functions

Keywords

Namespaces

Zero-basedindexing

ImportantPythonModulesforGISAnalysis

TheArcPymodule

TheOperatingSystem(OS)module

ThePythonSystem(SYS)module

TheXLRDandXLWTmodules

Commonlyusedbuilt-infunctions

Commonlyusedstandardlibrarymodules

Summary

2.ConfiguringthePythonEnvironment

WhatisaPythonscript?

HowPythonexecutesascript

WhatisthePythoninterpreter?

WhereisthePythoninterpreterlocated?

WhichPythoninterpretershouldbeused?

Howdoesthecomputerknowwheretheinterpreteris?

MakePythonscriptsexecutablewhenclickedon

IntegratedDevelopmentEnvironments(IDEs)

IDLE

PythonWin

AptanaStudio3

IDEsummary

Pythonfolderstructure

Wheremodulesreside

UsingPython’ssysmoduletoaddamodule

Thesys.path.append()method

Summary

3.CreatingtheFirstPythonScript

Prerequisites

ModelBuilder

CreatingamodelandexportingtoPython

ModelingtheSelectandBuffertools

AddingtheIntersecttool

Tallyingtheanalysisresults

Exportingthemodelandadjustingthescript

Theautomaticallygeneratedscript

FilepathsinPython

Continuingthescriptanalysis:theArcPytools

TheIntersecttoolandstringmanipulation

Thestringmanipulationmethod1–stringaddition

Thestringmanipulationmethod2–stringformatting#1

Thestringmanipulationmethod3–stringformatting#2

AdjustingtheScript

AddingtheCSVmoduletothescript

Accessingthedata:Usingacursor

Thefinalscript

Summary

4.ComplexArcPyScriptsandGeneralizingFunctions

Pythonfunctions–Avoidrepeatingcode

Technicaldefinitionoffunctions

Afirstfunction

Functionswithparameters

Usingfunctionstoreplacerepetitivecode

Moregeneralizationofthefunctions

Summary

5.ArcPyCursors–Search,Insert,andUpdate

Thedataaccessmodule

Attributefieldinteractions

Updatecursors

Updatingtheshapefield

Adjustingapointlocation

DeletingarowusinganUpdateCursor

UsinganInsertCursor

Insertingapolylinegeometry

Insertingapolygongeometry

Summary

6.WorkingwithArcPyGeometryObjects

ArcPygeometryobjectclasses

ArcPyPointobjects

ArcPyArrayobjects

ArcPyPolylineobjects

ArcPyPolygonobjects

Polygonobjectbuffers

OtherPolygonobjectmethods

ArcPygeometryobjects

ArcPyPointGeometryobjects

Summary

7.CreatingaScriptTool

Addingdynamicparameterstoascript

Displayingscriptmessagesusingarcpy.AddMessage

Addingdynamiccomponentstothescript

CreatingaScripttool

Labellinganddefiningparameters

Addingdatatypes

AddingtheBusStopfeatureclassasaparameter

AddingtheCensusBlockfeatureclassasaparameter

AddingtheCensusBlockfieldasaparameter

Addingtheoutputspreadsheetasaparameter

Addingthespreadsheetfieldnamesasaparameter

AddingtheSQLStatementasaparameter

Addingthebusstopfieldsasaparameter

Inspectingthefinalscript

RunningtheScriptTool

Summary

8.IntroductiontoArcPy.Mapping

UsingArcPywithmapdocuments

Inspectingandreplacinglayersources

Fixingthebrokenlinks

Fixingthelinksofindividuallayers

ExportingtoPDFfromanMXD

Adjustingmapdocumentelements

Automatedmapdocumentadjustment

Thevariables

Themapdocumentobjectandthetextelements

Thelayerobjects

Replacingthedatasources

Adjustinglayervisibility

Generatingabufferfromthebusstopsfeatureclass

Intersectingthebusstopbufferandcensusblocks

Populatingtheselectedbusstopandbufferfeatureclasses

Updatingthetextelements

ExportingtheadjustedmaptoPDF

RunningthescriptinthePythonWindow

Summary

9.MoreArcPy.MappingTechniques

Usingarcpy.mappingtocontrolLayerobjects

Layerobjectmethodsandproperties

Definitionqueries

Controllingthedataframewindowextentandscale

AddingaLayerobject

Exportingthemaps

Summary

10.AdvancedGeometryObjectMethods

CreatingaPythonmodule

The__init__.pyfile

Addingadvancedanalysiscomponents

AdvancedPolygonobjectmethods

Generatingrandompointstorepresentpopulation

Usingthefunctionswithinascript

CreatinganXLSusingXLWT

Summary

11.NetworkAnalystandSpatialAnalystwithArcPy

TheNetworkAnalystextension

UsingNetworkAnalyst

CreatingaFeatureDataset

Importingthedatasets

CreatingtheNetworkDataset

AccessingtheNetworkDatasetusingArcPy

Breakingdownthescript

TheNetworkAnalystmodule

AccessingtheSpatialAnalystExtension

Addingelevationtothebusstops

UsingMapAlgebratogenerateelevationinfeet

Addinginthebusstopsandgettingelevationvalues

Thefinalresult

Summary

12.TheEndoftheBeginning

Gettingfieldinformationfromfeatureclasses

AccessingtheListFields’properties

Listcomprehensions

Creatingthefieldinformationfunctions

Queryingfeatureclassinformation

GeneratingFileGeodatabasesandfeatureclasses

Generatingafeatureclass

Settingupthescripttoolparameters

Environmentalsettings

Resolutionandtolerancesettings

Summary

Index

ArcPyandArcGIS–GeospatialAnalysiswithPython

ArcPyandArcGIS–GeospatialAnalysiswithPythonCopyright©2015PacktPublishing

Allrightsreserved.Nopartofthisbookmaybereproduced,storedinaretrievalsystem,ortransmittedinanyformorbyanymeans,withoutthepriorwrittenpermissionofthepublisher,exceptinthecaseofbriefquotationsembeddedincriticalarticlesorreviews.

Everyefforthasbeenmadeinthepreparationofthisbooktoensuretheaccuracyoftheinformationpresented.However,theinformationcontainedinthisbookissoldwithoutwarranty,eitherexpressorimplied.Neithertheauthor,norPacktPublishing,anditsdealersanddistributorswillbeheldliableforanydamagescausedorallegedtobecauseddirectlyorindirectlybythisbook.

PacktPublishinghasendeavoredtoprovidetrademarkinformationaboutallofthecompaniesandproductsmentionedinthisbookbytheappropriateuseofcapitals.However,PacktPublishingcannotguaranteetheaccuracyofthisinformation.

Firstpublished:February2015

Productionreference:1210215

PublishedbyPacktPublishingLtd.

LiveryPlace

35LiveryStreet

BirminghamB32PB,UK.

ISBN978-1-78398-866-2

www.packtpub.com

CreditsAuthor

SilasToms

Reviewers

AlessioDiLorenzo

DaraO’Beirne

MarkPazolli

MarjorieRoswell

CommissioningEditor

AshwinNair

AcquisitionEditor

HarshaBharwani

ContentDevelopmentEditor

AkashdeepKundu

TechnicalEditor

DeeptiTuscano

CopyEditors

AartiSaldanha

AdithiShetty

ProjectCoordinator

MiltonDsouza

Proofreaders

SimranBhogal

JoannaMcMahon

BernadetteWatkins

Indexer

PriyaSane

ProductionCoordinator

AlwinRoy

CoverWork

AlwinRoy

AbouttheAuthorSilasTomsisageospatialprogrammerandanalystwithaloveofgeography,history,food,andsports.HeresidesintheSanFranciscoBayAreaandcan’tdecidewhichsideoftheBayismorebeautiful.Hereceivedabachelor’sdegreeinGeographyfromHumboldtStateUniversityandiscurrentlypursuingamaster’sdegreeinGISatSanFranciscoStateUniversity.WithabackgroundinGISanalysisforcitygovernmentsandenvironmentalconsulting,SilaslovesthecombinationofGISandPythonforanalysisautomationanddatamanipulation.

WorkingforAriniGeographics,SilasishelpinggovernmentsunderstandhowGIScanorganizeandsimplifythemanagementofinfrastructureandtheenvironment.ThisdualroleasaprogrammerandanalystallowshimtousePythonandGIStoquicklyproducegeospatialdataandtools.Combinedwithwebmapping,thesetoolsaretransforminghowgovernmentsworktoservethepublic.HealsoteachesworkshopsonArcPyandwebmappingattheCityCollegeofSanFrancisco,whilehopingtoonedayfinishhismaster’sthesis.

SilashasworkedasarevieweronthebookPythonGeospatialAnalysis,PacktPublishingandisworkingonthebookPythonGeospatialDevelopment,PacktPublishingtobepublishedin2015.

Iwouldliketothankmygirlfriend,Christine,forherencouragementandpatience.Iwouldliketothankmyboss,GabrielPaun,forhisinspirationandforpushingmetobecomeatrueGISprofessional.IwouldliketothankthefacultyatHSUandSFSUfortheirhelpalongtheway,andIwouldliketothankmyfamilyfortheirbeliefinmeandforneveraskingmeifIwasgoingtobecomeateacherwithmygeographydegree(eventhoughIhaveandIloveit!).

AbouttheReviewersAlessioDiLorenzoisamarinebiologistandhasanMScinGeographicalInformationSystems(GIS)andRemoteSensing.Since2006,hehasbeendealingwiththeanalysisanddevelopmentofGISapplicationsdedicatedtothestudyandspreadofenvironmentalandepidemiologicaldata.HeisexperiencedintheuseofthemainproprietaryandopensourceGISsoftwareandprogramminglanguages.

DaraO’BeirneisacertifiedGISProfessional(GISP)withovereightyearsofGISandPythonexperience.DaraearnedbothhisBachelorsandMastersofArtsdegreesingeographyfromSanFranciscoStateUniversity.DaraiscurrentlyaGISAnalystworkingatAriniGeographicsinSantaClara,CA.BeforejoiningAriniGeographics,DarawasaGISAnalystandtechnicalleadatTowillInc.,aGISandLandSurveyingcompanyinNorthernCalifornia.AtTowill,DaraplayedacentralroleindevelopingandimplementingproceduresrelatedtothecollectionandanalysisofLiDARdataforenvironmentalandengineeringapplications.PriortoTowill,DaragainedhisprofessionalGISexperienceworkingfortheGoldenGateNationalRecreationAreamanagedbytheNationalParkService,oneofthelargesturbanparksystemsintheworld,whichincludesNationaltreasures,suchasAlcatraz,MuirWoods,andtheMarinHeadlands.HisMaster’sThesisexaminedtheerrorsassociatedwithmeasuringtreeheightsinanurbanenvironmentwithbothtraditionalfieldmethodsandairborneLiDARdata.

Iwouldliketothankmywife,Kate,anddaughter,AnyaO’Beirne,fortheirpatienceandassistanceduringthereviewofthisbook.

MarjorieRoswellisawebdeveloperandmapmakerfromBaltimore,MD.ShepurchasedherfirstGISin1991,andbuiltanapplicationtoassistcitizencallerstotheBaltimoreOfficeofRecycling.Recentprojectsincludeinteractivemapsoflegislativescores,politicalendorsements,committees,electiondata,andadvocacyinterests.

Hersitehttp://committeemaps.org/detailsCongressionalcommitteemembership,whilethesitehttp://farmbillprimer.org/isdevotedtomappingandchartingfederalfoodandfarmpolicy.

MarjorieistheauthorofDrupal5ViewsRecipes,PacktPublishing.ShewasthetechnicalreviewerofjQueryUI1.10,TheUserInterfaceLibraryforjQuery,PacktPublishing.

MarkPazolliisanengineeranddatascientistwhousesArcGISandPythontohelphisemployersdecipherthemountainsofdatatheykeepontheassetsoftheWesternAustralianelectricalnetwork.HehasqualificationsinElectricalEngineering,ComputerScience,andAppliedMathematics.Heappreciatesexcellentdesignandenjoysbuildinginterestingthings.

www.PacktPub.com

Supportfiles,eBooks,discountoffers,andmoreForsupportfilesanddownloadsrelatedtoyourbook,pleasevisitwww.PacktPub.com.

DidyouknowthatPacktofferseBookversionsofeverybookpublished,withPDFandePubfilesavailable?YoucanupgradetotheeBookversionatwww.PacktPub.comandasaprintbookcustomer,youareentitledtoadiscountontheeBookcopy.Getintouchwithusat<service@packtpub.com>formoredetails.

Atwww.PacktPub.com,youcanalsoreadacollectionoffreetechnicalarticles,signupforarangeoffreenewslettersandreceiveexclusivediscountsandoffersonPacktbooksandeBooks.

https://www2.packtpub.com/books/subscription/packtlib

DoyouneedinstantsolutionstoyourITquestions?PacktLibisPackt’sonlinedigitalbooklibrary.Here,youcansearch,access,andreadPackt’sentirelibraryofbooks.

Whysubscribe?FullysearchableacrosseverybookpublishedbyPacktCopyandpaste,print,andbookmarkcontentOndemandandaccessibleviaawebbrowser

FreeaccessforPacktaccountholdersIfyouhaveanaccountwithPacktatwww.PacktPub.com,youcanusethistoaccessPacktLibtodayandview9entirelyfreebooks.Simplyuseyourlogincredentialsforimmediateaccess.

PrefaceArcGIS,theGISsoftwarefromindustryleaderESRI,allowsfortheanalysisandpresentationofgeospatialdata.

TheintegrationofPythonintoArcGIShasmadetheArcPymoduleanimportanttoolforGISstudentsandprofessionals.TheArcPymoduleprovidesapowerfulwaytoimproveproductivitywhenperforminggeospatialanalysis.FrombasicPythonscriptingthroughadvancedArcPymethodsandproperties,ArcPyandotherPythonmoduleswillimprovethespeedandrepeatabilityofanyGISworkflow.

ThisbookwillguideyoufrombasicPythonscriptingtoadvancedscriptingtools.Itfocusesongeospatialanalysisscriptingandtouchesonautomatingcartographicoutput.Bytheendofthisbook,youwillbeabletocreatereusablemodules,addrepeatableanalysesasscripttoolsinArcToolbox,andexportmapsautomatically.Byreducingthetime-consumingnatureofGISfromdaystohours,oneGISprofessionalcanbecomeaspowerfulasawholeteam.

WhatthisbookcoversChapter1,IntroductiontoPythonforArcGIS,offersaquickintroductiontothebasicsofPython,includingotherusesfortheprogramminglanguage.ItcoversPythondatatypesandimportantmodulesusedthroughoutthebook.

Chapter2,ConfiguringthePythonEnvironment,isanintroductiontohowPythonworks:itsfolderstructure,executables,andmodules.Italsoexplainsimportingmodulesintoscripts,thebuilt-inmodules,andcoversIntegratedDevelopmentEnvironments(IDEs),whicharepowerfulprogrammingaids.

Chapter3,CreatingtheFirstPythonScript,demonstrateshowtouseArcGISModelBuildertomodelthefirstanalysisandthenexportitasaPythonscript.StringmanipulationsandhowtousefilepathsinPythonarealsointroduced.

Chapter4,ComplexArcPyScriptsandGeneralizingFunctions,examineshowtoperformanalysesandproduceoutputsthatarenotpossibleusingModelBuilder.Byusingfunctions,orreusablecodeblocks,repeatingcodeisavoided.

Chapter5,ArcPyCursors–Search,Insert,andUpdate,coversArcPydataaccesscursorsandhowtheyareusedtosearch,update,orinsertrecordsinfeatureclassesandtables.Itexplainsthequirksofiteratingusingcursors,andhowtoonlyselectorupdatetherecordsofinterest.

Chapter6,WorkingwithArcPyGeometryObjects,exploresArcPyGeometryobjectsandhowtheyarecombinedwithcursorstoperformspatialanalysis.Itdemonstrateshowtobuffer,clip,reproject,andmoreusingthedatacursorsandtheArcpygeometrytypeswithoutusingArcToolbox.

Chapter7,CreatingaScriptTool,explainshowtomakescriptsintotoolsthatappearinArcToolboxandaredynamicinnature.ItexplainshowthetoolsandscriptscommunicateandhowtosetuptheArcTooldialogtocorrectlypassparameterstothescript.

Chapter8,IntroductiontoArcPy.Mapping,exploresthepowerfulArcpy.Mappingmoduleandhowtofixbrokenlayerlinks,turnlayersonandoff,anddynamicallyadjusttitlesandtext.Itshowshowtocreatedynamicmapoutputbasedonageospatialanalysis.

Chapter9,MoreArcPy.MappingTechniques,introducesLayerobjects,andtheirmethodsandproperties.Itdemonstrateshowtocontrolmapscalesandextentsfordataframes,andcoversautomatedmapexport.

Chapter10,AdvancedGeometryObjectMethods,expandsontheArcPyGeometryobjectmethodsandproperties.Italsoexplainshowtocreateamoduletosavecodeforreuseinsubsequentscripts,anddemonstrateshowtocreateExcelspreadsheetscontainingresultsfromageospatialanalysis.

Chapter11,NetworkAnalystandSpatialAnalystwithArcPy,introducesthebasicsofusingArcPyforadvancedgeospatialanalysisusingtheArcGISforDesktopNetworkAnalystandSpatialAnalystExtensions.

Chapter12,TheEndoftheBeginning,coversotherimportanttopicsthatneedtobeunderstoodtohaveafullgraspofArcPy.ThesetopicsincludetheEnvironmentSettings,XYvaluesandZandMresolutions,SpatialReferenceSystems(Projections),theDescribefunctions,andmore.

WhatyouneedforthisbookYouwillneedtheproprietaryorfreeversionofArcGIS10.1/10.2/10.3.Tosupportyourenvironment,youwillneed2GBRAM,32-bitor64bitmachinehardwareconfiguration,andWindows7/8.Python2.7isrequiredtodotheprogrammingandisinstalledalongwithArcGIS.

WhothisbookisforThisbookisintendedforGISstudentsandprofessionalswhoneedanunderstandingofhowtouseArcPytoreducerepetitivetasksandperformanalysisfaster.ItisalsoavaluablebookforPythonprogrammerswhowouldliketounderstandhowtoautomategeospatialanalysisusingtheindustrystandardArcGISsoftwareanditsArcPymodule.

ConventionsInthisbook,youwillfindanumberofstylesoftextthatdistinguishbetweendifferentkindsofinformation.Herearesomeexamplesofthesestyles,andanexplanationoftheirmeaning.

Codewordsintext,databasetablenames,foldernames,filenames,fileextensions,pathnames,dummyURLs,userinput,andTwitterhandlesareshownasfollows:“Thetwodatapieces,theBusStopIDandtheaveratePopvariablearethenaddedtoalist.”

Ablockofcodeissetasfollows:

witharcpy.da.SearchCursor(Intersect71Census,["STOPID","POP10"])as

cursor:

forrowincursor:

busStopID=row[0]

pop10=row[1]

ifbusStopIDnotindataDictionary.keys():

dataDictionary[busStopID]=[pop10]

else:

dataDictionary[busStopID].append(pop10)

Anycommand-lineinputoroutputiswrittenasfollows:

>>>aString="Thisisastring"

>>>bString="andthisisanotherstring"

>>>aString+bString

Newtermsandimportantwordsareshowninbold.Wordsthatyouseeonthescreen,inmenusordialogboxesforexample,appearinthetextlikethis:“Selectitbyclickingonit,andthenclickingontheEditbutton.”

NoteWarningsorimportantnotesappearinaboxlikethis.

TipTipsandtricksappearlikethis.

ReaderfeedbackFeedbackfromourreadersisalwayswelcome.Letusknowwhatyouthinkaboutthisbook—whatyoulikedormayhavedisliked.Readerfeedbackisimportantforustodeveloptitlesthatyoureallygetthemostoutof.

Tosendusgeneralfeedback,simplysendane-mailto<feedback@packtpub.com>,andmentionthebooktitleviathesubjectofyourmessage.

Ifthereisatopicthatyouhaveexpertiseinandyouareinterestedineitherwritingorcontributingtoabook,seeourauthorguideonwww.packtpub.com/authors.

CustomersupportNowthatyouaretheproudownerofaPacktbook,wehaveanumberofthingstohelpyoutogetthemostfromyourpurchase.

DownloadingtheexamplecodeYoucandownloadtheexamplecodefilesforallPacktbooksyouhavepurchasedfromyouraccountathttp://www.packtpub.com.Ifyoupurchasedthisbookelsewhere,youcanvisithttp://www.packtpub.com/supportandregistertohavethefilese-maileddirectlytoyou.

DownloadingthecolorimagesofthisbookWealsoprovideyouwithaPDFfilethathascolorimagesofthescreenshots/diagramsusedinthisbook.Thecolorimageswillhelpyoubetterunderstandthechangesintheoutput.Youcandownloadthisfilefromhttp://www.packtpub.com/sites/default/files/downloads/8662OS_ColorImages.pdf.

ErrataAlthoughwehavetakeneverycaretoensuretheaccuracyofourcontent,mistakesdohappen.Ifyoufindamistakeinoneofourbooks—maybeamistakeinthetextorthecode—wewouldbegratefulifyoucouldreportthistous.Bydoingso,youcansaveotherreadersfromfrustrationandhelpusimprovesubsequentversionsofthisbook.Ifyoufindanyerrata,pleasereportthembyvisitinghttp://www.packtpub.com/submit-errata,selectingyourbook,clickingontheErrataSubmissionFormlink,andenteringthedetailsofyourerrata.Onceyourerrataareverified,yoursubmissionwillbeacceptedandtheerratawillbeuploadedtoourwebsiteoraddedtoanylistofexistingerrataundertheErratasectionofthattitle.

Toviewthepreviouslysubmittederrata,gotohttps://www.packtpub.com/books/content/supportandenterthenameofthebookinthesearchfield.TherequiredinformationwillappearundertheErratasection.

PiracyPiracyofcopyrightmaterialontheInternetisanongoingproblemacrossallmedia.AtPackt,wetaketheprotectionofourcopyrightandlicensesveryseriously.Ifyoucomeacrossanyillegalcopiesofourworks,inanyform,ontheInternet,pleaseprovideuswiththelocationaddressorwebsitenameimmediatelysothatwecanpursuearemedy.

Pleasecontactusat<copyright@packtpub.com>withalinktothesuspectedpiratedmaterial.

Weappreciateyourhelpinprotectingourauthors,andourabilitytobringyouvaluablecontent.

QuestionsYoucancontactusat<questions@packtpub.com>ifyouarehavingaproblemwithanyaspectofthebook,andwewilldoourbesttoaddressit.

Chapter1.IntroductiontoPythonforArcGISInthischapter,wewilldiscussthedevelopmentofPythonasaprogramminglanguage,fromitsbeginninginthelate1980stoitscurrentstate.Wewilldiscussthephilosophyofdesignthatspurreditsdevelopment,andtouchonimportantmodulesthatwillbeusedthroughoutthebook,especiallyfocusingonthemodulesbuiltintothePythonstandardlibrary.ThisoverviewofthelanguageanditsfeatureswillhelpexplainwhatmakesPythonagreatlanguageforArcGISautomation.

Thischapterwillcover:

AquickoverviewofPython:Whatitisanddoes,whocreatedit,andwhereitisnowTheArcPymoduleandotherimportantmodulesPythonasageneralpurposeprogramminglanguage

OverviewofPythonPython,createdbyGuidovanRossumin1989,wasnamedafterhisfavoritecomedytroupe,MontyPython.HisworkgroupatthetimehadatraditionofnamingprogramsafterTVshows,andhewantedsomethingirreverentanddifferentfromitspredecessors-ABC,Pascal,Ada,Eiffel,FORTRAN,andothers.SohesettledonPython,feelingitwasabitedgyandcatchyaswell.It’scertainlymorefuntosaythanC,thelanguageonwhichPythonisbased.

Today,Pythonisamajorprogramminglanguage.Itisusedinwebdevelopment,databaseadministration,andeventoprogramrobots.MostimportantlytoGISAnalysts,PythoncanbeusedtocontrolArcGIStoolsandMapDocumentstoproducegeospatialdataandmapsinanorganizedandspeedymannerusingtheexcellentArcPymodule.

ArcPyisinstalledwithArcGISfordesktopandArcGISforserver.ArcPyhasbeentheofficialArcGISscriptinglanguagesinceArcGIS10.0andhassteadilyimprovedinfunctionalityandimplementation.ThisbookwilltargetArcGISforDesktop10.1andlater,andwilldemonstratehowtomakeuseofPythonanditspowerfulprogramminglibraries(ormodules)whencraftingcomplexgeospatialanalyses.

PythonasaprogramminglanguageOverthepast40years,programminglanguageshavedevelopedfromassemblyandmachinecodetowardshigh-levelabstractedlanguagesthataremuchclosertoEnglish.ThePythonprogramminglanguagewasdesignedtoovercomemanyissuesthatprogrammerswerecomplainingaboutinthe1980s:slowdevelopmenttime,overlycomplicatedsyntax,andhorriblereadability.VanRossumwantedtodevelopalanguagethatcouldenablerapidcodedevelopmentandtesting,havesimpleoratleastreadable)syntax,andproduceresultswithfewerlinesofcode,inlesstime.ThefirstversionofPython(0.9.0)wasreleasedin1991andwasfreelyobtainablefromthestart;Pythonwasopensourcebeforethetermopensourcewasinvented.

InterpretedlanguagePythonisaninterpretedlanguage.ItiswritteninC,acompiledlanguage,andthecodeisinterpretedfromPythonintoCbeforeitisexecuted.Practically,thismeansthatthecodeisexecutedassoonasitisconvertedandcompiled.WhilecodeinterpretationcanhavespeedimplicationsfortheexecutionofPython-basedprograms,thefasterdevelopmenttimeallowedbyPythonmakesthisdrawbackeasytoignore.Testingofcodesnippetsismuchfasterinaninterpretiveenvironment,anditisperfecttocreatescriptstoautomatebasic,repeatablecomputingtasks.Pythonscriptshavethe.pyextentions.Oncethecodehasbeeninterpreted,asecondPythonscript(withthe.pycextentions)isgeneratedtosavethecompiledcode.The.pycscriptwillbeautomaticallyrecompiledwhenchangesaremadeintheoriginal.pyscript.

Standard(built-in)libraryPython,wheninstalled,hasabasicsetoffunctionalitythatisreferredtoasthestandardlibrary.ThesetoolsallowPythontoperformstringmanipulations,mathcomputations,andHTTPcallsandURLparsing,alongwithmanyotherfunctions.Someofthetoollibraries,knowntoPythonprogrammersasmodules,arebuilt-inandavailableassoonasPythonisstarted,whileothersmustbeexplicitlycalledusingtheimportkeywordtomaketheirfunctionsandclassesavailable.OthermoduleshavebeendevelopedbythirdpartiesandcanbedownloadedandinstalledontothePythoninstallationasneeded.

ManynewprogrammerswonderifPythonisarealprogramminglanguage,whichisaloadedquestion.Theanswerisyes;Pythoncanbeusedtocreatecompleteprograms,buildwebsites,runcomputernetworks,andmuchmore.Thebuilt-inmodulesandadd-onmodulesmakePythonverypowerful,anditcanbe(andhasbeen)usedfornearlyanypartofacomputer—operatingsystems,databases,webservers,desktopapplications,andsoon.Itisnotalwaysthebestchoiceforthedevelopmentofthesetools,butthathasnotstoppedprogrammersfromtryingandevensucceeding.

ThegluelanguagePythonisatitsbestwhenitisusedasagluelanguage.ThistermdescribestheuseofPythontocontrolotherprograms,sendinginputstothemandcollectingoutputs,whicharethensenttoanotherprogramorwrittentodisk.AnArcGISexamplewouldbetousePythontodownloadzippedshapefilesfromawebsite,unzippingthefiles,processingthefilesusingArcToolbox,andcompilingtheresultsintoanExcelspreadsheet.AllofthisisaccomplishedusingfreelyavailablemodulesthatareeitherincludedinPython’sstandardlibrary,oraddedwhenArcGISisinstalled.

WrappermodulesTheArcPymoduleisawrappermodule.WrappermodulesarecommoninPython,andaresonamedbecausetheywrapPythonontothetoolswewillneed.TheyallowustousePythontointerfacewithotherprogramswritteninCorotherprogramminglanguages,usingtheApplicationProgrammingInterface(API)ofthoseprograms.Forexample,wrappersmakeitpossibletoextractdatafromanExcelspreadsheetandtransformorloadthedataintoanotherprogram,suchasArcGIS.Notallmodulesarewrappers;somemodulesarewritteninpurePythonandperformtheiranalysisandcomputationsusingthePythonsyntax.Eitherway,theendresultisthatacomputeranditsprogramsareavailabletobemanipulatedandcontrolledusingPython.

TheZenofPythonwascreatedtobestraightforward,readable,andsimplified,comparedtootherlanguagesthatexistedpreviously.ThisgoverningphilosophywasorganizedintoapoembyTimPeters,anearlyPythondevelopercalledtheZenofPython;itisanEasteregg(ahiddenfeature)includedineveryPythoninstallationandisshownwhenimportthisistypedinthePythoninterpreter:

TheZenofPython,byTimPeters:

Beautifulisbetterthanugly.

Explicitisbetterthanimplicit.

Simpleisbetterthancomplex.

Complexisbetterthancomplicated.

Flatisbetterthannested.

Sparseisbetterthandense.

Readabilitycounts.

Specialcasesaren'tspecialenoughtobreaktherules.

Althoughpracticalitybeatspurity.

Errorsshouldneverpasssilently.

Unlessexplicitlysilenced.

Inthefaceofambiguity,refusethetemptationtoguess.

Thereshouldbeone--andpreferablyonlyone--obviouswaytodo

Althoughthatwaymaynotbeobviousatfirstunlessyou'reDutch.

Nowisbetterthannever.

Althoughneverisoftenbetterthan*right*now.

Iftheimplementationishardtoexplain,it'sabadidea.

Iftheimplementationiseasytoexplain,itmaybeagoodidea.

Namespacesareonehonkinggreatidea—let'sdomoreofthose!

NoteGotohttps://www.python.org/doc/humor/formoreinformation.

ThebasicsofPythonPythonhasanumberoflanguagerequirementsandconventionsthatallowforthecontrolofmodulesandstructuringofcode.Thefollowingareanumberofimportantbasicconcepts,whichwillbeusedthroughoutthisbookandwhencraftingscriptsforusewithgeospatialanalyses.

ImportstatementsImportstatementsareusedtoaugmentthepowerofPythonbycallingothermodulesforuseinthescript.ThesemodulescanbepartofthestandardPythonlibraryofmodules,suchasthemathmodule(usedtodohighermathematicalcalculations)or,importantly,ArcPy,whichwillallowustointeractwithArcGIS.

NoteImportstatementscanbelocatedanywherebeforethemoduleisused,butbyconvention,theyarelocatedatthetopofascript.

Therearethreewaystocreateanimportstatement.Thefirst,andmoststandard,istoimportthewholemoduleasfollows:

importarcpy

Usingthismethod,wecanevenimportmorethanonemoduleonthesameline.Thefollowingimportsthreemodules:arcpy,os(theoperatingsystemmodule),andsys(thePythonsystemmodule):

importarcpy,os,sys

Thenextmethodofimportingascriptistoimportaspecificportionofamodule,insteadofimportingtheentiremodule,usingthefrom<module>import<submodule>syntax:

fromarcpyimportmapping

ThismethodisusedwhenonlyaportionofthecodefromArcPywillbeneeded;ithasthepracticaleffectoflimitingtheamountofmemoryusedbythemodulewhenitiscalled.Wecanalsoimportmultipleportionsofthemoduleinthesamefashion:

fromarcpyimportmapping,da

Thethirdwaytoimportamoduleisthefrom<module>import<submodule>syntax,butbyusinganasterisktoimportallpartsofthemodule:

fromarcpyimport*

Thislastmethodisstillusedbutitisdiscouragedasitcanhaveunforeseenconsequences.Forinstance,thenamesofthevariablesinthemodulemightconflictwithanothervariableinanothermoduleiftheyarenotexplicitlyimported.Forthisreason,itisbesttoavoidthisthirdmethod.However,lotsofexistingscriptsincludeimportstatementsofthistypesobeawareoftheseconsequences.

VariablesVariablesareapartofallprogramminglanguages.Theyareusedtoreferencedataandstoreitinmemoryforuselaterinascript.Therearealotofargumentsoverthebestmethodtonamevariables.NostandardhasbeendevelopedforPythonscriptingforArcGIS.Thefollowingaresomebestpracticestousewhennamingvariables.

Makethemdescriptive:Don’tjustnameavariablex;thatvariablewillbeuselesslaterwhenthescriptisreviewedandthereisnowaytoknowwhatitisusedfor,orwhy.Theyshouldbelongerratherthanshorter,andshouldhintatthedatatheyreferenceoreventhedatatypeoftheobjecttheyreference:

shapefilePath='C:/Data/shapefile.shp'

Usecamelcasetomakethevariablereadable:Camelcaseisatermusedforvariablesthatstartwithalowercaseletterbuthaveuppercaselettersinthemiddle,resemblingacamel’shump:

camelCase='thisisastring'

Includethedatatypeinthevariablename:Ifthevariablecontainsastring,callitvariableString.Thisisnotrequired,andwillnotbeuseddogmaticallyinthisbook,butitcanhelporganizethescriptandishelpfulforotherswhowillreadthesescripts.Pythonisdynamicallytypedinsteadofstatically.Aprogramminglanguagedistinctionmeansthatavariabledoesnothavetobedeclaredbeforeitcanbeused,unlikeVisualBasicorotherstaticallytypedlanguages.Thisimprovesthespeedofwritingascript,butitcanbeproblematicinlongscriptsasthedatatypeofavariablewillnotbeobvious.

NoteTheArcGISdoesnotusecamelcasewhenitexportsPythonscripts,andmanyexampleswillnotincludeit;nevertheless,itisrecommendedwhenwritingnewscripts.Also,variablescannotstartwithanumber.

ForloopsBuiltintoprogramminglanguagesistheabilitytoiterate,orperformarepeatingprocess,overadatasettotransformorextractdatathatmeetsspecificcriteria.Python’smainiterationtoolisknownasaforloop.Thetermforloopmeansthatanoperationwillloop,oriterate,overtheitemsinadatasettoperformtheoperationoneachitem.Thedatasetmustbeiterabletobeusedinaforloop,adistinctiondiscussedfurtherahead.

Wewillbeusingforloopsthroughoutthisbook.HereisasimpleexamplethatusesthePythonInterpretertotakestringvaluesandprinttheminanuppercaseformat,usingaforloop:

>>>newlist=['a','b','c','d']

>>>foriteminnewlist:

printitem.upper()

Theoutputisshownasfollows:

A

B

C

D

Thevariableitemisagenericvariableassignedtoeachobjectasitisenteredintotheforloop,andnotatermrequiredbyPython.Itcouldhavebeenxorvalueinstead.Withintheloop,thefirstobject(a)isassignedtothegenericvariableitemandhastheupperstringfunctionappliedtoittoproducetheoutputA.Oncethisactionhasbeenperformed,thenextobject(b)isassignedtothegenericvariabletoproduceanoutput.Thisloopisrepeatedforallmembersofthedatasetnewlist;oncecompleted,thevariableitemwillstillcarrythevalueofthelastmemberofthedataset(dinthiscase).

TipDownloadingtheexamplecode

YoucandownloadtheexamplecodefilesforallPacktbooksyouhavepurchasedfromyouraccountathttp://www.packtpub.com.Ifyoupurchasedthisbookelsewhere,youcanvisithttp://www.packtpub.com/supportandregistertohavethefilese-maileddirectlytoyou.

If/Elif/ElsestatementsConditionalstatements,calledif/elsestatementsinPython,arealsostandardinprogramminglanguages.Theyareusedwhenevaluatingdata;whencertainconditionsaremet,oneactionwillbetaken(theinitialifstatement;ifanotherconditionismet,anotheractionistaken;thisisanelifstatement),andifthedatadoesnotmeetthecondition,afinalactionisassignedtodealwiththosecases(theelsestatement).ThesearesimilartoawhereconditionalinaSQLstatementusedwiththeSelecttoolinArcToolbox.Hereisanexampleofhowtouseanif/elsestatementtoevaluatedatainalist(adatatypediscussedfurtherahead)andfindtheremainderwhendividedusingthemodulusoperator(%)andPython’sisequaltooperator(==):

>>>data=[1,2,4,5,6,7,10]

>>>forvalindata:

ifval%2==0:

printval,"noremainder"

elifval%3==2:

printval,"remainderoftwo"

else:

print"finalcase"

Theoutputisshownasfollows:

finalcase

2noremainder

4noremainder

5remainderoftwo

6noremainder

finalcase

10noremainder

WhilestatementsAnotherimportantevaluationtoolisthewhilestatement.Itisusedtoperformanactionwhileaconditionistrue;whentheconditionisfalse,theevaluationwillstop.Notethattheconditionmustbecomefalse,ortheactionwillbealwaysperformed,creatinganinfiniteloopthatwillnotstopuntilthePythoninterpreterisshutoffexternally.Hereisanexampleofusingawhilelooptoperformanactionuntilatrueconditionbecomesfalse:

>>>x=0

>>>whilex<5:

printx

x+=1

Theoutputisshownasfollows:

0

1

2

3

4

CommentsCommentsinPythonareusedtoaddnoteswithinascript.Theyaremarkedbyapoundsign,andareignoredbythePythoninterpreterwhenthescriptisrun.Commentsareusefultoexplainwhatacodeblockdoeswhenitisexecuted,ortoaddhelpfulnotesthatscriptauthorswouldlikefuturescriptuserstoread:

#Thisisacomment

Whileitisaprogrammingtruismthatgoodcodeiswell-commentedcode,manyprogrammersskipthisvaluablestep.Also,toomanycommentscanreducetheirusefulnessandthescript’sreadability.Ifvariablesaredescriptiveenough,andcodeiswell-organized,commentsarelessnecessary;writingthecodeasverboseandaswell-organizedaspossiblewillrequirelesstimetobespentoncomments.

DatatypesGISusespoints,lines,polygons,coverages,andrasterstostoredata.EachoftheseGISdatatypescanbeusedindifferentwayswhenperformingananalysisandhavedifferentattributesandtraits.Python,similartoGIS,hasdatatypesthatorganizedata.ThemaindatatypesinPythonarestrings,integers,floats,lists,tuples,anddictionaries.Theyeachhavetheirownattributesandtraits(orproperties),andareusedforspecificpartsofcodeautomation.Therearealsobuilt-infunctionsthatallowfordatatypestobeconverted(orcasted)fromonetypetoanother;forinstance,theinteger1canbeconvertedtothestring1usingthestr()function:

>>>variable=1

>>>newvar=str(variable)

>>>newvar

Theoutputisshownasfollows:

1

StringsStringsareusedtocontainanykindofcharacter.Theybeginandendwithquotationmarks,witheithersingleordoublequotesused,thoughthestringmustbeginandendwiththesametypeofquotationmarks.Withinastring,quotedtextcanappear;itmustusetheoppositequotationmarkstoavoidconflictingwiththestring.Checkthefollowingexample:

>>>quote='Thisstringcontainsaquote:"Hereisthequote"'

Athirdtypeofstringisalsoemployed,amultiplelinestringthatstartsandendswiththreesinglequotemarks:

>>>multiString='''Thisstringhas

multiplelinesandcangofor

aslongasIwantittoo'''

IntegersIntegersarewholenumbersthatdonothaveanydecimalplaces.Thereisaspecialconsequencetotheuseofintegersinmathematicaloperations;ifintegersareusedfordivision,anintegerresultwillbereturned.Checkoutthiscodesnippetbelowtoseeanexampleofthis:

>>>5/2

Theoutputisshownasfollows:

2

Insteadofanaccurateresultof2.5,Pythonwillreturnthefloorvalue,orthelowestwholeintegerforanyintegerdivisioncalculation.Thiscanobviouslybeproblematicandcancausesmallbugsinscriptsthatcanhavemajorconsequences.

TipPleasebeawareofthisissuewhenwritingscriptsandusefloatstoavoiditasdescribedinthefollowingsection.

FloatsFloatingpointvalues,orfloats,areusedbyPythontorepresentdecimalvalues.Theuseoffloatswhenperformingdivisionisrecommended:

>>>5.0/2

Theoutputisshownasfollows:

2.5

Becausecomputersstorevaluesinabase2binarysystem,therecanbeissuesrepresentingafloatingvaluethatwouldnormallyberepresentedinabase10system.Readdocs.python.org/2/tutorial/floatingpoint.htmlforafurtherdiscussionoftheramificationsofthislimitation.

ListsListsareorderedsetsofdatathatarecontainedinsquarebrackets([]).Listscancontainanyothertypeofdata,includingotherlists.Datatypescanbemixedwithinasinglelist.Listsalsohaveasetofmethodsthatallowthemtobeextended,reversed,sorted,summed,orextractthemaximumorminimumvalue,alongwithmanyothermethods.Datapieceswithinalistareseparatedbycommas.

Listmembersarereferencedbytheirindex,orpositioninthelist,andtheindexalwaysstartsatzero.Lookatthefollowingexampletounderstandthisbetter:

>>>alist=['a','b','c','d']

>>>alist[0]

Theoutputisshownasfollows:

'a'

Thisexampleshowsushowtoextractthefirstvalue(attheindex0)fromthelistcalledalist.Oncealisthasbeenpopulated,thedatawithinitisreferencedbyitsindex,whichispassedtothelistinsquarebrackets.Togetthesecondvalueinalist(thevalueatindex1),thesamemethodisused:

>>>alist[1]

Theoutputisshownasfollows:

'b'

Tomergetwolists,theextendmethodisused:

>>>blist=[2,5,6]

>>>alist.extend(blist)

>>>alist

Theoutputisshownasfollows:

['a','b','c','d',2,5,6]

TuplesTuplesarerelatedtolistsandaredenotedbyparentheses(()).Unlikelists,tuplesareimmutable—theycannotbeadjustedorextendedoncetheyhavebeencreated.Datawithinatupleisreferencedinthesamewayasalist,usingindexreferencesstartingatzero:

>>>atuple=('e','d','k')

>>>atuple[0]

Theoutputisshownasfollows:

'e'

DictionariesDictionariesaredenotedbycurlybrackets({})andareusedtocreatekey:valuepairs.Thisallowsustomapvaluesfromakeytoavalue,sothatthevaluecanreplacethekeyanddatafromthevaluecanbeusedinprocessing.Hereisasimpleexample:

>>>adic={'key':'value'}

>>>adic['key']

Theoutputisshownasfollows:

'value'

Notethatinsteadofreferringtoanindexposition,suchaslistsortuples,thevaluesarereferencedusingakey.Also,keyscanbeanyothertypeofdataexceptlists(becauselistsaremutable).

Thiscanbeveryvaluablewhenreadingashapefileorfeatureclass.UsinganObjectIDasakey,thevaluewouldbealistofrowattributesassociatedwithObjectID.Lookatthefollowingexampletobetterunderstandthisbehavior:

>>>objectIDdic={1:['100','Main','St']}

>>>objectIDdic[1]

Theoutputisshownasfollows:

['100','Main','St']

Dictionariesareveryvaluableforreadinginfeatureclassesandeasilyparsingthroughthedatabycallingonlytherowsofinterest,amongotheroperations.Theyaregreatfororderingandreorderingdataforuselaterinascript,sobesuretopayattentiontothemmovingforward.

IterabledatatypesLists,tuples,andstringsarealliterabledatatypesthatcanbeusedinforloops.Whenenteredintoaforloop,thesedatatypesareoperatedoninorder,unlessotherwisespecified.Forlistsandtuples,thisiseasytounderstand,astheyhaveanobviousorder:

>>>aList=[1,3,5,7]

>>>forvalueinaList:

printvalue*2

Theoutputisshownasfollows:

2

6

10

14

Forstrings,eachcharacterislooped:

>>>aString="esri"

>>>forvalueinaString:

printvalue.upper()

Theoutputisshownasfollows:

E

S

R

I

Dictionariesarealsoiterable,butwithaspecificimplementationthatwillonlyallowdirectaccesstothekeysofthedictionary(whichcanthenbeusedtoaccessthevalues).Also,thekeysarenotreturnedinaspecificorder:

>>>aDict={"key1":"value1",

"key2":"value2"}

>>>forvalueinaDict:

printvalue,aDict[value]

Theoutputisshownasfollows:

key2value2

key1value1

OtherimportantconceptsTheuseofPythonforprogrammingrequiresanintroductiontoanumberofconceptsthatareeitheruniquetoPythonbutrequiredorcommonprogrammingconceptsthatwillbeinvokedrepeatedlywhencreatingscripts.IncludedfollowingareanumberoftheseconceptsthatmustbecoveredtobefluentinPython.

IndentationPython,unlikemostotherprogramminglanguages,enforcesstrictrulesonindentinglinesofcode.ThisconceptisderivedagainfromGuido’sdesiretoproduceclean,readablecode.Whencreatingfunctionsorusingforloops,orif/elsestatements,indentationisrequiredonthesucceedinglinesofcode.Ifaforloopisincludedinsideanif/elsestatement,therewillbetwolevelsofindentation.VeteranprogrammersofotherlanguageshavecomplainedaboutthestrictnatureofPython’sindentation.Newprogrammersgenerallyfindittobehelpfulasitmakesiteasytoorganizecode.NotethatalotofprogrammersnewtoPythonwillcreateanindentationerroratsomepoint,somakesuretopayattentiontotheindentationlevels.

FunctionsFunctionsareusedtotakecodethatisrepeatedoverandoverwithinascript,oracrossscripts,andmakeformaltoolsoutofthem.Usingthekeyworddef,shortforthedefinefunction,functionsarecreatedwithdefinedinputsandoutputs.Theideaofafunctionincomputingisthatittakesdatainonestateandconvertsitintodatainanotherstate,withoutaffectinganyotherpartofthescript.ThiscanbeveryvaluabletoautomateaGISanalysis.

Hereisanexampleofafunctionthatreturnsthesquareofanynumbersupplied:

defsquare(inVal):

returninVal**2

>>>square(3)

Theoutputisshownasfollows:

9

Whilethisofcourseduplicatesasimilarfunctionbuiltintothemathmodule,itshowsthebasicsofafunction.Afunction(generally)acceptsdata,transformsitasneeded,andthenreturnsthenewstateofthedatausingthereturnkeyword.

KeywordsThereareanumberofkeywordsbuiltintoPythonthatshouldbeavoidedwhennamingvariables.Theseincludemax,min,sum,return,list,tuple,def,del,from,not,in,as,if,else,elif,or,while,and,with,amongmanyothers.Usingthesekeywordswillresultinanerror.

NamespacesNamespacesarealogicalwaytoorganizevariablenameswhenavariableinsidea

function(alocalvariable)sharesthesamenameasavariableoutsideofthefunction(aglobalvariable).Localvariablescontainedwithinafunction(eitherinthescriptorwithinanimportedmodule)andglobalvariablescanshareanameaslongastheydonotshareanamespace.

Thisissueoftenariseswhenavariablewithinanimportedmoduleunexpectedlyhasthesamenameofavariableinthescript.PythonInterpreterwillusenamespacerulestodecidewhichvariablehasbeencalled,whichcanleadtoundesirableresults.

Zero-basedindexingAsmentionedintheprecedingsectionthatdescribeslistsandtuples,Pythonindexingandcountingstartsatzero,insteadofone.Thismeansthatthefirstmemberofagroupofdataisatthezeroposition,andthesecondmemberisatthefirstposition,andsoontillthelastposition.

Thisrulealsoapplieswhenthereisaforloopiterationwithinascript.Whentheiterationstarts,thefirstmemberofthedatabeingiteratedisinthezeroposition.

Also,indexingcanbeperformedwhencountingfromthelastmemberofaniterableobject.Inthiscase,theindexofthelastmemberis-1,andthesecondtolastis-2,andsoonbacktothefirstmemberoftheobject.

ImportantPythonModulesforGISAnalysisModules,orcodelibrariesthatcanbecalledbyascripttoincreaseitsprogrammingpotential,areeitherbuiltintoPythonorarecreatedbythirdpartiesandaddedlatertoPython.MostofthesearewritteninPython,butanumberofthemarealsowritteninotherprogramminglanguagesandthenwrappedinPythontomakethemavailablewithinPythonscripts.ModulesarealsousedtomakeotherprogramsavailabletoPython,suchasthetoolsbuiltinMicrosoftWord.

TheArcPymoduleTheArcPymoduleisbothawrappermoduleusedtointeractwiththeArcGIStools,whicharethenexecutedbyArcGISinitsinternalcodeformat,andacodebasethatallowsforadditionalcontrolofgeospatialanalysesandmapproduction.ArcPyisusedtocontrolthetoolsinArcToolbox,butthetoolshavenotbeenrewritteninPython;instead,weareabletousetheArcGIStoolsusingArcPy.ArcPyalsogivesustheabilitytocontrolArcGISMapDocuments(MXDs)andtheobjectsthatMXDsinclude:legends,titles,images,layers,andthemapviewitself.ArcPyalsohastoolsthatarenotavailableinArcToolbox.Themostpowerfulofthesearethedatacursors,especiallythenewDataAnalysisCursorsthatcreateamorePythonicinterfacewithGISdata.Thedatacursors,coveredextensivelyinChapters5,ArcPyCursors:Search,InsertandUpdateandChapter6,WorkingwithArcPyGeometryObjectsareveryusefultoextractrowsofdatafromdatasourcesforanalysis.

TheabilitytocontrolgeospatialanalysesusingArcPyallowsfortheintegrationofArcGIStoolsintoworkflowsthatcontainotherpowerfulPythonmodules.Python’sgluelanguageabilitiesincreasetheusefulnessofArcGISbyreducingtheneedtotreatgeospatialdatainaspecialmanner.

TheOperatingSystem(OS)moduleTheOSmodule,partofthestandardlibrary,allowsPythontoaccessoperatingsystemfunctionality.Acommonuseofthemoduleistousetheos.pathmethodtocontrolfilepathsbydividingthemintodirectorypaths(thatis,folders)andbasepaths(thatis,files).Thereisalsoausefulmethod,os.walk,whichwillwalk-throughadirectoryandreturnallfileswithinthefoldersandsubfolders.TheOSmoduleisaccessedconstantlywhenperformingGISanalysis.

ThePythonSystem(SYS)moduleThesysmodule,partofthestandardlibrary,referstothePythoninstallationitself.IthasanumberofmethodsthatwillgetinformationabouttheversionofPythoninstalled,aswellasinformationaboutthescriptandanyarguments(orparameters)suppliedtothescript,usingthesys.argvmethod.Thesys.pathmethodisveryusefultoappendthePythonfilepath;practically,thismeansthatfolderscontainingscriptscanbereferencedbyotherscriptstomakethefunctionstheycontainimportabletootherscripts.

TheXLRDandXLWTmodulesTheXLRDandXLWTmodulesareusedtoreadandwriteExcelspreadsheets,respectively.ThemodulescanbeveryusefultoextractdatafromlegacyspreadsheetsandconvertthemintousabledataforGISanalysis,ortowriteanalysisresultswhenageospatialanalysisiscompleted.TheyarenotpartofthePythonstandardlibrary,butareinstalledalongwithArcGIS10.2andPython2.7.

Commonlyusedbuilt-infunctionsThereareanumberofbuilt-infunctionsthatwewillusethroughoutthebook.Themainonesarelistedasfollows:

str:Thestringfunctionisusedtoconvertanyothertypeofdataintoastringint:Theintegerfunctionisusedtoconvertastringorfloatintoaninteger.Tonotcreateanerror,anystringpassedtotheintegerfunctionmustbeanumbersuchas1.float:Thefloatfunctionisusedtoconvertastringoranintegerintoafloat,muchliketheintegerfunction.

CommonlyusedstandardlibrarymodulesThefollowingstandardlibrarymodulesmustbeimported:

datetime:Thedatetimemoduleisusedtogetinformationaboutthedateandtime,andconvertstringdatesintoPythondates.math:Themathmoduleisusedforhigherlevelmathfunctionsthatarenecessaryattimes,suchasgettingavalueforPiorcalculatingthesquareofanumber.string:Thestringmoduleisusedforstringmanipulations.csv:TheCSVmoduleisusedtocreateandeditcomma-separatedvaluetypefiles.

Checkouthttps://docs.python.org/2/libraryforacompletelistofthebuilt-inmodulesinthestandardlibrary.

SummaryInthischapter,wediscussedabouttheZenofPythonandcoveredthebasicsofprogrammingusingPython.WebeganourexplorationofArcPyandhowitcanbeintegratedwithotherPythonmodulestoproducecompleteworkflows.WealsodiscussedthePythonstandardlibraryandthebasicdatatypesofPython.

Next,wewilldiscusshowtoconfigurePythonforusewithArcGIS,andexplorehowtouseIntegratedDevelopmentEnvironments(IDEs)towritescripts.

Chapter2.ConfiguringthePythonEnvironmentInthischapter,wewillconfigurebothPythonandourcomputertoworktogethertoexecutePythonscripts.Pathvariablesandenvironmentvariableswillbeconfiguredtoensurethatimportstatementsworkasexpected,andthatscriptsrunwhentheyareclickedon.ThestructureofthePythonfolderwillbediscussed,aswillthelocationoftheArcPymodulewithintheArcGISfolderstructure.WewillalsodiscussIntegratedDevelopmentEnvironments(IDEs),programsdesignedtoassistincodecreationandcodeexecution,andcompareandcontrastexistingIDEstodeterminewhatbenefitseachIDEcanofferwhenscriptingPythoncode.

Thischapterwillcover:

ThelocationofthePythoninterpreter,andhowitiscalledtoexecuteascriptAdjustingthecomputer’senvironmentvariablestoensurecorrectcodeexecutionIntegratedDevelopmentEnvironmentsPython’sfolderstructure,withafocusonwheremodulesarestored

WhatisaPythonscript?Let’sstartwiththeverybasicsofwritingandexecutingaPythonscript.WhatisaPythonscript?Itisasimpletextfilethatcontainsaseriesoforganizedcommandswritteninaformalizedlanguage.Thetextfilehastheextension.py,butotherthanthat,thereisnothingtodistinguishitfromanyothertextfile.ItcanbeopenedusingatexteditorsuchasNotepadorWordpad,butthemagicthatisPythondoesnotresideinaPythonscript.WithoutthePythoninterpreter,aPythonscriptcannotberunandthecommandsitcontainscannotbeexecuted.

HowPythonexecutesascriptUnderstandinghowPythonworkstointerpretascriptandthenexecutethecommandswithinisasimportantasunderstandingthePythonlanguageitself.HoursofdebugginganderrorcheckingcanbeavoidedbytakingthetimetosetupPythoncorrectly.TheinterpretivenatureofPythonmeansthatascriptwillhavetobefirstconvertedintobytecodebeforeitcanbeexecuted.WewillcoverthestepsthatPythontakestoachieveourgoalofautomatingGISanalysis.

WhatisthePythoninterpreter?ThePythoninterpreter,onaWindowsenvironment,isaprogramthathasbeencompiledintoaWindowsexecutable,whichhastheextension.exe.ThePythoninterpreter,python.exe,hasbeenwritteninC,anolderandextensivelyusedprogramminglanguagewithamoredifficultsyntax.

ProgramswritteninC,whicharealsoinitiallywrittenastextfiles,mustbeconvertedintoexecutablesbyacompiler,aspecializedprogramthatconvertsthetextcommandsintomachinecodetocreateexecutableprograms.ThisisaslowprocessthatcanmakeproducingsimpleprogramsinCalaboriousprocess.Thebenefitisthattheprogramsproducedarestandaloneprogramscapableofrunningwithoutanydependencies.Python,ontheotherhand,interpretsandexecutesthePythoncommandsquickly,whichmakesitagreatscriptinglanguage,butthescriptsmustberunthroughaninterpreterandcannotbeexecutedbythemselves.

ThePythoninterpreter,asitsnameimplies,interpretscommandscontainedwithinaPythonscript.WhenaPythonscriptisrun,orexecuted,thesyntaxisfirstcheckedtomakesurethatitconformstotherulesofPython(forexample,indentationrulesarefollowedandthevariablesfollownamingconventions).Then,ifthescriptisvalid,thecommandscontainedwithinareconvertedintobytecode,aspecializedcodethatisexecutedbythebytecodeinterpreter,avirtualmachinewritteninC.Thebytecodeinterpreterfurtherconvertsthebytecode(whichiscontainedwithinfilesthatendwiththeextension.pyc)intothecorrectmachinecodeforthecomputerbeingused,andthentheCPUexecutesthescript.Thisisacomplexprocess,whichallowsPythontomaintainasemblanceofsimplicity.

ThereareotherversionsofthePythoninterpreterthathavebeenwritteninJava(knownasJython)andin.NET(knownasIronPython);thesevariantsareusedtowritePythonscriptsinothercomputingenvironmentsandwillnotbeaddressedinthisbook.TheArcGISinstallerincludesthestandardimplementationofPython,whichisalsocalledCPythontodistinguishitfromthesevariants.

WhereisthePythoninterpreterlocated?ThelocationofthePythoninterpreterwithinthefolderstructureofacomputerisanimportantdetailtomaster.Pythonisoftendownloadeddirectlyfromwww.python.organdinstalledseparatelyfromArcGIS.However,eachArcGISversionwillrequireaspecificversionofPython;giventhisrequirement,theinclusionofPythonwithintheArcGISinstallationpackageishelpful.Forthisbook,wewillbeusingArcGIS10.2,andthiswillrequirePython2.7.

OnaWindowsmachine,thePythonfolderstructureisplaceddirectlyontheC:drive,unlessitisexplicitlyloadedonanotherdrive.TheinstallationprocessforArcGIS10.2willcreateafolderatC:\Python27,whichwillcontainanotherfoldercalledeitherArcGIS10.2orArcGIS10.2x64,dependingontheoperatingsystemandtheversionofArcGISthathasbeeninstalled.Forthisbook,Iwillbeusingthe32-bitversionofArcGIS,sothefinalfolderpathwillbeatC:\Python27\ArcGIS10.2.

Withinthisfolderareanumberofsubfolders,aswellaspython.exe(thePythoninterpreter).Alsoincludedisasecondversionoftheinterpretercalledpythonw.exe.Pythonw.exewillexecuteascriptwithoutaterminalwindowwithprogramfeedbackappearing.Bothpython.exeandpythonw.execontaincompletecopiesofallPythoncommandsandcanbeusedtoexecuteascript.

WhichPythoninterpretershouldbeused?ThegeneralruletoexecuteascriptdirectlyusingthePythoninterpretersistousepythonw.exe,asnoterminalwindowwillappear.Whenthereisaneedtotestcodesnippets,ortoseetheoutputwithinaterminalwindow,startpython.exebydouble-clickingontheexecutable.

Whenpython.exeisstarted,aPythoninterpreterconsolewillappear:

Notethedistinctivethreechevrons(>>>)thatappearbelowtheheaderexplainingversioninformation.ThatisthePythonprompt,wherecodeisenteredtobeexecutedlinebyline,insteadofinacompletedscript.Thisdirectaccesstotheinterpreterisusefultotestcodesnippetsandunderstandsyntax.Aversionofthisinterpreter,thePythonWindow,hasbeenbuiltintoArcMapandArcCatalogsinceArcGIS10.Itwillbediscussedmoreinlaterchapters.

Howdoesthecomputerknowwheretheinterpreteris?TobeabletoexecutePythonscriptsdirectly(thatis,tomakethescriptsrunbydouble-clickingonthem),thecomputerwillalsoneedtoknowwheretheinterpretersitswithinitsfolderstructure.ToaccomplishthisrequiresbothadministrativeaccountaccessandadvancedknowledgeofhowWindowssearchesforaprogram.Wewillhavetoadjustanenvironmentvariablewithintheadvancedsystemsettingsdialoguetoregistertheinterpreterwiththesystempath.

OnaWindows7machine,clickonthestartmenuandright-clickonComputer,thenselectPropertiesfromthemenu.OnaWindows8machine,clickonWindowsexplorerandrightclickonThisPC,andselectPropertiesfromthemenu.ThesecommandsareshortcutstogettotheControlPanel’sSystemandSecurity/Systemmenus.SelectAdvancedsystemsettingsfromthepanelontheleft.ClickontheEnvironmentVariablesbuttonatthebottomoftheSystemPropertiesmenuthatappears.InthelowerportionoftheEnvironmentVariablesmenu,scrollthroughtheSystemvariableswindowuntilthePathvariableappears.Selectitbyclickingonit,andthenclickingontheEditbutton.Thefollowingwindowwillappear:

Thisvariablehastwocomponents:Variablename(path)andVariablevalue.Thevalueisaseriesoffolderpathsseparatedbysemicolons.ThisisthepaththatissearchedwhenWindowslooksforspecificexecutablesthathavebeenassociatedwithafileextension.Inourcase,wewillbeaddingthefolderpaththatcontainsthePythoninterpreter.TypeC:\Python27\ArcGIS10.2(ortheequivalentonyourmachine)intotheVariablevaluefield,makingsuretoseparateitfromthevaluebeforeitwithasemicolon.ClickonOKtoexittheEditdialogue,andOKtoexittheEnvironmentVariablesmenu,andOKtoexittheSystemPropertiesmenu.ThemachinewillnowknowwherethePythoninterpreteris,asitwillsearchallfolderscontainedwithinthePathvariabletolookforanexecutablecalledPython.Totestthatthepathadjustmentworkedcorrectly,openupacommandwindow(Startmenu/runcmd)andtypepython.Theinterpretershoulddirectlyruninthecommandwindow:

IfthePythonheaderwithversioninformationandthetriplechevronappears,thepathadjustmenthasworkedcorrectly.

NoteIfthereisnoadminaccessavailable,thereisaworkaround.Inacommand-linewindow,passtheentirepathtothePythoninterpreter(C:\Python27\ArcGIS10.2\python.exe)tostarttheinterpreter.

MakePythonscriptsexecutablewhenclickedonThefinalstepinmakingthescriptsrunwhendouble-clicked(whichalsomeanstheycanrunoutsideoftheArcGISenvironment,savinglotsofmemoryoverhead)istoassociatefileswiththe.pyextensionwiththePythoninterpreter.Ifthescriptshavenotalreadybeenassociatedwiththeinterpreter,theywillappearasfilesofanunknowntypeorasatextfile.

Tochangethis,right-clickonaPythonscript.SelectOpenWith,andthenselectChooseDefaultProgram.Ifpython.exeorpythonw.exedoesnotappearasachoice,navigatetothefolderthatholdsthem(C:\Python27\ArcGIS10.2,inthiscase)andselecteitherpython.exeorpythonw.exe.Again,thedifferencebetweenthetwoistheappearanceofaterminalwindowwhenthescriptsarerunusingpython.exe,whichwillcontainanyoutputfromthescript(butthiswindowwilldisappearwhenthescriptisdone).Irecommendusingpythonw.exewhenexecutingscripts,andpython.exetotestcode.

NotePythonscriptscanalsoexplicitlycallpythonw.exebyadjustingtheextensionto.pywinsteadof.py.

IntegratedDevelopmentEnvironments(IDEs)ThePythoninterpretercontainseverythingthatisneededtoexecuteaPythonscriptortotestPythoncodebyinteractingwiththePythoninterpreter.However,writingscriptsrequiresatexteditor.ThereareusuallyatleasttwosimpletexteditorsincludedonaWindowsmachine(NotepadandWordpad)andtheyworkinanemergencytoeditascriptorevenwriteawholescript.Unfortunately,theyareverysimpleanddonotallowtheuserfunctionalitythatwouldmakeiteasiertowritemultiplescriptsorverylongscripts.

Tobridgethegap,aseriesofprogramscollectivelyknownasIntegratedDevelopmentEnvironmentshavebeendeveloped.IDEsexistforallprogramminglanguages,andincludefunctionssuchasvariablelisting,codeassist,andmore,thatmakethemidealtocraftprogrammingscripts.WewillreviewafewofthemtoassesstheirusefulnesstowritePythonscripts.Thethreediscussedasfollowsareallfreeandwell-establishedwithindifferentPythoncommunities.

IDLEPythonincludesanIDEwhenitisinstalled.TheIDEiscalledIDLE,whichisawordplayonbothIDEandthenameofaprominentmemberofMontyPython,EricIdle.ItcanbestartedinWindows7bygoingtotheStartmenuandfindingtheArcGISfolderwithintheProgramsmenu.WithinthePythonfolder,IDLEwillbeoneofthechoiceswithinthatfolder.SelectittostartIDLE.

IDLEcontainsaninteractiveinterpreter(i.e.thetriplechevron)andtheabilitytoruncompletePythonscripts.ItisalsowrittenusingPython’sbuilt-inGUImodule,calledTkinter,soithastheadvantageofbeingwritteninthesamelanguagethatitexecutes.

AnotheradvantageofusingIDLEoverthePythonconsole(python.exe)isthatanyprintstatementsorotherscriptoutputisdirectedtotheIDLEinteractivewindow,whichdoesnotdisappearafterexecutingthescript.IDLEisalsolightweightwithrespecttomemoryuse.ScriptsareopenedusingafiledialoguecontainedwithintheFilemenu,andrecentlyrunscriptsarelistedwithintheFilemenu’s,RecentFiles.

DisadvantagesofIDLEincludealimitedcodeassist(orcodeauto-complete),ausefulIDEtool,andhavingnowaytoorganizescriptsintologicalprojects.Thereisnowaytofindallvariablescontainedwithinascript,anotherusefulfeatureofotherIDEs.Also,theRecentFilesmenuhasalimitonthenumberofscriptsthatitwilllist,makingithardertofindascriptthathasnotbeenruninmonths(whichisacommonoccurrence,believeme!).IDLEisapassableIDEthatisusefulifnootherprogramscanbeinstalledonthemachine.Itisalsoveryusefulforrapidtestingofcodesnippets.WhileitisnotmymainIDE,IfindmyselfusingIDLEalmostdaily.

PythonWinPythonWin(shortforPythonforWindows)isavailableathttp://sourceforge.net/projects/pywin32/files/pywin32,andincludesbothanIDEandhelpfulmodulestousePythoninaWindowsenvironment.SelectthenewestbuildofPythonWin,andthenselectthecorrectversion32modulebasedontheinstalledversionofPython(formymachine,Iselectedpywin32-218.win32-py2.7.exe,thecorrectversionformy32-bitPython2.7installation).Runtheexecutable,andifthecorrectversionhasbeendownloaded,theinstallationGUIwillrecognizePython2.7inthesystemregistryandwillinstallitself.

PythonWinincludesanInteractiveWindowwheretheusercandirectlyinteractwiththePythoninterpreter.ScriptscanalsobeopenedwithinPythonWin,anditincludesasetoftilingcommandsintheWindowsmenuthatallowstheusertoorganizethedisplayofallopenscriptsandtheInteractiveWindow.

AnotherniceadvantagethatPythonWinhasoverIDLEistheabilitytodisplaydifferentportionsofascriptwithinthesamescriptwindow.Ifascripthasgrowntoolong,itcanbeapaintoscrollupanddownthescriptwhenediting.PythonWinallowstheusertopulldownfromthetopofthescripttocreateasecondscriptwindow,whichcanfocusonaseparatepartofthescript.Also,ontheleftside,anotherwindowcanbeopenedthatwilllistPythonclassesandvariables,makingiteasiertonavigatetoaparticularsectionofthescript.

OnesmallbuthelpfulfeaturebuiltintoPythonWin’sInteractiveWindowistheabilityto

searchthroughpreviouslyenteredcodestatements.Atthetriplechevronprompt,holddowntheCtrlkeyandusetheupanddownarrowkeystonavigatethroughthelinestofindoneofinterest.Thissavesalotoftimewhentestingaparticularsnippetofcode.

Allinall,PythonWinisausefulandeasy-to-useIDE,andmostArcGISprofessionalswhocreatePythonscriptsusePythonWin.ThedrawbacksIfindwithPythonWinincludeitslackofabilitytoorganizescriptsintoprojects,anditslackofalistofvariablesthatexistwithinthescript,whichcanbeveryhelpfulwhennavigatinglargerscripts.

AptanaStudio3Sometimesthetoolsofthegreaterprogrammingcommunitycanseemdauntingtonewscripters,whoaremorefocusedonsimplycreatingascriptthatwillsavetimeonaGISanalysisthanusingthecorrecttoolforprogrammingdaily.Itremindsmeofinexperiencedcomputerusers,whodon’tfeelliketheyneedthefullpowerofatop-of-the-linecomputerbecausetheyonlywanttobrowsetheinternetandsende-mails.

However,theexactoppositeistrue:thecomputeradverseisbetteroffhavinganeasiertousetop-of-the-linecomputer,whileanexperiencedcomputerusercouldmakedowithanetbook.

Thesamecanbesaidforprogrammersandscripters.Sometimes,it’sbettertohaveanover-the-topIDEthatwillactuallymakeascriptermoreproductive,whileanexperiencedprogrammercouldmakedowithNotepad.AllofthebellsandwhistlesincludedinanIDEsuchasAptanaStudio3willsavescripterstimeandtakeremarkablylittletimetolearn.

AptanaStudio3isavailableathttp://aptana.com.Downloadandruntheinstallerprovidedtoinstallit.Chooseadefaultmainprojectfolderthatcancontainallofthescriptsprojects;forthisbook,IcreatedafoldercalledC:\Projects.Foreachprojectcreated,Aptanawillcreateaprojectfileholdinginformationabouteachproject.WhenusingAptanaStudioatwork,usinganetworkfoldercanbeusefulasotherscanthenaccesstheprojectswiththeirrespectiveAptanainstallations.

Onceithasbeeninstalled,thenextstepistocreateaPyDevproject.GototheFilemenuandselectNew,andthenselectPyDevproject.Whencreatingthisfirstproject,PythonInterpreterwillhavetobeaddedtoAptana’sPythonpath.Aptanacansupportmorethanoneinterpreter;forourpurposes,onewilldo.GotothebottomofthePyDevprojectmenuandclickonClickheretoconfigureaninterpreter.WhenthePreferences/PythonInterpretersmenuappears,makesuretoselectInterpreter-Pythonontheleft,andthenclickonNewinthetop-rightmenu.

OnceNewhasbeenselected,asmalldialogwillappearaskingforanamefortheinterpreterandthepathtotheexecutable.Clickonbrowseandnavigatetothefolderwithpython.exe.NoterminalwindowwillbegeneratedwhenrunningaPythonscriptusingAptanaStudioasalloutputisredirectedtotheAptanaStudioconsole.Selectpython.exeandclickonOK.Next,clickonOKintheSelectInterpretermenu,andthenclickonOKinthePreferencesmenu.BackinthePyDevProjectmenu,givetheprojectaname,andeitherusethedefaultworkspacelocationoracustomone(forexample,C:\Projects).

Allofthisconfigurationonlyhastohappenthefirsttime;oncethatisdone,creatingaPyDevprojectwillonlyrequiregivinganameandlocation.Now,allofthescriptsassociatedwiththatprojectwillalwaysbelistedintheleftmenu(PyDevPackageExplorer),whichisaverypowerfulwaytoorganizeprojectsandscripts.

MakingsurethatAptanaStudioisinthePyDevperspective(intheWindows/OpenPerspective/Othermenu,choosePyDev)willgivethreemainwindows–PackageExplorerontheleft,Scriptwindowinthemiddle,andOutlinewindowontheright,wherevariablescontainedwithinascriptarelisted.Clickingononeofthevariablesontherightwillmovethescriptwindowtothatsectionofthecode,makingscriptnavigation

fast.Also,IliketoaddtheConsolewindowinthemiddlebelowtheScriptwindow,wheretheoutputofthescriptcanbedisplayed.

OpenscriptseachhaveatabwithintheScriptwindow,makingiteasytoswitchbetweenthescripts.Also,thewindowscanbeclosedtogivemoreroomtotheScriptwindowasneeded.Hoveringoveravariablewithinascriptwillcallupapop-upmenuthatdescribeswherethevariablewasfirstcreated,whichcanbealifesaverasitiseasytoforgetattimeswhichvariableiswhich(unless,ofcourse,theyareclearlynamedaccordingtotherulesdescribedinthepreviouschapter;eventhen,itcanbeapainattimes).

IDEsummaryTherearemanyotherIDEs,bothcommercialandfree,availableforcodinginPython.Intheend,eachGISanalystmustchoosethetoolthatmakesthemfeelproductiveandcomfortable.Thismaychangeasprogrammingbecomesabiggerpartoftheirdailyworkflow.BesuretotestoutafewdifferentIDEstofindonethatiseasytouseandintuitive.

PythonfolderstructurePython’sfolderstructureholdsmorethanjustthePythonInterpreter.Withinthesubfoldersresideanumberofimportantscripts,digitallinklibraries,andevenClanguagemodules.Notallofthescriptsareusedallthetime,buteachhasaroleinmakingthePythonprogrammingenvironmentpossible.Themostimportantfoldertoknowaboutisthesite-packagesfolder,wheremostmodulesthatwillbeimportedinPythonscriptsarecontained.

WheremodulesresideWithineveryPythonfolderisafoldercalledLib,andwithinthatfolderisafoldercalledsite-packages.Onmymachine,thefoldersitsatC:\Python27\ArcGIS10.2\Lib\site-packages.Almostallthird-partymodulesarecopiedintothisfoldertobeimportedasneeded.Themainexceptiontothisrule,forourpurposes,istheArcPymodule,whichisstoredwithintheArcGISfolderintheProgramFilesfolder(forexample,C:\ProgramFiles(x86)\ArcGIS\Desktop10.2\arcpy).Tomakethatpossible,theArcGISinstalleradjuststhePythonsystempath(usingthesysmodule)tomakethearcPymoduleimportable.

UsingPython’ssysmoduletoaddamodulePython’ssysmoduleisamodulethatallowstheusertotakeadvantageofsystemtoolsbuiltintothePythonInterpreter.Oneofthemostusefulofthefunctionsinthesysmoduleissys.path.Itisalistoffilepaths,whichtheusercanmodifytoadjustwherePythonwilllookforamoduletoimport,withoutneedingadministrativeaccess.

WhenPython2.7isinstalledbytheArcGIS10.2installer,theinstallertakesadvantageofthesys.pathfunctionstoaddC:\ProgramFiles(x86)\ArcGIS\Desktop10.2\arcpytothesystempath.Totestthis,startthePythonInterpreteroranIDEandtypethefollowing:

>>>importsys

>>>printsys.path

Theoutputisasfollows:

['','C:\\WINDOWS\\SYSTEM32\\python27.zip',

'C:\\Python27\\ArcGIS10.2\\Dlls','C:\\Python27\\ArcGIS10.2\\lib',

'C:\\Python27\\ArcGIS10.2\\lib\\plat-win',

'C:\\Python27\\ArcGIS10.2\\lib\\lib-tk',

'C:\\Python27\\ArcGIS10.2\\Lib\\site-packages\\pythonwin',

'C:\\Python27\\ArcGIS10.2','C:\\Python27\\ArcGIS10.2\\lib\\site-packages',

'C:\\ProgramFiles(x86)\\ArcGIS\\Desktop10.2\\bin','C:\\ProgramFiles

(x86)\\ArcGIS\\Desktop10.2\\arcpy','C:\\ProgramFiles

(x86)\\ArcGIS\\Desktop10.2\\ArcToolbox\\Scripts',

'C:\\Python27\\ArcGIS10.2\\lib\\site-packages\\win32',

'C:\\Python27\\ArcGIS10.2\\lib\\site-packages\\win32\\lib']

Thesystempath(storedinthevariablesys.path)includesallofthefoldersthatArcPyrequirestoautomateArcGIS.ThesystempathincorporatesalldirectorieslistedinthePYTHONPATHenvironmentvariable(ifonehasbeencreated);thisisseparatefromtheWindowspathenvironmentvariablediscussedpreviously.ThetwoseparatepathvariablesworktogethertohelpPythonlocatemodules.

Thesys.path.append()methodThesys.pathfunctionisalist(didyounoticethesquarebracketsintheprecedingcodeoutput?)andassuchcanbeappendedorextendedtoincludenewfilepathsthatwillpointtomodulestheuserwantstoimport.Toavoidtheneedtoadjustsys.path,copythemoduleintothesite-packagesfolder.Whenthisisnotpossible,usethesys.path.append()methodinstead:

>>>sys.path.append("C:\\Projects\\Requests")

>>>sys.path

['','C:\\WINDOWS\\SYSTEM32\\python27.zip',

'C:\\Python27\\ArcGIS10.2\\Dells',

'C:\\Python27\\ArcGIS10.2\\lib',

..'C:\\Python27\\ArcGIS10.2\\lib\\plat-win',

..'C:\\Python27\\ArcGIS10.2\\lib\\lib-tk',

..'C:\\Python27\\ArcGIS10.2\\Lib\\site-packages\\pythonwin',

..'C:\\Python27\\ArcGIS10.2',..'C:\\Python27\\ArcGIS10.2\\lib\\site-

packages','C:\\Program

..Files(x86)\\ArcGIS\\Desktop10.2\\bin','C:\\ProgramFiles

..(x86)\\ArcGIS\\Desktop10.2\\arcpy','C:\\ProgramFiles

..(x86)\\ArcGIS\\Desktop10.2\\ArcToolbox\\Scripts',

..'C:\\Python27\\ArcGIS10.2\\lib\\site-packages\\win32',

..'C:\\Python27\\ArcGIS10.2\\lib\\site-packages\\win32\\lib',

..'C:\\Projects\\Requests']

Whenthesys.path.append()methodisused,theadjustmentistemporary.AdjustthePYTHONPATHenvironmentvariableintheWindowsSystemPropertiesmenu(discussedinthepathenvironmentvariablesection)tomakeapermanentchange(andcreatethePYTHONPATHifithasnotbeencreated).

Onelastnoteisthattoimportamodulewithoutadjustingthesystempathorcopyingthemoduleintothesite-packagesfolder,placethemoduleinthefolderwiththescriptthatisimportingit.Aslongasthemoduleisconfiguredcorrectly,itwillworknormally.Thisisusefulwhenthereisnoadministrativeaccessavailabletoamachine.

SummaryInthischapter,wecoveredalotabouthowPythonworkstoexecutescriptsandcommands,andaboutdevelopmentenvironmentsusedtocraftscripts.Inparticular,wediscussedhowaPythonscriptisreadandexecutedbythePythonInterpreter,wherethePythonInterpreterislocatedwithinthePythonfolderstructure,andwhatthedifferentPythonscriptextensionsmean(.py,.pyc,.pyw).WealsocoveredIntegratedDevelopmentEnvironmentsandhowtheycompareandcontrast.

Inthenextchapter,wewillcoverhowtouseModelBuildertoconvertamodeledanalysisintoaPythonscript,andhowtomakeitmorepowerfulthantheexportedversion.

Chapter3.CreatingtheFirstPythonScriptNowthatwehavePythonconfiguredtofitourneeds,wecancreatePythonscripts.ThischapterwillexplorehowtouseArcGISModelBuildertomodelasimpleanalysisasthebasisforourscript.ModelBuilderisveryusefulonitsownandforcreatingPythonscriptsasithasanoperationalandavisualcomponent,andallmodelscanbeoutputtedasPythonscripts.ThiswillallowustocomparehowthemorefamiliarModelBuilderutilizestoolsintheArcToolboxtohowPythonhandlesthesametools.WewillalsodiscussiterationandwhenitisbesttousePythonoverModelBuilder.

Inthischapter,wewillcoverthefollowingtopics:

ModelingasimpleanalysisusingModelBuilderExportingthemodelouttoaPythonscript

Prerequisites“AlongwithArcGISModelBuilder,adatasetandscriptsarerequired.”

Forthischapter,theaccompanyingdataandscriptsshouldbedownloadedfromPacktPublishing’swebsite.Thecompletedscriptsareavailableforcomparisonpurposesandthedatawillbeusedforthischapter’sanalysis.

ModelBuilderArcGIShasbeenindevelopmentsincethe1970s.Duringthattime,itincludedavarietyofprogramminglanguagesandtoolstohelpGISanalystsautomateanalysesandmapproduction.TheseincludetheAvenuescriptinglanguageintheArcGIS3xseriesandtheARCMacroLanguage(AML)intheARC/Infoworkstationdays,aswellasVBScriptupuntilArcGIS10xwhenPythonwasintroduced.AnotherusefultoolintroducedinArcGIS9xwasModelBuilder,avisualprogrammingenvironmentusedforbothmodelinganalysisandcreatingtoolsthatcanbeusedrepeatedlywithdifferentinputfeatureclasses.

AnotherusefulfeatureofModelBuilderisanexportfunctionthatallowsmodelerstocreatePythonscriptsdirectlyfromamodel.ThiswillmakeiteasiertocomparehowinputsinaModelBuildertoolareacceptedversushowaPythonscriptcallsthesametoolandsuppliestheinputstoit,orhowthefeatureclassesthatarecreatedarenamedandplacedwithinthefilestructure.ModelBuilderisafantastictoolthatwillmakeiteasyforaGISanalysttobridgethegapfromnormalGISworkflowstoautomatedPython-basedworkflows.

CreatingamodelandexportingtoPythonThischapterwilldependonthedownloadableSanFrancisco.gdbfilegeodatabase,availablefromthePacktPublishingwebsite.TheSanFranciscoGDBcontainsdatadownloadedfromdata.sfgov.organdtheUSCensus’AmericanFactfinderwebsiteavailableatfactfinder2.census.gov.Allcensusandgeographicdataincludedinthegeodatabaseisfromthe2010census.ThedataiscontainedwithinafeaturedatasetcalledSanFrancisco.ThedatainthisfeaturedatasetisinNAD83CaliforniaStatePlaneZone3andthelinearunitofmeasureistheUSFoot(thiscorrespondstoSRID2227intheEuropeanPetroleumSurveyGroup,orEPSG,format).

Theanalysiswewillcreatewiththemodel,andeventuallyexporttoPythonforfurtherrefinement,willusebusstopsalongaspecificlineinSanFrancisco.Thesebusstopswillbebufferedtocreatearepresentativeregionaroundeachbusstop.Thebufferedareaswillthenbeintersectedwithcensusblockstofindouthowmanypeoplearewithineachrepresentativeregionaroundthebusstops.

ModelingtheSelectandBuffertoolsUsingModelBuilder,wewillfirstmodelthebasisofthebusstopanalysis.Onceithasbeenmodeled,itwillbeexportedasanautomaticallygeneratedPythonscript.Followthesestepstobegintheanalysis:

1. OpenupArcCatalogandcreateafolderconnectiontothefoldercontainingSanFrancisco.gdb.Right-clickongeodatabaseandaddanewtoolboxcalledChapter3Tools.

2. Next,openModelBuilderandcreateaModel,savingitintheChapter3ToolstoolboxasChapter3Model1.

3. DragtheBus_StopsfeatureclassandtheSelecttoolfromtheAnalysis/ExtracttoolsetinArcToolbox.

4. OpentheSelecttoolandnametheoutputfeatureclassInbound71.MakesurethatthefeatureclassiswrittentotheChapter3Resultsfeaturedatasetintothemodel.

5. OpentheExpressionSQLQueryBuilderandcreatethefollowingSQLexpression:NAME=‘71IB’ANDBUS_SIGNAG=‘FerryPlaza’.

6. ThenextstepistoaddaBuffertoolfromtheAnalysis/Proximitytoolset.TheBuffertoolwillbeusedtocreatebuffersaroundeachbusstop.Thebufferedbusstopsallowustointersectwithcensusdataintheformofcensusblocks,creatingtherepresentativeregionsaroundeachbusstop.

7. ConnecttheoutputoftheSelecttool(Inbound71)totheBuffertool.OpenuptheBuffertoolandadd400totheDistancefield,andmaketheunitsFeet.Leavetherestoftheoptionsblank.ClickonOKandreturntothemodel.

AddingtheIntersecttoolNowthatwehaveselectedthebuslineofinterest,andbufferedthestopstocreaterepresentativeregions,wewillneedtointersecttheregionswiththecensusblockstofindthepopulationofeachrepresentativeregion:

1. First,addtheCensusBlocks2010featureclassfromtheSanFranciscofeaturedatasettothemodel.

2. Next,addtheIntersecttool,locatedintheAnalysis/OverlaytoolsetinArcToolbox.WhilewecoulduseSpatialJointoachieveasimilarresult,IamusingtheIntersecttooltocapturetheareaofintersectforuselaterinthemodelandscript.

Atthispoint,ourmodelshouldlooklikethis:

TallyingtheanalysisresultsAfterwecreatedthissimpleanalysis,thenextstepistodeterminetheresultsforeachbusstop.Findingthenumberofpeoplethatliveincensusblockstouchedbythe400feetbufferofeachbusstopinvolvesexaminingeachrowofdatainthefinalfeatureclassandselectingrowsthatcorrespondtothebusstop.Oncetheseareselected,asumoftheselectedrowswouldbecalculatedeitherusingtheFieldCalculatorortheSummarizetool.Allofthesemethodswillwork,andyetnoneareperfect.Theytaketoolong,andworse,arenotrepeatableautomaticallyifanassumptioninthemodelisadjusted(ifthebufferisadjustedfrom400feetto500feet,forinstance).

ThisiswherethetraditionalusesofModelBuilderbegintofailanalysts.Itshouldbeeasytoinstructthemodeltoselectallrowsassociatedwitheachbusstop,andthengenerateasummedpopulationfigureforeachbusstop’srepresentativeregion.Itwouldbeevenbettertohavethemodelcreateaspreadsheettocontainthefinalresultsoftheanalysis.It’stimetousePythontotakethisanalysistothenextlevel.

ExportingthemodelandadjustingthescriptWhilemodelinganalysisinModelBuilderhasitsdrawbacks,thereisonefantasticoptionbuiltintoModelBuilder;theabilitytocreateamodelandthenexportthemodeltoPython.AlongwiththeArcGIShelpdocumentation,itisthebestwaytodiscoverthecorrectPythonsyntaxtousewhenwritingArcPyscripts.

CreateafolderthatcanholdtheexportedscriptsnexttotheSanFranciscogeodatabase(forexample,C:\Projects\Scripts).ThiswillholdboththeexportedscriptsthatArcGISautomaticallygenerates,andtheversionsthatwewillbuildfromthosegeneratedscripts.

OpenthemodelcalledChapter3Model1andclickontheModelmenuintheupperleft.SelectExportfromthemenu,andthenselectToPythonScript.SavethescriptinthescriptfolderasChapter3Model1.py.

NoteNotethatthereisalsotheoptiontoexportthemodelasagraphic.Creatingagraphicofthemodelisagoodwaytosharewhatthemodelisdoingwithotheranalystswithouttheneedtosharethemodelandthedata,andcanalsobeusefulwhensharingPythonscriptsaswell.

TheautomaticallygeneratedscriptOpentheautomaticallygeneratedscriptinanIDE.Itshouldlooklikethis:

#-*-coding:utf-8-*-

#-------------------------------------------------------------------------

--

#8662_Chapter3Model1.py

#Createdon:2014-04-2221:59:31.00000

#(generatedbyArcGIS/ModelBuilder)

#Description:

#-------------------------------------------------------------------------

--

#Importarcpymodule

importarcpy

#Localvariables:

Bus_Stops="C:\\Projects\\PacktDB.gdb\\SanFrancisco\\Bus_Stops"

CensusBlocks2010=

"C:\\Projects\\PacktDB.gdb\\SanFrancisco\\CensusBlocks2010"

Inbound71="C:\\Projects\\PacktDB.gdb\\Chapter3Results\\Inbound71"

Inbound71_400ft_buffer=

"C:\\Projects\\PacktDB.gdb\\Chapter3Results\\Inbound71_400ft_buffer"

Intersect71Census=

"C:\\Projects\\PacktDB.gdb\\Chapter3Results\\Intersect71Census"

#Process:Select

arcpy.Select_analysis(Bus_Stops,

Inbound71,

"NAME='71IB'ANDBUS_SIGNAG='FerryPlaza'")

#Process:Buffer

arcpy.Buffer_analysis(Inbound71,

Inbound71_400ft_buffer,

"400Feet","FULL","ROUND","NONE","")

#Process:Intersect

arcpy.Intersect_analysis("C:\\Projects\\PacktDB.gdb\\Chapter3Results\\Inbou

nd71_400ft_buffer

#;C:\\Projects\\PacktDB.gdb\\SanFrancisco\\CensusBlocks2010#",

Intersect71Census,"ALL","","INPUT")

Let’sexaminethisscriptlinebyline.Thefirstlineisprecededbyapoundsign(#),whichagainmeansthatthislineisacomment;however,itisnotignoredbythePythoninterpreterwhenthescriptisexecutedasusualbutisusedtohelpPythoninterprettheencodingofthescriptasdescribedhere:http://legacy.python.org/dev/peps/pep-0263.

Thesecondcommentedlineandthethirdlineareincludedfordecorativepurposes.Thenextfourlines,allcommented,areusedtoprovidereaderswithinformationaboutthescript,whatitiscalledandwhenitwascreated,alongwithadescriptionthatispulledfromthemodel’sproperties.Anotherdecorativelineisincludedtoseparateouttheinformativeheaderfromthebodyofthescriptvisually.Whilethecommentedinformationsectionisnicetoincludeinascriptforotherusersofthescript,itisnotnecessary.

Thebodyofthescript,ortheexecutableportionofthescript,startswiththeimportarcpyline.Importstatementsare,byconvention,includedatthetopofthebodyofthescript.Inthisinstance,theonlymodulethatisbeingimportedisArcPy.

ModelBuilder’sexportfunctioncreatesnotonlyanexecutablescript,butalsocommentseachsectiontohelpmarkthedifferentsectionsofthescript.ThecommentslettheuserknowwherethevariablesarelocatedandwheretheArcToolboxtoolsarebeingexecuted.Thecommentswillgrowtobesuperfluousasthereadergrowstounderstandthecode,butitwasniceofESRItoincludethecomments.

Belowtheimportstatementsarethevariables.Inthiscase,thevariablesrepresentthefilepathstotheinputandoutputfeatureclasses.Thevariablenamesarederivedfromthenamesofthefeatureclasses(thebasenamesofthefilepaths).Thefilepathsareassignedtothevariablesusingtheassignmentoperator(=),andthepartsofthefilepathsareseparatedbytwobackslashes.

FilepathsinPythonItwouldbegoodtoreviewhowfilepathsareusedinPythoncomparedtohowtheyarerepresentedinWindows.InPython,filepathsarestrings,andstringsinPythonhavespecialcharactersusedtorepresenttabs(\t),newlines(\n),orcarriagereturns(\r),amongmanyothers.Thesespecialcharactersallincorporatesinglebackslashes,makingitveryhardtocreateafilepaththatusessinglebackslashes.Thiswouldnotbeabigdeal,exceptthatfilepathsinWindowsExplorerallusesinglebackslashes.

Thereareanumberofmethodsusedtoavoidthisissue.PythonwasdevelopedwithintheLinuxenvironment,wherefilepathshaveforwardslashes.ThismorePythonicrepresentationisalsoavailablewhenusingPythoninaWindowsenvironment,demonstratedasfollows:

WindowsExplorer:

"C:\Projects\PacktDB.gdb\Chapter3Results\Intersect71Census"

Pythonicversion:

"C:/Projects/PacktDB.gdb/Chapter3Results/Intersect71Census"

WithinaPythonscript,thefilepathwiththeforwardslasheswillwork,whiletheWindowsExplorerversionmightcausethescripttothrowanexception.

AnothermethodusedtoavoidtheissuewithspecialcharactersistheoneemployedbyModelBuilderwhenitautomaticallycreatesthePythonscriptsfromamodel.Inthiscase,thebackslashesareescapedusingasecondbackslash.Theprecedingscriptusesthissecondmethodtoproducethefollowingresults:

Pythonescapedversion:

"C:\\Projects\\PacktDB.gdb\\Chapter3Results\\Intersect71Census"

Thethirdmethod,whichIprefer,istocreatewhatisknownasarawstring.Thisisthesameasaregularstring,butitincludesanrbeforethescriptbegins.ThisralertsthePythonInterpreterthatthefollowingscriptdoesnotcontainanyspecialcharactersorescapecharacters.Hereisanexampleofhowitisused:

Pythonrawstring:

r"C:\Projects\PacktDB.gdb\Chapter3Results\Intersect71Census"

UsingrawstringswillmakeiteasiertograbafilepathfromWindowsExplorerandaddittoastringinsideascript.Itwillalsomakeiteasiertoavoidaccidentallyforgettingtoincludeasetofdoublebackslashesinafilepath,whichhappensallthetimeandisthecauseofmanyscriptbugs.

Continuingthescriptanalysis:theArcPytoolsThenext,andmostimportant,sectionofthescriptiswheretheanalysisisexecuted.Thesametoolsthatwecreatedinthemodel,theSelect,theBuffer,andtheIntersecttools,areincludedinthissection.Thesameparametersthatwesuppliedinthemodelarealsoincludedhere:theinputsandoutputs,plustheSQLstatementintheSelecttool,andthebufferdistanceintheBuffertool.

Thetoolparametersaresuppliedtothetoolsinthescriptinthesameorderastheyappearinthetoolinterfacesinthemodel.HereistheSelecttoolinthescript:

arcpy.Select_analysis(Bus_Stops,Inbound71,"NAME='71IB'ANDBUS_SIGNAG

='FerryPlaza'")

Itworkslikethis.ThearcPymodulehasamethod,oraspecialproperty,calledSelect_analysis.Thismethod,whencalled,requiresthreeparameters:theinputfeatureclass(orshapefile),theoutputfeatureclass,andtheSQLstatement.Inthisexample,theinputisrepresentedbythevariableBus_StopsandtheoutputfeatureclassisrepresentedbythevariableInbound71,bothofwhicharedefinedinthevariablesection.TheSQLstatementisincludedasthethirdparameter.Notethatitcouldalsoberepresentedbyavariable,ifthevariablewasdefinedabovethisline;theSQLstatement,asastring,couldbeassignedtoavariableandthevariablecouldreplacetheSQLstatementasthethirdparameter.Hereisanexampleofparameterreplacementusingavariable:

sqlStatement="NAME='71IB'ANDBUS_SIGNAG='FerryPlaza'"

arcpy.Select_analysis(Bus_Stops,Inbound71,sqlStatement)

WhileModelBuilderisgoodaboutassigninginputandoutputfeatureclassestovariables,itdoesnotassignvariablestoeveryportionoftheparameter.Thiswillbeanimportantthingtocorrectwhenweadjustandbuildourownscripts.

TheBuffertoolacceptsasimilarsetofparametersastheSelecttool.Thereisaninputfeatureclassrepresentedbyavariable,anoutputfeatureclassvariable,andthedistancethatweprovided(400feetinthiscase),alongwithaseriesofparametersthataresuppliedbydefault.Notethattheparametersrelyonkeywords,andthesekeywordscanbeadjustedwithinthetextofthescripttoadjusttheresultingbufferoutput.Forinstance,FeetcouldbeadjustedtoMetersandthebufferwouldmuchlarger.CheckthehelpsectionofthetooltobetterunderstandhowtheotherparameterswillaffectthebufferandtofindthekeywordsargumentsthatwillbeacceptedbytheBuffertoolinArcPy.Also,asnotedearlier,alloftheparameterscouldbeassignedtovariables,whichcansavetimeifthesameparametersareusedrepeatedlythroughoutascript.

Sometimesthesuppliedparameterismerelyanemptystring,asisthecaseherewiththelastparameter:

arcpy.Buffer_analysis(Inbound71,Inbound71_400ft_buffer,

"400Feet","FULL","ROUND","NONE","")

Theemptystring,whichinthiscasesignifiesthatthereisnotadissolvefieldforthisbuffer,isfoundquitefrequentlywithinArcPy.Itcouldalsoberepresentedbytwosinglequotes,butModelBuilderhasbeenbuilttousedoublequotestoencasestrings.

TheIntersecttoolandstringmanipulationThelasttool,theIntersecttool,usesadifferentmethodtorepresentthefilesthatneedtobeintersectedtogetherwhenthetoolisexecuted.Becausethetoolacceptsmultiplefilesintheinputsection(meaningthereisnolimittothenumberoffilesthatcanbeintersectedtogetherinoneoperation),itstoresallofthefilepathswithinonestring.Thestringusesthehashorpoundsign(#)toseparatethefilepathswithintheinputstring.ThisslightdeviationmustbedealtwithifwearetousetheIntersecttoolinaScripttool.Ifwearebuildingatoolfromthisscript,wewillnotknowthefilesthatwillbeintersectedbeforetheyarerun,soweneedtoknowthemethodstodealwithinsertingvariablesintostrings.

Therearethreemethodstoinsertvariablesintostrings.Eachmethodhasdifferentadvantagesanddisadvantagesofatechnicalnature.It’sgoodtoknowaboutallthreeofthemastheyhaveusesbeyondourneedshere,solet’sreviewthem.

Thestringmanipulationmethod1–stringadditionStringadditionisanoddconceptatfirstasitwouldnotseempossibletoaddstringstogether,unlikeintegersorfloats,whicharenumbers.However,withinPythonandotherprogramminglanguages,thisisanormalstep.Usingtheplussign(+),stringsareaddedtogethertomakelongerstringsorallowvariablestobeaddedtothemiddleofexistingstrings.Herearesomeexamplesofthisprocess:

>>>aString="Thisisastring"

>>>bString="andthisisanotherstring"

>>>aString+bString

Theoutputisasfollows:

'Thisisastringandthisisanotherstring'

>>>cString=aString+bString

>>>cString

Theoutputisasfollows:

'Thisisastringandthisisanotherstring'

Twoormorestringscanbeaddedtogether,andcanevenbeassignedtoathirdvariable.ThisprocesscanbeusefulforsituationssuchastheinputstringfortheIntersecttool.Thestringcanbebrokenupandvariablesrepresentingthefilepathscanbeinsertedintothemiddleofthestring:

filePath1=r"C:\Projects\Inbound71_400ft_buffer"

filePath2=r"C:\Projects\CensusBlocks2010"

arcpy.Intersect_analysis(filePath1+"#;"+filePath2+"#",

Intersect71Census,"ALL","","INPUT")

Thisisapowerfulandusefulwaytoinsertthefilepathsintotheinputstring.Aslongastheseparatorsarestillincludedinthestring,thestringwillstillbevalidandtheIntersecttoolwillrunasexpected.Hereiswhatthestringwilllooklikewhenthestringadditioniscompleted:

>>>filePath1=r"C:\Projects\Inbound71_400ft_buffer"

>>>filePath2=r"C:\Projects\CensusBlocks2010"

>>>inputString=filePath1+"#;"+filePath2+"#"

>>>printinputString

Theoutputisasfollows:

C:\Projects\Inbound71_400ft_buffer#;C:\Projects\CensusBlocks2010#

Anothersimilaroffshootofstringadditionisstringmultiplication,wherestringsaremultipliedbyanintegertoproducerepeatedversionsofthestring:

>>>"string"*3

Theoutputisasfollows:

'stringstringstring'

Thestringmanipulationmethod2–stringformatting#1Thesecondmethodofstringmanipulation,knownasstringformatting,involvesaddingplaceholdersintothestringthatwillacceptspecifickindsofdata.Thismeansthatthesespecialstringscanacceptotherstringsaswellasintegersandfloatvalues.Theseplaceholdersusethemodulo(%)andakeylettertoindicatethetypeofdatatoexpect.Stringsarerepresentedusing%s,floatsarerepresentedusing%f,andintegersarerepresentedusing%d.Thefloatscanalsobeadjustedtolimitthedigitsincludedbyaddingamodifyingnumberafterthemodulo.Ifthereismorethanoneplaceholderinastring,thevaluesarepassedtothestringinatuple.

ThismethodhasbecomelesspopularsincethethirdmethoddiscussedinthefollowingsectionwasintroducedinPython2.6,butitisstillvaluabletoknowasmanyolderscriptsuseit.Hereisanexampleofthismethod:

>>>origString="Thisstringhasasaplaceholder%s"

>>>newString=origString%"andthistextwasadded"

>>>printnewString

Theoutputisasfollows:

Thisstringhasasaplaceholderandthistextwasadded

Hereisanexamplewhenusingafloatplaceholder:

>>>floatString1="Thisstringhasafloathere:%f"

>>>newString=floatString1%1.0

>>>printnewString

Theoutputisasfollows:

Thisstringhasafloathere:1.000000

>>>floatString2="Thisstringhasafloathere:%.1f"

>>>newString2=floatString2%1.0

>>>printnewString2

Theoutputisasfollows:

Thisstringhasafloathere:1.0

Hereisanexampleusinganintegerplaceholder:

>>>intString="Hereisaninteger:%d"

>>>newString=intString%1

>>>printnewString

Theoutputisasfollows:

Hereisaninteger:1

FortheIntersecttool,the%ssymbolcanbeusedtoacceptthefilepathstringvariables:

filePath1=r"C:\Projects\Inbound71_400ft_buffer"

filePath2=r"C:\Projects\CensusBlocks2010"

arcpy.Intersect_analysis("%s#;%s#"%(filePath1,filePath2),

Intersect71Census,"ALL","","INPUT")

Thestringmanipulationmethod3–stringformatting#2Thefinalmethod,themostrecentlyintroduced,isalsoknownasstringformatting.Itissimilartothestringformattingdiscussedearlier,withtheaddedbenefitofnotrequiringaspecifictypeofplaceholder.Theplaceholders,ortokensastheyarealsoknown,areonlyrequiredtobeinordertobeaccepted.Theformatfunctionisbuiltintostrings;byadding.formattothestring,andpassinginparameters,thestringacceptsthevalues:

>>>formatString="Thisstringhas3tokens:{0},{1},{2}"

>>>newString=formatString.format("String",2.5,4)

>>>printnewString

Theoutputisasfollows:

Thisstringhas3tokens:String,2.5,4

Thetokensdon’thavetobeinorderwithinthestring,andcanevenberepeated.Theorderisderivedfromtheparameterssuppliedtothe.formatfunctionthatpassesthevaluestothestring.

FortheIntersecttool,thestringformattingwouldlooklikethis:

filePath1=r"C:\Projects\Inbound71_400ft_buffer"

filePath2=r"C:\Projects\CensusBlocks2010"

arcpy.Intersect_analysis("{0}#;{1}#".format(filePath1,filePath2),

Intersect71Census,"ALL","","INPUT")

Thethirdmethodhasbecomemygo-tomethodforstringmanipulationbecauseoftheabilitytoaddthevaluesrepeatedlyandmakeitpossibletoavoidsupplyingthewrongtypeofdatatoaspecificplaceholder,unlikethesecondmethod.

AdjustingtheScriptNowisthetimetotaketheautomaticallygeneratedscriptandadjustittofitourneeds.Wewantthescripttobothproducetheoutputdata,andtohaveitanalyzethedataandtallytheresultsintoaspreadsheet.Thisspreadsheetwillholdanaveragedpopulationvalueforeachbusstop.Theaveragewillbederivedfromeachcensusblockthatthebufferedrepresentativeregionsurroundingthestopsintersected.SavetheoriginalscriptasChapter3Model1Modified.py.

AddingtheCSVmoduletothescriptForthisscript,wewillusetheCSVmodule,ausefulmoduletocreateCommaSeparatedValuespreadsheets.Itssimplesyntaxwillmakeitausefultooltocreatescriptoutputs.ItshouldbenotedthatArcGISforDesktopalsoinstallsthexlrdandxlwtmodules,usedtoreadorgenerateExcelspreadsheetsrespectively,whenitisinstalled.

JustbelowtheimportarcPyline,addimportcsv.Thiswillallowustousethecsvmoduletocreatethespreadsheet:

#Importarcpymodule

importarcpy

importcsv

ThenextadjustmentismadetotheIntersecttool.Noticethatthetwopathsincludedintheinputstringarealsodefinedasvariablesinthevariablesection.Removethefilepathsfromtheinputstringsandreplacethemwithnumberedplaceholdertokens,andthenaddtheformatfunctionandsupplythevariablesasplaceholders:

#Process:Intersect

arcpy.Intersect_analysis("{0}#;{1}#".format(..............

Inbound71_400ft_buffer,CensusBlocks2010),

Intersect71Census,"ALL","","INPUT")

Accessingthedata:UsingacursorNowthatthescriptisinplacetogeneratetherawdataweneed,weneedawaytoaccessthedataheldintheoutputfeatureclassfromtheIntersecttool.Thisaccesswillallowustoaggregatetherowsofdatarepresentingeachbusstop.Wealsoneedsomethingtoholdtheaggregatedatainthememory,tobewrittentothespreadsheet.

Toaccomplishthesecondpart,wewilluseaPythondictionary.Toaccomplishthefirstpart,wewilluseamethodbuiltintotheArcPymodule:theDataAccessSearchCursor.

ThePythondictionarywillbeaddedbelowtheIntersecttool.AdictionaryinPythoniscreatedusingcurlybrackets.Addthefollowinglinetothescript:

dataDictionary={}

ThisscriptwillusetheBusStopIDsaskeysforthedictionary.Thevalueswillbelists,whichwillholdallofthepopulationvaluesassociatedwitheachBusStopID.AddthefollowinglinestogenerateaDataCursor:

witharcpy.da.SearchCursor(Intersect71Census,["STOPID","POP10"])as

cursor:

forrowincursor:

busStopID=row[0]

pop10=row[1]

ifbusStopIDnotindataDictionary.keys():

dataDictionary[busStopID]=[pop10]

else:

dataDictionary[busStopID].append(pop10)

ThisiterationcombinesafewideasinPythonandArcPy.Thewith…asstatementisusedtocreateavariable(cursor)thatrepresentsthearcpy.da.SearchCursorobject.Itcouldalsobewrittenlikethis:

cursor=arcpy.da.SearchCursor(Intersect71Census,["STOPID","POP10"])

NoteTheadvantageofthewith…asstructureisthatthecursorobjectiserasedfrommemorywhentheiterationiscompleted,whicheliminateslocksonthefeatureclassesbeingevaluated.

Thearcpy.da.SearchCursor()functionrequiresaninputfeatureclass,andalistoffieldstobereturned.Optionally,aSQLstatementcanlimitthenumberofrowsreturned.

Thenextline,forrowincursor,istheiterationthroughthedata.ItisnotanormalPythoniciteration,adistinctionthatwillhaveramificationsincertaininstances.Forinstance,however,itdoesallowforrow-by-rowaccesstodatacontainedwithinthesuppliedfeatureclass.NotethatwhenusingaSearchCursor,eachrowofdataisreturnedasatuple,whichcannotbemodified.Thedatacanbeaccessedusingindexes,asshownintheprecedingcode,wherethetwomembersofthetupleareassignedtovariables.

Theif/elseconditionalallowsthedatatobesorted.Asnotedearlier,theBusStopIDs,whicharethefirstmemberofthedataincludedinthetuple,willbeusedasakey.The

conditionalevaluateswhethertheBusStopIDisincludedinthedictionary’sexistingkeys(whicharecontainedinalistandaccessedusingthedictionary.keys()method).Ifitisnot,itisaddedtothekeys,andassignedavaluethatisalistcontaining(atfirst)onepieceofdata,thepopulationvaluecontainedinthatrow.Ifitdoesexistinthekeys,thelistisappendedwiththenextpopulationvalueassociatedwiththatBusStopID.Withthiscode,wehavenowsortedeachcensusblockpopulationaccordingtotheBusStopwithwhichitisassociated.

Next,weneedtoaddcodetocreatethespreadsheet.Thiscodewillusethesamewith…asstructure,andwillgenerateanaveragepopulationvaluebyusingtwobuilt-inPythonfunctions,sum,whichcreatesasumfromalistofnumbers,andlen,whichwillgetthelengthofalist,tuple,orstring:

withopen(r'C:\Projects\Output\Averages.csv','wb')ascsvfile:

csvwriter=csv.writer(csvfile,delimiter=',')

forbusStopIDindataDictionary.keys():

popList=dataDictionary[busStopID]

averagePop=sum(popList)/len(popList)

data=[busStopID,averagePop]

csvwriter.writerow(data)

TheaveragepopulationvalueisretrievedfromthedictionaryusingtheBusStopIDkey,andthenassignedtothevariableaveragePop.Thetwodatapieces,theBusStopIDandtheaveratePopvariablearethenaddedtoalist,whichissuppliedtoaCSVwriterobject,whichknowshowtoacceptthedataandwriteittoafilelocatedatthefilepathsuppliedtothebuilt-inPythontheopen()function,usedtocreatesimplefiles.

Thescriptiscomplete,althoughitisnicetoaddonemorelineattheendtogiveusvisualconfirmationthatthescripthasrun:

print"DataAnalysisComplete"

Thiswillcreateanoutputindicatingthatthescripthasrun.Onceitisdone,gotothelocationoftheoutputcsvfileandopenit,usingExcelorNotepad,andseetheresultsoftheanalysis.Ourfirstscriptiscomplete!

ThefinalscriptHereishowthescriptshouldlookintheend:

#-*-coding:utf-8-*-

#-------------------------------------------------------------------------

--

#8662_Chapter3Model1.py

#Createdon:2014-04-2221:59:31.00000

#(generatedbyArcGIS/ModelBuilder)

#Description:

#-------------------------------------------------------------------------

--

#Importarcpymodule

importarcpy

importcsv

#Localvariables:

Bus_Stops=r"C:\Projects\PacktDB.gdb\SanFrancisco\Bus_Stops"

CensusBlocks2010=r"C:\Projects\PacktDB.gdb\SanFrancisco\CensusBlocks2010"

Inbound71=r"C:\Projects\PacktDB.gdb\Chapter3Results\Inbound71"

Inbound71_400ft_buffer=

r"C:\Projects\PacktDB.gdb\Chapter3Results\Inbound71_400ft_buffer"

Intersect71Census=

r"C:\Projects\PacktDB.gdb\Chapter3Results\Intersect71Census"

#Process:Select

arcpy.Select_analysis(Bus_Stops,

Inbound71,

"NAME='71IB'ANDBUS_SIGNAG='FerryPlaza'")

#Process:Buffer

arcpy.Buffer_analysis(Inbound71,

Inbound71_400ft_buffer,

"400Feet","FULL","ROUND","NONE","")

#Process:Intersect

arcpy.Intersect_analysis("{0}#;{1}

#".format(Inbound71_400ft_buffer,CensusBlocks2010),

Intersect71Census,"ALL","","INPUT")

dataDictionary={}

witharcpy.da.SearchCursor(Intersect71Census,["STOPID","POP10"])as

cursor:

forrowincursor:

busStopID=row[0]

pop10=row[1]

ifbusStopIDnotindataDictionary.keys():

dataDictionary[busStopID]=[pop10]

else:

dataDictionary[busStopID].append(pop10)

withopen(r'C:\Projects\Output\Averages2.csv','wb')ascsvfile:

spamwriter=csv.writer(csvfile,delimiter=',')

forbusStopIDindataDictionary.keys():

popList=dataDictionary[busStopID]

averagePop=sum(popList)/len(popList)

data=[busStopID,averagePop]

spamwriter.writerow(data)

print"DataAnalysisComplete"

SummaryInthischapter,wecoveredhowtocraftamodelofananalysisandexportittoascript.Afterdiscussingthescript,weadjustedthescripttoincludearesultsanalysisandsummation,whichwasoutputtedtoaCSVfile.Inparticular,wediscussedhowtouseModelBuildertocreateananalysisandexportitasascript,andhowtoadjustthescripttobemorePythonic.WealsobrieflytouchedontheuseofSearchCursors,whichwillbecoveredingreaterdetailinChapter5,ArcPyCursors–Search,Insert,andUpdate.Also,wesawhowbuilt-inmodulessuchastheCSVmodulecanbeusedalongwithArcPytocaptureanalysisoutputinformattedspreadsheets.

Inthenextchapter,wewilldiscusshowtocreatemorecomplexscriptsandbuildfunctionstoavoidrepeatingcode.Thesefunctionswillmakeitpossibletowritecodeonceanduseitforever.ThisreuseofcodewilldemonstratehowPythongoesbeyondautomationofanalysistobecomeanewproductivitytoolset.

Chapter4.ComplexArcPyScriptsandGeneralizingFunctionsInthischapter,wewillmovefromcreatingsimplescriptsbasedonautogeneratedscriptsfromModelBuildertocomplexscriptsthatincorporateadvancedPythonandArcPyconcepts,suchasfunctions.Functionscanimprovecodeandsavetimewhenwritingscripts.Theyarealsousefulwhencreatingmodulesorotherreusablecode,allowingforstandardprogrammingoperationstobescriptedandreadyforfutureuse.

Inthischapter,willcoverthefollowingtopics:

CreatingfunctionstoavoidrepeatingcodeCreatinghelperfunctionstoworkwithArcPylimitationsGeneralizingfunctionstomakethemreusable

Pythonfunctions–AvoidrepeatingcodeProgramminglanguagesshareaconceptthathasaidedprogrammersfordecades:functions.Theideaofafunction,looselyspeaking,istocreateblocksofcodethatwillperformanactiononapieceofdata,transformingitasrequiredbytheprogrammerandreturningthetransformeddatabacktothemainbodyofcode.We’vealreadybeenintroducedtosomeofPython’sbuilt-infunctionsinthelastfewchapters,theintfunction,forinstance,willconvertastringorafloatingnumberintoaninteger;nowit’stimetowriteourown.

Functionsareusedbecausetheysolvemanydifferentneedswithinprogramming.Functionsreducetheneedtowriterepetitivecode,whichinturnreducesthetimeneededtocreateascript.Theycanbeusedtocreaterangesofnumbers(therange()function),ortodeterminethemaximumvalueofalist(themaxfunction),ortocreateaSQLstatementtoselectasetofrowsfromafeatureclass.Theycanevenbecopiedandusedinanotherscriptorincludedaspartofamodulethatcanbeimportedintoscripts.Functionreusehastheaddedbonusofmakingprogrammingmoreusefulandlessofachore.Whenascripterstartswritingfunctions,itisamajorsteptowardsmakingprogrammingpartofaGISworkflow.

TechnicaldefinitionoffunctionsFunctions,alsocalledsubroutinesorproceduresinotherprogramminglanguages,areblocksofcodethathavebeendesignedtoeitheracceptinputdataandtransformit,orprovidedatatothemainprogramwhencalledwithoutanyinputrequired.Intheory,functionswillonlytransformdatathathasbeenprovidedtothefunctionasaparameter;itshouldnotchangeanyotherpartofthescriptthathasnotbeenincludedinthefunction.Tomakethispossible,theconceptofnamespacesisinvoked.AsdiscussedinChapter1,IntroductiontoPythonforArcGIS,namespacesareusedtoisolatevariableswithinascript;variablesareeitherglobal,andavailabletobeusedinthemainbodyofascriptaswellasinafunction,orarelocalandonlyavailablewithinafunction.

Namespacesmakeitpossibletouseavariablenamewithinafunction,andallowittorepresentavalue,whilealsousingthesamevariablenameinanotherpartofthescript.Thisbecomesespeciallyimportantwhenimportingmodulesfromotherprogrammers;withinthatmoduleanditsfunctions,thevariablesthatitcontainsmighthaveavariablenamethatisthesameasavariablenamewithinthemainscript.

Inahigh-levelprogramminglanguagesuchasPython,thereisbuilt-insupportforfunctions,includingtheabilitytodefinefunctionnamesandthedatainputs(alsoknownasparameters).Functionsarecreatedusingthekeyworddefplusafunctionname,alongwithparenthesesthatmayormaynotcontainparameters.Parameterscanalsobedefinedwithdefaultvalues,soparametersonlyneedtobepassedtothefunctionwhentheydifferfromthedefault.Thevaluesthatarereturnedfromthefunctionarealsoeasilydefined.

AfirstfunctionLet’screateafunctiontogetafeelforwhatispossiblewhenwritingfunctions.First,weneedtoinvokethefunctionbyprovidingthedefkeywordandprovidinganamealongwiththeparentheses.ThefirstFunction()willreturnastringwhencalled:

deffirstFunction():

'asimplefunctionreturningastring'

return"MyFirstFunction"

>>>firstFunction()

Theoutputisasfollows:

'MyFirstFunction'

Noticethatthisfunctionhasadocumentationstringordocstring(asimplefunctionreturningastring)thatdescribeswhatthefunctiondoes;thisstringcanbecalledlatertofindoutwhatthefunctiondoes,usingthe__doc__internalfunction:

>>>printfirstFunction.__doc__

Theoutputisasfollows:

'asimplefunctionreturningastring'

Thefunctionisdefinedandgivenaname,andthentheparenthesesareaddedfollowedbyacolon.Thefollowinglinesmustthenbeindented(agoodIDEwilladdtheindentionautomatically).Thefunctiondoesnothaveanyparameters,sotheparenthesesareempty.Thefunctionthenusesthekeywordreturntoreturnavalue,inthiscaseastring,fromthefunction.

Next,thefunctioniscalledbyaddingparenthesestothefunctionname.Whenitiscalled,itwilldowhatithasbeeninstructedtodo:returnthestring.

FunctionswithparametersNowlet’screateafunctionthatacceptsparametersandtransformsthemasneeded.Thisfunctionwillacceptanumberandmultiplyitby3:

defsecondFunction(number):

'thisfunctionmultiplesnumbersby3'

returnnumber*3

>>>secondFunction(4)

Theoutputisasfollows:

12

Thefunctionhasoneflaw,however;thereisnoassurancethatthevaluepassedtothefunctionisanumber.Weneedtoaddaconditionaltothefunctiontomakesureitdoesnotthrowanexception:

defsecondFunction(number):

'thisfunctionmultiplesnumbersby3'

iftype(number)==type(1)ortype(number)==type(1.0):

returnnumber*3

>>>secondFunction(4.0)

Theoutputisasfollows:

12.0

>>>secondFunction(4)

Theoutputisasfollows:

12

>>>secondFunction("String")

>>>

Thefunctionnowacceptsaparameter,checkswhattypeofdataitis,andreturnsamultipleoftheparameterwhetheritisanintegerorafunction.Ifitisastringorsomeotherdatatype,asshowninthelastexample,novalueisreturned.

Thereisonemoreadjustmenttothesimplefunctionthatweshoulddiscuss:parameterdefaults.Byincludingdefaultvaluesinthedefinitionofthefunction,weavoidhavingtoprovideparametersthatrarelychange.If,forinstance,wewantedadifferentmultiplierthan3inthesimplefunction,wewoulddefineitlikethis:

defthirdFunction(number,multiplier=3):

'thisfunctionmultiplesnumbersby3'

iftype(number)==type(1)ortype(number)==type(1.0):

returnnumber*multiplier

>>>thirdFunction(4)

Theoutputisasfollows:

12

>>>thirdFunction(4,5)

Theoutputisasfollows:

20

Thefunctionwillworkwhenonlythenumbertobemultipliedissupplied,asthemultiplierhasadefaultvalueof3.However,ifweneedanothermultiplier,thevaluecanbeadjustedbyaddinganothervaluewhencallingthefunction.Notethatthesecondvaluedoesn’thavetobeanumberasthereisnotypecheckingonit.Also,thedefaultvalue(s)inafunctionmustfollowtheparameterswithnodefaults(orallparameterscanhaveadefaultvalueandtheparameterscanbesuppliedtothefunctioninorderorbyname).

Thesesimplefunctionscombinemanyoftheconceptsthatwediscussedinearlierchapters,includingbuilt-infunctionssuchastype,conditionals,parameters,parameterdefaults,andfunctionreturns.WecannowmoveontocreatingfunctionswithArcPy.

UsingfunctionstoreplacerepetitivecodeOneofthemainusesoffunctionsistoensurethatthesamecodedoesnothavetobewrittenoverandover.Let’sreturntoourexamplefromthelastchapterandmakeafunctionfromthescripttomakeitpossibletoperformthesameanalysisforanybuslineinSanFrancisco.

ThefirstportionofthescriptthatwecouldconvertintoafunctionisthethreeArcPyfunctions.DoingsowillallowthescripttobeapplicabletoanyofthestopsintheBusStopfeatureclassandhaveanadjustablebufferdistance:

bufferDist=400

buffDistUnit="Feet"

lineName='71IB'

busSignage='FerryPlaza'

sqlStatement="NAME='{0}'ANDBUS_SIGNAG='{1}'"

defselectBufferIntersect(selectIn,selectOut,bufferOut,intersectIn,

intersectOut,sqlStatement,bufferDist,buffDistUnit,lineName,

busSignage):

'afunctiontoperformabusstopanalysis'

arcpy.Select_analysis(selectIn,selectOut,

sqlStatement.format(lineName,busSignage))

arcpy.Buffer_analysis(selectOut,bufferOut,"{0}

{1}".format(bufferDist),"FULL","ROUND","NONE","")

arcpy.Intersect_analysis("{0}#;{1}#".format(bufferOut,intersectIn),

intersectOut,"ALL","","INPUT")

returnintersectOut

Thisfunctiondemonstrateshowtheanalysiscanbeadjustedtoaccepttheinputandoutputfeatureclassvariablesasparameters,alongwithsomenewvariables.

ThefunctionaddsavariabletoreplacetheSQLstatementandvariablestoadjustthebusstop,andalsotweaksthebufferdistancestatementsothatboththedistanceandtheunitcanbeadjusted.Thefeatureclassnamevariables,definedearlierinthescript,haveallbeenreplacedwithlocalvariablenames;whiletheglobalvariablenamescouldhavebeenretained,itreducestheportabilityofthefunction.

ThenextfunctionwillaccepttheresultoftheselectBufferIntersect()functionandsearchitusingtheSearchCursor,passingtheresultsintoadictionary.Thedictionarywillthenbereturnedfromthefunctionforlateruse:

defcreateResultDic(resultFC):

'searchresultsofanalysisandcreateresultsdictionary'

dataDictionary={}

witharcpy.da.SearchCursor(resultFC,["STOPID","POP10"])ascursor:

forrowincursor:

busStopID=row[0]

pop10=row[1]

ifbusStopIDnotindataDictionary.keys():

dataDictionary[busStopID]=[pop10]

else:

dataDictionary[busStopID].append(pop10)

returndataDictionary

Thisfunctiononlyrequiresoneparameter:thefeatureclassreturnedfromthesearchBufferIntersect()function.Theresultsholdingdictionaryisfirstcreated,thenpopulatedbythesearchcursor,withthebusStopidattributeusedasakey,andthecensusblockpopulationattributeaddedtoalistassignedtothekey.

Thedictionary,havingbeenpopulatedwithsorteddata,isreturnedfromthefunctionforuseinthefinalfunction,createCSV().ThisfunctionacceptsthedictionaryandthenameoftheoutputCSVfileasastring:

defcreateCSV(dictionary,csvname):

'afunctiontakesadictionaryandcreatesaCSVfile'

withopen(csvname,'wb')ascsvfile:

csvwriter=csv.writer(csvfile,delimiter=',')

forbusStopIDindictionary.keys():

popList=dictionary[busStopID]

averagePop=sum(popList)/len(popList)

data=[busStopID,averagePop]

csvwriter.writerow(data)

ThefinalfunctioncreatestheCSVusingthecsvmodule.Thenameofthefile,astring,isnowacustomizableparameter(meaningthescriptnamecanbeanyvalidfilepathandtextfilewiththeextension.csv).ThecsvfileparameterispassedtotheCSVmodule’swritermethodandassignedtothevariablecsvwriter,andthedictionaryisaccessedandprocessed,andpassedasalisttocsvwritertobewrittentotheCSVfile.Thecsv.writer()methodprocesseseachiteminthelistintotheCSVformatandsavesthefinalresult.OpentheCSVfilewithExceloratexteditorsuchasNotepad.

Torunthefunctions,wewillcalltheminthescriptfollowingthefunctiondefinitions:

analysisResult=selectBufferIntersect(Bus_Stops,Inbound71,

Inbound71_400ft_buffer,CensusBlocks2010,Intersect71Census,bufferDist,

lineName,busSignage)

dictionary=createResultDic(analysisResult)

createCSV(dictionary,r'C:\Projects\Output\Averages.csv')

Now,thescripthasbeendividedintothreefunctions,whichreplacethecodeofthefirstmodifiedscript.Themodifiedscriptlookslikethis:

#-*-coding:utf-8-*-

#-------------------------------------------------------------------------

--

#8662_Chapter4Modified1.py

#Createdon:2014-04-2221:59:31.00000

#(generatedbyArcGIS/ModelBuilder)

#Description:

#AdjustedbySilasToms

#20140505

#-------------------------------------------------------------------------

--

#Importarcpymodule

importarcpy

importcsv

#Localvariables:

Bus_Stops=r"C:\Projects\PacktDB.gdb\SanFrancisco\Bus_Stops"

CensusBlocks2010=r"C:\Projects\PacktDB.gdb\SanFrancisco\CensusBlocks2010"

Inbound71=r"C:\Projects\PacktDB.gdb\Chapter3Results\Inbound71"

Inbound71_400ft_buffer=

r"C:\Projects\PacktDB.gdb\Chapter3Results\Inbound71_400ft_buffer"

Intersect71Census=

r"C:\Projects\PacktDB.gdb\Chapter3Results\Intersect71Census"

bufferDist=400

lineName='71IB'

busSignage='FerryPlaza'

defselectBufferIntersect(selectIn,selectOut,bufferOut,intersectIn,

intersectOut,bufferDist,lineName,busSignage):

arcpy.Select_analysis(selectIn,

selectOut,

"NAME='{0}'ANDBUS_SIGNAG=

'{1}'".format(lineName,busSignage))

arcpy.Buffer_analysis(selectOut,

bufferOut,

"{0}Feet".format(bufferDist),

"FULL","ROUND","NONE","")

arcpy.Intersect_analysis("{0}#;{1}#".format(bufferOut,intersectIn),

intersectOut,"ALL","","INPUT")

returnintersectOut

defcreateResultDic(resultFC):

dataDictionary={}

witharcpy.da.SearchCursor(resultFC,

["STOPID","POP10"])ascursor:

forrowincursor:

busStopID=row[0]

pop10=row[1]

ifbusStopIDnotindataDictionary.keys():

dataDictionary[busStopID]=[pop10]

else:

dataDictionary[busStopID].append(pop10)

returndataDictionary

defcreateCSV(dictionary,csvname):

withopen(csvname,'wb')ascsvfile:

csvwriter=csv.writer(csvfile,delimiter=',')

forbusStopIDindictionary.keys():

popList=dictionary[busStopID]

averagePop=sum(popList)/len(popList)

data=[busStopID,averagePop]

csvwriter.writerow(data)

analysisResult=selectBufferIntersect(Bus_Stops,Inbound71,

Inbound71_400ft_buffer,CensusBlocks2010,Intersect71Census,

bufferDist,lineName,busSignage)

dictionary=createResultDic(analysisResult)

createCSV(dictionary,r'C:\Projects\Output\Averages.csv')

print"DataAnalysisComplete"

Furthergeneralizationofthefunctions,whilewehavecreatedfunctionsfromtheoriginalscriptthatcanbeusedtoextractmoredataaboutbusstopsinSanFrancisco,ournew

functionsarestillveryspecifictothedatasetandanalysisforwhichtheywerecreated.Thiscanbeveryusefulforlongandlaboriousanalysisforwhichcreatingreusablefunctionsisnotnecessary.Thefirstuseoffunctionsistogetridoftheneedtorepeatcode.Thenextgoalistothenmakethatcodereusable.Let’sdiscusssomewaysinwhichwecanconvertthefunctionsfromone-offsintoreusablefunctionsorevenmodules.

First,let’sexaminethefirstfunction:

defselectBufferIntersect(selectIn,selectOut,bufferOut,intersectIn,

intersectOut,bufferDist,lineName,busSignage):

arcpy.Select_analysis(selectIn,

selectOut,

"NAME='{0}'ANDBUS_SIGNAG=

'{1}'".format(lineName,busSignage))

arcpy.Buffer_analysis(selectOut,

bufferOut,

"{0}Feet".format(bufferDist),

"FULL","ROUND","NONE","")

arcpy.Intersect_analysis("{0}#;{1}#".format(bufferOut,intersectIn),

intersectOut,"ALL","","INPUT")

returnintersectOut

Thisfunctionappearstobeprettyspecifictothebusstopanalysis.It’ssospecific,infact,thatwhilethereareafewwaysinwhichwecantweakittomakeitmoregeneral(thatis,usefulinotherscriptsthatmightnothavethesamestepsinvolved),weshouldnotconvertitintoaseparatefunction.Whenwecreateaseparatefunction,weintroducetoomanyvariablesintothescriptinanefforttosimplifyit,whichisacounterproductiveeffort.Instead,let’sfocusonwaystogeneralizetheArcPytoolsthemselves.

ThefirststepwillbetosplitthethreeArcPytoolsandexaminewhatcanbeadjustedwitheachofthem.TheSelecttoolshouldbeadjustedtoacceptastringastheSQLselectstatement.TheSQLstatementcanthenbegeneratedbyanotherfunctionorbyparametersacceptedatruntime(forexample,passedtothescriptbyaScripttool,whichwillbediscussedinalaterchapter).

Forinstance,ifwewantedtomakethescriptacceptmultiplebusstopsforeachrunofthescript(forexample,theinboundandoutboundstopsforeachline),wecouldcreateafunctionthatwouldacceptalistofthedesiredstopsandaSQLtemplate,andwouldreturnaSQLstatementtoplugintotheSelecttool.Hereisanexampleofhowitwouldlook:

defformatSQLIN(dataList,sqlTemplate):

'afunctiontogenerateaSQLstatement'

sql=sqlTemplate#"OBJECTIDIN"

step="("

fordataindataList:

step+=str(data)

sql+=step+")"

returnsql

defformatSQL(dataList,sqlTemplate):

'afunctiontogenerateaSQLstatement'

sql=''

forcount,datainenumerate(dataList):

ifcount!=len(dataList)-1:

sql+=sqlTemplate.format(data)+'OR'

else:

sql+=sqlTemplate.format(data)

returnsql

>>>dataVals=[1,2,3,4]

>>>sqlOID="OBJECTID={0}"

>>>sql=formatSQL(dataVals,sqlOID)

>>>printsql

Theoutputisasfollows:

OBJECTID=1OROBJECTID=2OROBJECTID=3OROBJECTID=4

Thisnewfunction,formatSQL(),isaveryusefulfunction.Let’sreviewwhatitdoesbycomparingthefunctiontotheresultsfollowingit.Thefunctionisdefinedtoaccepttwoparameters:alistofvaluesandaSQLtemplate.Thefirstlocalvariableistheemptystringsql,whichwillbeaddedtousingstringaddition.Thefunctionisdesignedtoinsertthevaluesintothevariablesql,creatingaSQLstatementbytakingtheSQLtemplateandusingstringformattingtoaddthemtothetemplate,whichinturnisaddedtotheSQLstatementstring(notethatsql+=isequivelenttosql=sql+).Also,anoperator(OR)isusedtomaketheSQLstatementinclusiveofalldatarowsthatmatchthepattern.Thisfunctionusesthebuilt-inenumeratefunctiontocounttheiterationsofthelist;onceithasreachedthelastvalueinthelist,theoperatorisnotaddedtotheSQLstatement.

NotethatwecouldalsoaddonemoreparametertothefunctiontomakeitpossibletouseanANDoperatorinsteadofOR,whilestillkeepingORasthedefault:

defformatSQL2(dataList,sqlTemplate,operator="OR"):

'afunctiontogenerateaSQLstatement'

sql=''

forcount,datainenumerate(dataList):

ifcount!=len(dataList)-1:

sql+=sqlTemplate.format(data)+operator

else:

sql+=sqlTemplate.format(data)

returnsql

>>>sql=formatSQL2(dataVals,sqlOID,"AND")

>>>printsql

Theoutputisasfollows:

OBJECTID=1ANDOBJECTID=2ANDOBJECTID=3ANDOBJECTID=4

WhileitwouldmakenosensetouseanANDoperatoronObjectIDs,thereareothercaseswhereitwouldmakesense,henceleavingORasthedefaultwhileallowingforAND.Eitherway,thisfunctioncannowbeusedtogenerateourbusstopSQLstatementformultiplestops(ignoring,fornow,thebussignagefield):

>>>sqlTemplate="NAME='{0}'"

>>>lineNames=['71IB','71OB']

>>>sql=formatSQL2(lineNames,sqlTemplate)

>>>printsql

Theoutputisasfollows:

NAME='71IB'ORNAME='71OB'

However,wecan’tignoretheBusSignagefieldfortheinboundline,astherearetwostartingpointsfortheline,sowewillneedtoadjustthefunctiontoacceptmultiplevalues:

defformatSQLMultiple(dataList,sqlTemplate,operator="OR"):

'afunctiontogenerateaSQLstatement'

sql=''

forcount,datainenumerate(dataList):

ifcount!=len(dataList)-1:

sql+=sqlTemplate.format(*data)+operator

else:

sql+=sqlTemplate.format(*data)

returnsql

>>>sqlTemplate="(NAME='{0}'ANDBUS_SIGNAG='{1}')"

>>>lineNames=[('71IB','FerryPlaza'),('71OB','48thAvenue')]

>>>sql=formatSQLMultiple(lineNames,sqlTemplate)

>>>printsql

Theoutputisasfollows:

(NAME='71IB'ANDBUS_SIGNAG='FerryPlaza')OR(NAME='71OB'AND

BUS_SIGNAG='48thAvenue')

Theslightdifferenceinthisfunction,theasteriskbeforethedatavariable,allowsthevaluesinsidethedatavariabletobecorrectlyformattedintotheSQLtemplatebyexplodingthevalueswithinthetuple.NoticethattheSQLtemplatehasbeencreatedtosegregateeachconditionalbyusingparentheses.Thefunction(s)arenowreadyforreuse,andtheSQLstatementisnowreadyforinsertionintotheSelecttool:

sql=formatSQLMultiple(lineNames,sqlTemplate)

arcpy.Select_analysis(Bus_Stops,Inbound71,sql)

NextupistheBuffertool.Wehavealreadytakenstepstowardsmakingitgeneralizedbyaddingavariableforthedistance.Inthiscase,wewillonlyaddonemorevariabletoit,aunitvariablethatwillmakeitpossibletoadjustthebufferunitfromfeettometeroranyotherallowedunit.Wewillleavetheotherdefaultsalone.

HereisanadjustedversionoftheBuffertool:

bufferDist=400

bufferUnit="Feet"

arcpy.Buffer_analysis(Inbound71,

Inbound71_400ft_buffer,

"{0}{1}".format(bufferDist,bufferUnit),

"FULL","ROUND","NONE","")

Now,boththebufferdistanceandbufferunitarecontrolledbyavariabledefinedinthepreviousscript,andthiswillmakeiteasilyadjustableifitisdecidedthatthedistancewasnotsufficientandthevariablesmightneedtobeadjusted.

ThenextsteptowardsadjustingtheArcPytoolsistowriteafunction,whichwillallowforanynumberoffeatureclassestobeintersectedtogetherusingtheIntersecttool.ThisnewfunctionwillbesimilartotheformatSQLfunctionsasprevious,astheywillusestringformattingandadditiontoallowforalistoffeatureclassestobeprocessedintothecorrectstringformatfortheIntersecttooltoacceptthem.However,asthisfunctionwillbebuilttobeasgeneralaspossible,itmustbedesignedtoacceptanynumberoffeatureclassestobeintersected:

defformatIntersect(features):

'afunctiontogenerateanintersectstring'

formatString=''

forcount,featureinenumerate(features):

ifcount!=len(features)-1:

formatString+=feature+"#;"

else:

formatString+=feature+"#"

returnformatString

>>>shpNames=["example.shp","example2.shp"]

>>>iString=formatIntersect(shpNames)

>>>printiString

Theoutputisasfollows:

example.shp#;example2.shp#

NowthatwehavewrittentheformatIntersect()function,allthatneedstobecreatedisalistofthefeatureclassestobepassedtothefunction.ThestringreturnedbythefunctioncanthenbepassedtotheIntersecttool:

intersected=[Inbound71_400ft_buffer,CensusBlocks2010]

iString=formatIntersect(intersected)

#Process:Intersect

arcpy.Intersect_analysis(iString,

Intersect71Census,"ALL","","INPUT")

Becauseweavoidedcreatingafunctionthatonlyfitsthisscriptoranalysis,wenowhavetwo(ormore)usefulfunctionsthatcanbeappliedinlateranalyses,andweknowhowtomanipulatetheArcPytoolstoacceptthedatathatwewanttosupplytothem.

MoregeneralizationofthefunctionsTheotherfunctionsthatweinitiallycreatedtosearchtheresults,andgeneratethespreadsheetofresults,canalsobemanipulatedintobeingmoregeneralizedwithafewtweaks.

Ifwewanttogeneratemoreinformationabouteachcensusblockwithinadistancetoabusstop(forexample,ifwehadacensusblockdatasetwithincomedataaswellaspopulationdata),wewouldpasstothefunctionalistofattributestobeextractedfromthefinalfeatureclass.Tomakethispossible,itwouldbenecessarytoadjustthecreateResultDic()functiontoacceptthislistofattributes:

defcreateResultDic(resultFC,key,values):

dataDictionary={}

fields=[key]

fields.extend(values)

witharcpy.da.SearchCursor(resultFC,fields)ascursor:

forrowincursor:

busStopID=row[0]

data=row[1:]

ifbusStopIDnotindataDictionary.keys():

dataDictionary[busStopID]=[data]

else:

dataDictionary[busStopID].append(data)

returndataDictionary

ThisnewversionofthecreateResultDic()functionwillgeneratealistoflists(thatis,thevaluesfromeachrowarecontainedwithinalistandareaddedtoamasterlist)foreachbusstop,whichcanthenbeparsedlaterbyknowingthepositionofeachvalueinthelist.Thissolutionisusefulwhenneedingtosortdataintoadictionary.

However,thisisanunsatisfactorywaytosorttheresults.Whatifthelistoffieldsisnotpassedontothedictionaryandthereisnowayofknowingtheorderofthedatainthelists?Instead,wewanttobeabletousethefunctionalityofPythondictionariestosortthedatabyfieldname.Inthiscase,wewillusenesteddictionariestocreatelistsofresultsaccessiblebythetypeofdatatheycontain(thatis,population,income,oranotherfield):

defcreateResultDic(resultFC,key,values):

dataDic={}

fields=[]

iftype(key)==type((1,2))ortype(key)==type([1,2]):

fields.extend(key)

length=len(key)

else:

fields=[key]

length=1

fields.extend(values)

witharcpy.da.SearchCursor(resultFC,fields)ascursor:

forrowincursor:

busStopID=row[:length]

data=row[length:]

ifbusStopIDnotindataDictionary.keys():

dataDictionary[busStopID]={}

forcounter,fieldinenumerate(values):

iffieldnotindataDictionary[busStopID].keys():

dataDictionary[busStopID][field]=[data[counter]]

else:

dataDictionary[busStopID][field].append(data[counter])

returndataDictionary

>>>rFC=r'C:\Projects\PacktDB.gdb\Chapter3Results\Intersect71Census'

>>>key='STOPID'

>>>values='HOUSING10','POP10'

>>>dic=createResultDic(rFC,key,values)

>>>dic[1122023]

Theoutputisasfollows:

{'HOUSING10':[104,62,113,81,177,0,52,113,0,104,81,177,52],

'POP10':[140,134,241,138,329,0,118,241,0,140,138,329,118]}

Inthisexample,thefunctionispassedasparameterstoafeatureclass,theSTOPID,andthefieldstobeconglomerated.ThefieldsvariableiscreatedtopasstherequiredfieldsontotheSearchCursor.Thecursorreturnseachrowasatuple;thefirstmemberofthetupleisbusStopID,andtherestofthetupleisthedataassociatedwiththatbusstop.Thefunctionthenusesaconditiontoassesswhetherthebusstophasbeenpreviouslyanalyzed;ifnot,itisaddedtothedictionaryandassignedasecondinternaldictionary,whichwillbeusedtostoretheresultsassociatedwiththatstop.Byusingadictionary,wecanthensortthroughtheresultsandassignthemtothecorrectfieldtowhichtheybelong.

Thepreviousexampleshowstheresultsofrequestingdataforoneparticularbusstop(1122023).Astherearetwofieldspassedhere,thedatahasbeenorganizedintotwosets,andthefieldnamesarenowkeysfortheinternaldictionary.Becauseofthisorganization,wecannowcreateaveragesforeachfieldinsteadofjustone.

Speakingofaverages,weleftthejobofaveragingtheresultsofthesearchcursoranalysistothecreateCSV()function.Thisshouldalsobeavoided,asitreducestheusefulnessofthecreateCSV()functionbyaddingadditionaldatamanipulationdutiesthatshouldbetheresponsibilityofanotherfunction.Let’saddressthisissuebyadjustingthecreateCSV()functionfirst:

defcreateCSV(data,csvname,mode='ab'):

withopen(csvname,mode)ascsvfile:

csvwriter=csv.writer(csvfile,delimiter=',')

csvwriter.writerow(data)

Thisisastrippeddownversionofthefunction,butitisinfinitelymoreuseful.Byadjustingthefunctionlikethis,wearelimitingittoonlydoingtwothings:openingtheCSVfileandaddingarowofdatatoit.Becauseweusedtheabmode,iftheCSVfileexists,wewillonlybeaddingdatatoitinsteadofwritingoverit(ifitdoesn’texist,itwillbecreated).Thisaddingmodecanbeoverriddenbypassingwbasthemode,whichwillgenerateanewscripteachtime.

Nowwecansortthroughtheresultsoftheanalysis,averagethem,andpassthemtoour

newcreateCSVscript.Todothis,wewilliteratethroughthedictionarycreatedbythecreateResultDic()function:

csvname=r'C:\Projects\Output\Averages.csv'

dataKey='STOPID'

fields='HOUSING10','POP10'

dictionary=createResultDic(Intersect71Census,dataKey,fields)

header=[dataKey]

forfieldinfields:

header.append(field)

createCSV(header,csvname,'wb')

forcounter,busStopinenumerate(dictionary.keys()):

datakeys=dictionary[busStop]

averages=[busStop]

forkeyindatakeys:

data=datakeys[key]

average=sum(data)/len(data)

averages.append(average)

createCSV(averages,csvname)

ThislaststepshowshowtheCSVfileiscreated:byiteratingthroughthedatacontainedinthedictionaryandthenaveragingthevaluesforeachbusstop.Then,theseaveragesareaddedtoalistthatcontainsthenameofeachbusstop(andthelineitbelongstointhisinstance)andpassedtothecreateCSV()functiontobewrittenintotheCSVfile.

Hereisthefinalcode.NotethatIhaveconvertedmanyoftheautogeneratedcommentsintoprintstatementstogivesomefeedbackonthestateofthescript:

#-*-coding:utf-8-*-

#-------------------------------------------------------------------------

--

#8662_Chapter4Modified2.py

#Createdon:2014-04-2221:59:31.00000

#(generatedbyArcGIS/ModelBuilder)

#Description:

#AdjustedbySilasToms

#20140423

#-------------------------------------------------------------------------

--

#Importarcpymodule

importarcpy

importcsv

Bus_Stops=r"C:\Projects\PacktDB.gdb\SanFrancisco\Bus_Stops"

CensusBlocks2010=r"C:\Projects\PacktDB.gdb\SanFrancisco\CensusBlocks2010"

Inbound71=r"C:\Projects\PacktDB.gdb\Chapter4Results\Inbound71"

Inbound71_400ft_buffer=

r"C:\Projects\PacktDB.gdb\Chapter4Results\Inbound71_400ft_buffer"

Intersect71Census=

r"C:\Projects\PacktDB.gdb\Chapter4Results\Intersect71Census"

bufferDist=400

bufferUnit="Feet"

lineNames=[('71IB','FerryPlaza'),('71OB','48thAvenue')]

sqlTemplate="NAME='{0}'ANDBUS_SIGNAG='{1}'"

intersected=[Inbound71_400ft_buffer,CensusBlocks2010]

dataKey='NAME','STOPID'

fields='HOUSING10','POP10'

csvname=r'C:\Projects\Output\Averages.csv'

defformatSQLMultiple(dataList,sqlTemplate,operator="OR"):

'afunctiontogenerateaSQLstatement'

sql=''

forcount,datainenumerate(dataList):

ifcount!=len(dataList)-1:

sql+=sqlTemplate.format(*data)+operator

else:

sql+=sqlTemplate.format(*data)

returnsql

defformatIntersect(features):

'afunctiontogenerateanintersectstring'

formatString=''

forcount,featureinenumerate(features):

ifcount!=len(features)-1:

formatString+=feature+"#;"

else:

formatString+=feature+"#"

returnformatString

defcreateResultDic(resultFC,key,values):

dataDictionary={}

fields=[]

iftype(key)==type((1,2))ortype(key)==type([1,2]):

fields.extend(key)

length=len(key)

else:

fields=[key]

length=1

fields.extend(values)

witharcpy.da.SearchCursor(resultFC,fields)ascursor:

forrowincursor:

busStopID=row[:length]

data=row[length:]

ifbusStopIDnotindataDictionary.keys():

dataDictionary[busStopID]={}

forcounter,fieldinenumerate(values):

iffieldnotindataDictionary[busStopID].keys():

dataDictionary[busStopID][field]=[data[counter]]

else:

dataDictionary[busStopID][field].append(data[counter])

returndataDictionary

defcreateCSV(data,csvname,mode='ab'):

withopen(csvname,mode)ascsvfile:

csvwriter=csv.writer(csvfile,delimiter=',')

csvwriter.writerow(data)

sql=formatSQLMultiple(lineNames,sqlTemplate)

print'Process:Select'

arcpy.Select_analysis(Bus_Stops,

Inbound71,

sql)

print'Process:Buffer'

arcpy.Buffer_analysis(Inbound71,

Inbound71_400ft_buffer,

"{0}{1}".format(bufferDist,bufferUnit),

"FULL","ROUND","NONE","")

iString=formatIntersect(intersected)

printiString

print'Process:Intersect'

arcpy.Intersect_analysis(iString,

Intersect71Census,"ALL","","INPUT")

print'ProcessResults'

dictionary=createResultDic(Intersect71Census,dataKey,fields)

print'CreateCSV'

header=[dataKey]

forfieldinfields:

header.append(field)

createCSV(header,csvname,'wb')

forcounter,busStopinenumerate(dictionary.keys()):

datakeys=dictionary[busStop]

averages=[busStop]

forkeyindatakeys:

data=datakeys[key]

average=sum(data)/len(data)

averages.append(average)

createCSV(averages,csvname)

print"DataAnalysisComplete"

SummaryInthischapter,wediscussedhowtotakeautogeneratedcodeandmakeitgeneralized,whileaddingfunctionsthatcanbereusedinotherscriptsandwillmakethegenerationofthenecessarycodecomponents,suchasSQLstatements,mucheasier.Wealsoaddressedwhenitisbestnottogotoofarwiththecreationoffunctionstoavoidmakingthemtoospecific.

Inthenextchapter,wewillinvestigatethepowerfulDataAccessmoduleanditsSearchCursors,UpdateCursors,andInsertCursors.

Chapter5.ArcPyCursors–Search,Insert,andUpdateNowthatweunderstandhowtointeractwithArcToolboxtoolsusingArcPy,andwehavealsocoveredusingPythontocreatefunctionsandimportmodules,wehaveabasicunderstandingofhowtoimproveGISworkflowsusingPython.InthischapterwewillcoverdatacursorsandtheDataAccessmodule,introducedin10.1.Thesedataaccesscursorsareavastimprovementonthecursorsusedinthearcgisscriptingmodule(theprecursortoArcPy)andinearlierversionsofArcPy.Notonlycanthecursorssearchdata,aswehaveseen,buttheycanupdatedatausingtheUpdateCursorsandcanaddnewrowsofdatausingtheInsertCursor.

Datacursorsareusedtoaccessdatarecordscontainedwithindatatables,usingarowbyrowiterativeapproach.Theconceptwasborrowedfromrelationaldatabases,wheredatacursorsareusedtoextractdatafromtablesreturnedfromaSQLexpression.Cursorsareusedtosearchfordata,butalsotoupdatedataortoaddnewdata.

WhenwediscusscreatingdatasearchesusingArcPycursors,wearenotjusttalkingaboutattributeinformation.Thenewdataaccessmodelcursorscaninteractdirectlywiththeshapefield,andwhencombinedwithArcPyGeometryobjects,canperformgeospatialfunctionsandreplacetheneedtopassdatatoArcToolboxtools.DataaccesscursorsrepresentthemostusefulinnovationyetintherealmofPythonautomationforGIS.

Inthischapterwewillcover:

UsingSearchCursorstoaccessattributeandspatialdataUsingUpdateCursorstoadjustvalueswithinrowsUsinginsertcursorstoaddnewdatatoadatasetUsingcursorsandtheArcPyGeometryobjecttypestoperformgeospatialanalysesinmemory

ThedataaccessmoduleIntroducedwiththereleaseofArcGIS10.1,thenewdataaccessmoduleknownasarcpy.dahasmadedatainteractioneasier,andfaster,thanallowedbypreviousdatacursors.Byallowingfordirectaccesstotheshapefieldinavarietyofforms(shapeobject,Xvalues,Yvalues,centroid,area,length,andmore),andavarietyofformats(JavaScriptObjectNotation(JSON),KeyholeMarkupLanguage(KML),WellKnownBinary(WKB),Well-KnownText(WKT)),thedataaccessmodulegreatlyincreasestheabilityofaGISanalysttoextractandcontrolshapefielddata.

Thedataaccesscursorsacceptanumberofrequiredandoptionalparameters.Therequiredparametersarethepathtothefeatureclassasastring(oravariablerepresentingthepath)andthefieldstobereturned.Ifallfieldsaredesired,usingtheasterisknotationandprovidealistwithanasteriskasastringasthefield’sparameter([*]).Ifonlyafewfieldsarerequired,providethosefieldsasstringfieldnames(forexample[“NAME”,“DATE”]).

Theotherparametersareoptionalbutareveryimportant,forbothsearchandUpdateCursors.AwhereclauseintheformofaSQLexpressioncanbeprovidednext;thisclausewilllimitthenumberofrowsreturnedfromthedataset(asdemonstratedbytheSQLexpressioninthescriptsinthelastchapter).TheSQLexpressionsusedbythesearchandupdatecursorsarenotcompleteSQLexpressions,astheSELECTorUPDATEcommandsareprovidedautomaticallybythechoiceofcursor.OnlythewhereclauseoftheSQLexpressionisrequiredforthisparameter.

AspatialreferencecanbeprovidednextintheArcPySpatialReferenceformat;thisisnotnecessaryifthedataisinthecorrectformatbutcanbeusedtotransformdataintoanotherprojectiononthefly.Thereisnowaytospecifythespatialtransformationused,however.ThethirdoptionalparameterisaBoolean(orTrue/False)valuethatdeclareswhetherdatashouldbereturnedinexplodedpoints(thatis,alistoftheindividualvertices)orintheoriginalgeometryformat.Thefinaloptionalparameterisanotherlistthatcanbeusedtoorganizethedatareturnedbythecursor;thislistwouldincludeSQLkeywordssuchasDISTINCT,ORBERBY,orGROUPBY.However,thisfinalparameterisonlyavailablewhenworkingwithageodatabase.

Let’stakealookatusingarcpy.da.SearchCursorforshapefieldinteractions.Ifweneededtoproduceaspreadsheetlistingallbusstopsalongaparticularroute,andincludethelocationofthedatainanX/Yformat,wecouldusetheAddXYtoolfromtheArcToolbox.However,thishastheeffectofaddingtwonewfieldstoourdata,whichisnotalwaysallowed,especiallywhenthedataisstoredinenterprisegeodatabaseswithfixedschemas.Instead,we’llusetheSHAPE@XYtokenbuiltintothedataaccessmoduletoeasilyextractthedataandpassittothecreateCSV()functionfromChapter4,ComplexArcPyScriptsandGeneralizingFunctions,alongwiththeSQLexpressionlimitingresultstothestopsofinterest:

csvname="C:\Projects\Output\StationLocations.csv"

headers='BusLineName','BusStopID','X','Y'

createCSV(headers,csvname,'wb')

sql="(NAME='71IB'ANDBUS_SIGNAG='FerryPlaza')OR(NAME='71OB'

ANDBUS_SIGNAG='48thAvenue')"

witharcpy.da.SearchCursor(Bus_Stops,['NAME','STOPID','SHAPE@XY'],sql)

ascursor:

forrowincursor:

linename=row[0]

stopid=row[1]

locationX=row[2][0]

locationY=row[2][1]

locationY=row[2][1]

data=linename,stopid,locationX,locationY

createCSV(data,csvname)

Notethateachrowofdataisreturnedasatuple;thismakessenseastheSearchCursordoesnotallowanydatamanipulationandtuplesareimmutableassoonastheyarecreated.Incontrast,datareturnedfromUpdateCursorsisinlistformat,aslistscanbeupdated.Bothcanbeaccessedusingtheindexingasshownpreviously.

Eachrowreturnedbythecursorisatuplewiththreeobjects:thenameofthebusstop,thebusstopID,andfinallyanothertuplecontainingtheX/Ylocationofthestop.Theobjectsinthetuple,containedinthevariablerow,areaccessibleusingindexing:thebusstopnameisatindex0,theIDisatindex1,andthelocationtupleisatindex2.

Withinthelocationtuple,theXvalueisatindex0andtheYvalueisatindex1;thismakesiteasytoaccessthedatainthelocationtuplebypassingavalueasshowninthefollowing:

locationX=row[2][0]

TheabilitytoaddlistsandtuplesandevendictionariestoanotherlistortupleordictionaryisastrongcomponentofPython,makingdataaccesslogicalanddataorganizationeasy.

However,thespreadsheetreturnedfromthepreviouscodehasafewissues:thelocationisreturnedinthenativeprojectionofthefeatureclass(inthiscase,aStatePlaneprojection),andtherearerowsofdatathatarerepeated.Itwouldbemuchmorehelpfulifwecouldprovidelatitudeandlongitudevaluesinthespreadsheetandtheduplicatevalueswereremoved.Let’susetheoptionalspatialreferenceparameterandalisttosortthedatabeforewepassittothecreateCSV()function:

spatialReference=arcpy.SpatialReference(4326)

sql="(NAME='71IB'ANDBUS_SIGNAG='FerryPlaza')OR(NAME='71OB'

ANDBUS_SIGNAG='48thAvenue')"

dataList=[]

witharcpy.da.SearchCursor(Bus_Stops,['NAME','STOPID','SHAPE@XY'],sql,

spatialReference)ascursor:

forrowincursor:

linename=row[0]

stopid=row[1]

locationX=row[2][0]

locationY=row[2][1]

data=linename,stopid,locationX,locationY

ifdatanotindataList:

dataList.append(data)

csvname="C:\Projects\Output\StationLocations.csv"

headers='BusLineName','BusStopID','X','Y'

createCSV(headers,csvname,'wb')

fordataindataList:

Thespatialreferenceiscreatedbypassingacoderepresentingthedesiredprojectionsystem.InthiscasethecodefortheWGS1984LatitudeandLongitudegeographicsystemis4326andispassedtothearcpy.SpatialReference()methodtocreateaspatialreferenceobjectthatcanbepassedtotheSearchCursor.Also,theifconditionalisusedtofilterthedata,acceptingonlyonelistperstopintothelistcalleddataList.ThisnewversionofthecodewillproduceaCSVfilewiththedesireddata.ThisCSVcouldthenbeconvertedintoaKMLwiththeserviceprovidedbywww.convertcsv.com/csv-to-kml.htm,orevenbetter,usingPython.Usestringformattingandloopstoinsertthedataintopre-builtKMLstrings.

AttributefieldinteractionsApartfromtheshapefieldinteractions,anotherimprovementofferedbythedataaccessmodulecursorsistheabilitytocallthefieldsinafeatureclassbyusingalist,asdiscussedpreviously.Earlierdatacursorsrequiredtheuseofalessefficientgetvaluefunctioncall,orrequiredthefieldstobecalledasiftheyweremethodsavailabletothefunction.Thenewmethodallowsforallfieldstobecalledbypassinganasterisk,avaluablemethodtoaccessfieldsinfeatureclassesthathavenotbeeninspectedpreviously.

OneofthemorevaluableimprovementsistheabilitytoaccesstheUniqueIDfieldwithoutneedingtoknowwhetherthedatasetisafeatureclassorashapefile.BecauseshapefileshadafeatureIDorFID,andfeatureclasseshadanobjectID,itwashardertoprogramaScripttooltoaccesstheuniqueIDfield.DataaccessmodulecursorsallowfortheuseoftheOID@stringtorequesttheuniqueIDfromeithertypeofinput.ThismakestheneedtoknowthetypeofuniqueIDirrelevant.

Asdemonstratedpreviously,otherattributefieldsarerequestedbyastringinalist.Thefieldnamesmustmatchthetruenameofthefield;aliasnamescannotbepassedtothecursor.Thefieldscanbeinthelistinanyorderdesired,andwillbereturnedintheorderrequested.Onlytherequiredfieldshavetobeincludedinthelist.

Hereisademonstrationofrequestingfieldinformation:

sql="OBJECTID=1"

witharcpy.da.SearchCursor(Bus_Stops,

['STOPID','NAME','OID@'],

sql)ascursor:

forrowincursor:

Ifthefieldsinthefieldslistwereadjusted,thedataintheresultingrowwouldreflecttheadjustment.Also,allofthemembersofthetuplereturnedbythecursorareaccessiblebyzero-basedindexing.

UpdatecursorsUpdatecursorsareusedtoadjustdatawithinexistingrowsofdata.Updatesbecomeveryimportantwhencalculatingdataorconvertingnullvaluestoanon-nullvalue.CombinedwithspecificSQLexpressions,datacanbetargetedforupdatingwithnewlycollectedorcalculatedvalues.

NotethatrunningcodecontaininganUpdateCursorwillchange,orupdate,thedataonwhichitoperates.Itisagoodideatomakeacopyofthedatatotestoutthecodebeforerunningitontheoriginaldata.

AlldataaccessmoduleSearchCursorparametersdiscussedpreviouslyarevalidforUpdateCursors.ThemaindifferenceisthatdatarowsreturnedbyUpdateCursorsarereturnedaslists.Becauselistsaremutable,theycanbeadjustedusingalistvalueassignment.

Asanexample,let’simaginethatthebusline71willberenamedtothe75.Bothinboundandoutboundlineswillbeaffected,soaSQLexpressionmustbeincludedtogetallrowsofdataassociatedwiththeline.Oncethedatacursoriscreated,therowsreturnedmusthavethenameadjusted,addedbackintothelist,andtheUpdatecursor’supdateRowmethodmustbeinvoked.Hereishowthisscenariowouldlookincode:

sql="NAMELIKE'71%'"

witharcpy.da.UpdateCursor(Bus_Stops,['NAME'],sql),)ascursor:

forrowincursor:

lineName=row[0]

newName=lineName.replace('71','75')

row[0]=newName

TheSQLexpressionwillreturnallrowsofdatawithanamestartingwith71;thiswillinclude71IBand71OB.NotethattheSQLexpressionmustbeenclosedindoublequotes,astheattributevalueneedstobeinsinglequotes.

Foreachrowofdata,thenameatpositionzerointherowreturnedisassignedtothevariablelineName.Thisvariable,astring,usesthereplace()methodtoreplacethecharacters71withthecharacters75.Thiscouldalsojustbereplacing1with5butIwantedtobeexplicitastowhatisbeingreplaced.

Oncethenewstringhasbeengenerated,itisassignedtothevariablenewName.Thisvariableisthenaddedtothelistreturnedbythecursorusinglistassignment;thiswillreplacethedatavaluethatinitiallyoccupiedthezeropositioninthelist.Oncetherowvaluehasbeenassigned,itisthenpassedtothecursor’supdateRow()method.Thismethodacceptstherowandupdatesthevalueinthefeatureclassforthatparticularrow.

UpdatingtheshapefieldForeachrow,allvaluesincludedinthelistreturnedbythecursorareavailableforupdate,excepttheuniqueID(whilenoexceptionwillbethrown,theUIDvalueswillnotbeupdated).Eventheshapefieldcanbeadjusted,withafewcaveats.Themaincaveatisthattheupdatedshapefieldmustbethesamegeometrytypeastheoriginalrow,apointcanbereplacedwithapoint,alinewithaline,andapolygonwithanotherpolygon.

AdjustingapointlocationIfabusstopwasmoveddownthestreetfromitscurrentposition,itwouldneedtobeupdatedusinganUpdateCursor.ThisoperationwillrequireanewlocationinanX/Yformat,preferablyinthesameprojectionasthefeatureclasstoavoidanylossoflocationfidelityinaspatialtransformation.Therearetwomethodsavailabletousforcreatinganewpointlocation,dependingonthemethodusedtoaccessthedata.ThefirstmethodisusedwhenthelocationdataisrequestedusingtheSHAPE@tokens,andrequirestheuseofanArcPyGeometrytype,inthiscasethePointtype.TheArcPyGeometrytypesarediscussedindetailinthenextchapter.

sql='OBJECTID<5'

witharcpy.da.UpdateCursor(Bus_Stops,['OID@','SHAPE@'],sql)ascursor:

forrowincursor:

row[1]=arcpy.Point(5999783.78657,2088532.563956)

BypassinganXandYvaluetotheArcPyPointGeometry,aPointshapeobjectiscreatedandpassedtothecursorintheupdatedlistreturnedbythecursor.Assigninganewlocationtotheshapefieldinatuple,thenusingthecursor’supdateRow()methodallowstheshapefieldvaluetobeadjustedtothenewlocation.Becausethefirstfourbusstopsareatthesamelocation,theyareallmovedtothenewlocation.

Thesecondmethodappliestoallotherformsofshapefieldinteractions,includingtheSHAPE@XY,SHAPE@JSON,SHAPE@KML,SHAPE@WKT,andSHAPE@WKBtokens.Theseareupdatedbypassingthenewlocationintheformatrequestedbacktothecursorandupdatingthelist:

sql='OBJECTID<5'

witharcpy.da.UpdateCursor(Bus_Stops,['OID@','SHAPE@XY'],sql)ascursor:

forrowincursor:

row[1]=(5999783.786500007,2088532.5639999956)

HereisthesamecodeusingtheSHAPE@JSONkeywordandaJSONrepresentationofthedata:

sql='OBJECTID<5'

witharcpy.da.UpdateCursor(Bus_Stops,['OID@','SHAPE@JSON'],sql)as

cursor:

forrowincursor:

printrow

row[1]=u'{"x":5999783.7865000069,"y":2088532.5639999956,

"spatialReference":{"wkid":102643}}'

Aslongasthekeyword,thedataformat,andthegeometrytypematch,thelocationisupdatedtothenewcoordinates.Thekeywordmethodisveryusefulwhenupdatingpoints,however,theSHAPE@XYkeyworddoesnotworkwithlinesorpolygonsasthelocationreturnedrepresentsthecentroidoftherequestedgeometry.

DeletingarowusinganUpdateCursorIfweneedtoremovearowofdata,theUpdateCursorhasadeleteRowmethodthatworkstoremovetherow.Notethatthiswillcompletelyremovethedatarow,makingitunrecoverable.Thismethoddoesnotrequireaparametertobepassedtoit;instead,itwillremovethecurrentrow:

sql='OBJECTID<2'

Bus_Stops=r'C:\Projects\PacktDB.gdb\Bus_Stops'

witharcpy.da.UpdateCursor(Bus_Stops,

['OID@',

'SHAPE@XY'],sql)ascursor:

forrowincursor:

UsinganInsertCursorNowthatwehaveagrasponhowtoupdateexistingdata,let’sinvestigateusingInsertCursorstocreatenewdataandaddittoafeatureclass.Themethodsinvolvedareverysimilartousingotherdataaccesscursors,exceptthatwedonotneedtocreateaniterablecursortoextractrowsofdata;instead,wewillcreateacursorthatwillhavethespecialinsertRowmethodthatiscapableofaddingdatatothefeatureclassrowbyrow.

TheInsertCursorcanbecalledusingthesamewith..assyntaxbutgenerallyitiscreatedasavariableintheflowofthescript.

NoteNotethatonlyonecursorcanbeinvokedatatime;anexception(aPythonerror)willbegeneratedwhencreatingtwoinsert(orupdate)cursorswithoutfirstremovingtheinitialcursorusingthePythondelkeywordtoremovethecursorvariablefrommemory.Thisiswhythewith..assyntaxispreferredbymany.

Thedataaccessmodule’sInsertCursorrequiressomeofthesameparametersastheothercursors.Thefeatureclasstobewrittentoandthelistoffieldsthatwillhavedatainserted(thisincludestheshapefield)arerequired.Thespatialreferencewillnotbeusedasthenewshapedatamustbeinthesamespatialreferenceasthefeatureclass.NoSQLexpressionisallowedforanInsertCursor.

Thedatatobeaddedtothefeatureclasswillbeintheformofatupleoralist,inthesameorderasthefieldsthatarelistedinthefieldslistparameter.Onlyfieldsofinterestneedtobeincludedinthelistoffields,meaningnoteveryfieldneedsavalueinthelisttobeadded.Whenaddinganewrowofdatatoafeatureclass,theuniqueIDwillautomaticallybegenerated,makingitunnecessarytoexplicitlyincludetheuniqueID(intheformoftheOID@keyword)inthelistoffieldstobeadded.

Let’sexplorecodethatcouldbeusedtogenerateanewbusstop.We’llwritetoatestdatasetcalledTestBusStops.WeareonlyinterestedintheNameandStopIDfields,sothosefieldsalongwiththeshapefield(whichisinaStatePlaneprojectionsystem)willbeincludedinthedatalisttobeadded:

Bus_Stops=r'C:\Projects\PacktDB.gdb\TestBusStops'

insertCursor=arcpy.da.InsertCursor(Bus_Stops,['SHAPE@',

'NAME','STOPID'])

coordinatePair=(6001672.5869999975,2091447.0435000062)

newPoint=arcpy.Point(*coordinatePair)

dataList=[newPoint,'NewStop1',112121]

insertCursor.insertRow(dataList)

delinsertCursor

Ifthereisaniterablelistofdatatobeinsertedintothefeatureclass,createtheInsertCursorvariablebeforeenteringtheiteration,anddeletetheInsertCursorvariableoncethedatahasbeeniteratedthrough,orusethewith..asmethodtoautomaticallydeletetheInsertCursorvariablewhentheiterationiscomplete:

Bus_Stops=r'C:\Projects\PacktDB.gdb\TestBusStops'

listOfLists=[[(6002672.58675,2092447.04362),'NewStop2',112122],

[(6003672.58675,2093447.04362),'NewStop3',112123],

[(6004672.58675,2094447.04362),'NewStop4',112124]

]

witharcpy.da.InsertCursor(Bus_Stops,

['SHAPE@',

'NAME',

'STOPID'])asiCursor:

fordataListinlistOfLists:

newPoint=arcpy.Point(*dataList[0])

dataList[0]=newPoint

Asalist,thelistOfListsvariableisiterable.EachlistwithinitisconsideredasdataListintheiteration,andthefirstvalueindataList(thecoordinatepair)ispassedtothearcpy.Point()functiontocreateaPointobject.Thearcpy.Point()functionrequirestwoparameters,XandY;theseareextractedfromthecoordinatepairtupleusingtheasterisk,which‘explodes’thetupleandpassesthevaluesitcontainstothefunction.ThePointobjectisthenaddedbackintodataListusinganindex-basedlistassignment,whichwouldnotbeavailabletousifthedataListvariablewasatuple(wewouldinsteadhavetocreateanewlistandaddinthePointobjectandtheotherdatavalues).

InsertingapolylinegeometryTocreateandinsertapolyline-typeshapefieldfromaseriesofpoints,it’sbesttousetheSHAPE@keyword.WewillalsofurtherexploretheArcPyGeometrytypes,whichwillbediscussedinthenextchapter.WhenworkingwiththeSHAPE@keyword,wehavetoworkwithdatainESRI’sspatialbinaryformats,andthedatamustbewrittenbacktothefieldinthesameformatusingtheArcPyGeometrytypes.

Tocreateapolyline,thereisonerequirement,atleasttwovalidpointsmadeoftwocoordinatepairs.WhenworkingwiththeSHAPE@keyword,thereisamethodologytoconvertingthecoordinatepairsintoanArcPyPointandthenaddingittoanArcPyArray,whichisthenconvertedintoanArcPyPolylinetobewrittenbacktotheshapefield:

listOfPoints=[(6002672.58675,2092447.04362),

(6003672.58675,2093447.04362),

(6004672.58675,2094447.04362)

]

line='NewBusLine'

lineID=12345

busLine=r'C:\Projects\PacktDB.gdb\TestBusLine'

insertCursor=arcpy.da.InsertCursor(busLine,['SHAPE@',

'LINE','LINEID'])

lineArray=arcpy.Array()

forpointsPairinlistOfPoints:

newPoint=arcpy.Point(*pointsPair)

lineArray.add(newPoint)

newLine=arcpy.Polyline(lineArray)

insertData=newLine,line,lineID

ThethreecoordinatepairsintuplesareiteratedandconvertedintoPointobjects,whichareinturnaddedtotheArrayobjectcalledlineArray.TheArrayobjectisthenaddedtothePolylineobjectcallednewLine,whichisthenaddedtoatuplewiththeotherdataattributesandinsertedintothefeatureclassbytheInsertCursor.

InsertingapolygongeometryPolygonsarealsoinserted,orupdated,usingcursors.TheArcPyPolygonGeometrytypedoesnotrequirethecoordinatepairstoincludethefirstpointtwice(thatis,asthefirstpointandasthelastpoint).Thepolygonisclosedautomaticallybythearcpy.Polygon()function:

listOfPoints=[(6002672.58675,2092447.04362),

(6003672.58675,2093447.04362),

(6004672.58675,2093447.04362),

(6004672.58675,2091447.04362)

]

polyName='NewPolygon'

polyID=54321

blockPoly=r'C:\Projects\PacktDB.gdb\Chapter5Results\TestPolygon'

insertCursor=arcpy.da.InsertCursor(blockPoly,['SHAPE@','BLOCK',

'BLOCKID'])

polyArray=arcpy.Array()

forpointsPairinlistOfPoints:

newPoint=arcpy.Point(*pointsPair)

polyArray.add(newPoint)

newPoly=arcpy.Polygon(polyArray)

insertData=newPoly,polyName,polyID

insertCursor.insertRow(insertData)

Hereisavisualizationoftheresultoftheinsertoperation:

SummaryInthischapterwecoveredthebasicusesofdataaccessmodulecursors.Search,updateandInsertCursorswereexploredanddemonstrated,andaspecialfocuswasplacedontheuseofthesecursorsforextractingshapedatafromtheshapefield.Cursorparameterswerealsointroduced,includingthespatialreferenceparameterandtheSQLexpressionwhereclauseparameter.Inthenextchapter,wewillfurtherexploretheuseofcursors,especiallywiththeuseofArcPyGeometrytypes.

Chapter6.WorkingwithArcPyGeometryObjectsTheessenceofgeospatialanalysisisusinggeometricshapes–points,lines,andpolygons–tomodelthegeographyofrealworldobjectsandtheirlocation-basedrelationships.Thesimpleshapesandtheirgeometricpropertiesoflocation,lengthandareaareprocessedusinggeospatialoperationstogenerateanalysisresults.Itisthecombinationofmodeledgeographicdataandtheassociatedattributeinformationthatseparategeospatialinformationsystemsfromallotherinformationsystems.

UntilArcPy,processingthefeatureclassgeometryusingthegeospatialoperationswasdependedonthepre-builttoolswithinArcToolbox.ArcPyhasmadeitpossibletodirectlyaccessthegeometricshapeswhicharestoredasmathematicalrepresentationsintheshapefieldoffeatureclasses.Onceaccessed,thisgeometricdataisloadedintoArcPygeometryobjectstomakethedataavailableforanalysiswithinanArcPyscript.Becauseofthisadvance,writingscriptsthataccessgeometryfieldsandusethemtoperformanalysishastransformedArcGISgeospatialanalysis.Inthischapter,we’llexplorehowtogenerateandusetheArcPygeometryobjectstoperformgeospatialoperations,andapplythemtothebusstopsanalysis.

Inthischapter,wewillcover:PointandArrayconstructorobjectsandPointGeometry,Polyline,andPolygongeometryobjects

HowtousethegeometryobjectstoperformgeospatialoperationsHowtointegratethegeometryobjectsintoscriptsHowtoperformcommongeospatialoperationsusingthegeometryobjectsHowtoreplacetheuseofArcToolboxtoolsinthescriptwithgeometryobjectmethods

ArcPygeometryobjectclassesIndesigninggeometryobjects,theauthorsofArcPymadeitpossibletoperformgeospatialoperationsinmemory,reducingtheneedtousetoolsintheArcToolboxfortheseoperations.Thiswillresultinspeedgainsasthereisnoneedtowritetheresultsofthecalculationstodiskateachstepoftheanalysis.Instead,theresultsofthestepscanbepassedfromfunctiontofunctionwithinthescript.Thefinalresultsoftheanalysiscanbewrittentotheharddriveasafeatureclass,ortheycanbewrittenintoaspreadsheetorpassedtoanotherprogram.

ThegeometryobjectsarewrittenasPythonclasses-specialblocksofcodethatcontaininternalfunctions.Theinternalfunctionsarethemethodsandpropertiesofthegeometryobjects;whencalledtheyallowtheobjecttoperformanoperation(amethod)ortorevealinformationaboutthegeometryobject(aproperty).Pythonclassesarewrittenwithamainclassthatcontainssharedmethodsandproperties,andwithsub-classesthatreferencethemainclassbutalsohavespecificmethodsandpropertiesthatarenotshared.Here,themainclassistheArcPyGeometryobject,whilethesub-classesarethePointGeometry,Multipoint,PolylineandPolygonobjects.

Thegeometryobjectsaregeneratedinthreeways.Thefirstrequiresusingdatacursorstoreadexistingfeatureclassesandpassingaspecialkeywordasafieldname.Theshapedatareturnedbythecursorisageometryobject.Thesecondmethodistocreatenewdatabypassingrawcoordinatestoaconstructorobject(eitheraPointorArrayobject),whichisthenpassedtoageometryobject.ThethirdmethodistoreaddatafromafeatureclassusingtheCopyFeaturestoolfromtheArcToolbox.

Eachgeometryobjecthasmethodsthatallowforreadaccessandwriteaccess.Thereadaccessmethodsareimportantforaccessingthecoordinatepointsthatconstitutethepoints,linesandpolygons.Thewriteaccessmethodsareimportantwhengeneratingnewdataobjectsthatcanbeanalyzedorwrittentodisk.

ThePointGeometry,Multipoint,Polyline,andPolygongeometryobjectsareusedforperforminganalysisupontheirrespectivegeometrytypes.Thegenericgeometryobjectcanacceptanygeometrytypeandanoptionalspatialreferencetoperformgeospatialoperationswhenthereisnoneedtodiscernthegeometrytype.

TwootherArcPyclasseswillbeusedforperforminggeospatialoperationsinmemory:theArrayobjectandthePointobject.Theyareconstructorobjects,astheyarenotsub-classedfromthegeometryclass,butareinsteadusedtoconstructthegeometryobjects.ThePointobjectisusedtocreatecoordinatepointsfromrawcoordinates.TheArrayobjectisalistofcoordinatepointsthatcanbepassedtoaPolylineorPolygonobject,asaregularPythonlistofArcPyPointobjectscannotbeusedtogeneratethosegeometryobjects.

ArcPyPointobjectsPointobjectsarethebuildingblocksusedtogenerategeometryobjects.Also,allofthegeometryobjectswillreturncomponentcoordinatesasPointobjectswhenusingreadaccessmethods.PointobjectsallowforsimplegeometryaccessusingitsX,YandZproperties,andalimitednumberofgeospatialmethods,suchascontains,overlaps,within,touches,crosses,equals,anddisjoint.Let’suseIDLEtoexploresomeofthesemethodswithtwoPointgeometryobjectswiththesamecoordinates:

>>>Point=arcpy.Point(4,5)

>>>point1=arcpy.Point(4,5)

>>>Point.equals(point1)

True

>>>Point.contains(point1)

True

>>>Point.crosses(point1)

False

>>>Point.overlaps(point1)

False

>>>Point.disjoint(point1)

False

>>>Point.within(point1)

True

>>>point.X,Point.Y

(4.0,5.0)

Intheseexamples,weseesomeoftheidiosyncrasiesofthePointobject.Withtwopointsthathavethesamecoordinates,theresultsoftheequalsmethodandthedisjointmethodareasexpected.ThedisjointmethodwillreturnTruewhenthetwoobjectsdonotsharecoordinates,whiletheoppositeistruewiththeequalsmethod.ThecontainsmethodwillworkwiththetwoPointobjectsandreturnTrue.Thecrossesmethodandoverlapsmethodaresomewhatsurprisingresults,asthetwoPointobjectsdooverlapinlocationandcouldbeconsideredtocross;however,thosemethodsdonotreturntheexpectedresultastheyarenotbuilttocomparetwopoints.

ArcPyArrayobjectsBeforeweprogressuptoPolylineandPolygonobjects,weneedtounderstandtheArcPyArrayobject.ItisthebridgebetweenthePointobjectsandthosegeometryobjectsthatrequiremultiplecoordinatepoints.ArrayobjectsacceptPointobjectsasparameters,andtheArrayobjectisinturnpassedasaparametertothegeometryobjecttobecreated.Let’susePointobjectswithanArrayobjecttounderstandbetterhowtheyworktogether.

TheArrayobjectissimilartoaPythonlist,withextend,append,andreplacemethods,andalsohasuniquemethodssuchasaddandclone.TheaddmethodwillbeusedtoaddPointobjectsindividually:

>>>Point=arcpy.Point(4,5)

>>>point1=arcpy.Point(7,9)

>>>Array=arcpy.Array()

>>>Array.add(point)

>>>Array.add(point1)

Theextend()methodwouldaddalistofPointobjectsallatonce:

>>>Point=arcpy.Point(4,5)

>>>point1=arcpy.Point(7,9)

>>>pList=[Point,point1]

>>>Array=arcpy.Array()

>>>Array.extend(pList)

TheinsertmethodwillputaPointobjectintheArrayataspecificindex,whilethereplacemethodisusedtoreplaceaPointobjectinanArraybypassinganindexandanewPointobject:

>>>Point=arcpy.Point(4,5)

>>>point1=arcpy.Point(7,9)

>>>point2=arcpy.Point(11,13)

>>>pList=[Point,point1]

>>>Array=arcpy.Array()

>>>Array.extend(pList)

>>>Array.replace(1,point2)

>>>point3=arcpy.Point(17,15)

>>>Array.insert(2,point3)

TheArrayobject,whenloadedwithPointobjects,canthenbeusedtogeneratetheothergeometryobjects.

ArcPyPolylineobjectsThePolylineobjectisgeneratedwithanArrayobjectthathasatleasttwoPointobjects.AsgiveninthefollowingIDLEexample,onceanArrayobjecthasbeengeneratedandloadedwiththePointobjects,itcanthenbepassedasaparametertoaPolylineobject:

>>>Point=arcpy.Point(4,5)

>>>point1=arcpy.Point(7,9)

>>>pList=[Point,point1]

>>>Array=arcpy.Array()

>>>Array.extend(pList)

>>>pLine=arcpy.Polyline(Array)

NowthatthePolylineobjecthasbeencreated,itsmethodscanbeaccessed.Thisincludesmethodstorevealtheconstituentcoordinatepointswithinthepolyline,andotherrelevantinformation:

>>>pLine.firstPoint

<Point(4.0,5.0,#,#)>

>>>pLine.lastPoint

<Point(7.0,9.0,#,#)>

pLine.getPart()

<Array[<Array[<Point(4.0,5.0,#,#)>,<Point(7.0,9.0,#,#)>]>]>

>>>pLine.trueCentroid

<Point(5.5,7.0,#,#)>

>>>pLine.length

5.0

>>>pLine.pointCount

2

ThisexamplePolylineobjecthasnotbeenassignedaspatialreferencesystem,sothelengthisunitless.Whenageometryobjectdoeshaveaspatialreferencesystem,thelinearandarealunitswillbereturnedinthelinearunitofthesystem.

ThePolylineobjectisalsoourfirstgeometryobjectwithwhichwecaninvokegeometryclassmethodsthatperformgeospatialoperations,suchasbuffers,distanceanalyses,andclips:

>>>bufferOfLine=pLine.buffer(10)

>>>bufferOfLine.area

413.93744395

>>>bufferOfLine.contains(pLine)

True

>>>newPoint=arcpy.Point(25,19)

>>>pLine.distanceTo(newPoint)

20.591260281974

AnotherusefulmethodofPolylineobjectsisthepositionAlongLinemethod.ItisusedtoreturnaPointGeometryobject,discussedinthefollowing,ataspecificpositionalongtheline.ThispositionalongthelinecaneitherbeanumericdistancefromthefirstPointorasapercentage(expressedasafloatfrom0-1),whenusingtheoptionalsecondparameter:

>>>nPoint=pLine.positionAlongLine(3)

>>>nPoint.firstPoint.X,nPoint.firstPoint.Y

(5.8,7.4)>>>pPoint=pLine.positionAlongLine(.5,True)

>>>pPoint.firstPoint.X,pPoint.firstPoint.Y

(5.5,7.0)

ThereareanumberofothermethodsavailabletoPolylineobjects.Moreinformationisavailablehere:http://resources.arcgis.com/en/help/main/10.2/index.html#//018z00000008000000

ArcPyPolygonobjectsTocreateaPolygonobject,anArrayobjectmustbeloadedwithPointobjectsandthenpassedasaparametertothePolygonobject.OncethePolygonobjecthasbeengenerated,themethodsavailabletoitareveryusefulforperforminggeospatialoperations.ThegeometryobjectscanalsobesavedtodiskusingtheArcToolboxCopyFeaturestool.ThisIDLEexampledemonstrateshowtogenerateashapefilebypassingaPolygonobjectandarawstringfilenametothetool:

>>>importarcpy

>>>point1=arcpy.Point(12,16)

>>>point2=arcpy.Point(14,18)

>>>point3=arcpy.Point(11,20)

>>>Array=arcpy.Array()

>>>Points=[point1,point2,point3]

>>>Array.extend(points)

>>>Polygon=arcpy.Polygon(array)

>>>arcpy.CopyFeatures_management(polygon,r'C:\Projects\Polygon.shp')

<Result'C:\\Projects\\Polygon.shp'>

PolygonobjectbuffersPolygonobjects,likePolylineobjects,havemethodsthatmakeiteasytoperformgeospatialoperationssuchasbuffers.Bypassinganumbertothebuffermethodasaparameter,abufferwillbegeneratedinmemory.TheunitofthenumberisdeterminedbytheSpatialReferencesystem.Internalbufferscanbegeneratedbysupplyingnegativebuffernumbers;thebuffergeneratedbeingtheareawithinthePolygonobjectatthespecifieddistancefromthePolygonperimeter.Clips,unions,symmetricaldifferences,andmoreoperationsareavailableasmethods,asarewithinorcontainsoperations;evenprojectionscanbeperformedusingthePolygonobjectmethodsaslongasithasaSpatialReferencesystemobjectpassedasaparameter.FollowingisascriptthatwillcreatetwoshapefileswithtwoseparateSpatialReferencesystems,eachidentifiedbyanumericcode(2227and4326)fromtheEPSGcodingsystem:

importarcpyPoint=arcpy.Point(6004548.231,2099946.033)

point1=arcpy.Point(6008673.935,2105522.068)

point2=arcpy.Point(6003351.355,2100424.783)Array=arcpy.Array()

Array.add(point1)

Array.add(point)

array.add(point2)

Polygon=arcpy.Polygon(array,2227)

buffPoly=Polygon.buffer(50)

features=[Polygon,buffPoly]

arcpy.CopyFeatures_management(features,

r'C:\Projects\Polygons.shp')

spatialRef=arcpy.SpatialReference(4326)

polygon4326=Polygon.projectAs(spatialRef)

arcpy.CopyFeatures_management(polygon4326,

r'C:\Projects\polygon4326.shp')

HereishowthesecondshapefilelooksintheArcCatalogPreviewwindow:

OtherPolygonobjectmethodsUnlikethecliptoolintheArcToolbox,whichcanclipaPolygonusinganotherpolygon,theclipmethodrequiresanextentobject(anotherArcPyclass)andislimitedtoarectangularenvelopearoundtheareatobeclipped.Toremoveareasfromapolygon,thedifferencemethodcanworklikethecliporerasetoolintheArcToolbox:

buffPoly=Polygon.buffer(500)

donutHole=buffPoly.difference(Polygon)

features=[Polygon,donutHole]

arcpy.CopyFeatures_management(features,

r"C:\Projects\Polygons2.shp")

Hereisthedonuthole-likeresultofthebufferanddifferenceoperation.ThebufferwiththedonutholesurroundstheoriginalPolygonobject:

ArcPygeometryobjectsThegenericgeometryobjectisquiteusefulforcreatinginmemoryacopyofthegeometryofafeatureclass,withoutfirstneedingtoknowwhichtypeofgeometrythefeatureclasscontains.LikealloftheArcPygeometryobjects,itsreadmethodsincludetheextractionofthedatainmanyformatssuchasJSON,WKT,andWKB.Thearea(ifitisapolygon),thecentroid,theextent,andtheconstituentpointsofeachgeometryarealsoavailable,asdemonstratedpreviously.

HereisanexampleofreadingthegeometryofafeatureclassintomemoryusingtheCopyFeaturestool:

importarcpy

cen2010=r'C:\Projects\ArcPy.gdb\SanFrancisco\CensusBlocks2010'

blockPolys=arcpy.CopyFeatures_management(cen2010,

arcpy.Geometry())

ThevariableblockPolysisaPythonlistcontainingallofthegeometriesloadedintoit;inthiscaseitiscensusblocks.Thelistcanthenbeiteratedtobeanalyzed.

ArcPyPointGeometryobjectsThePointGeometryobjectisveryusefulforperformingthesesamegeospatialoperationswithpoints,whicharenotavailablewiththePointobjects.WhenacursorisusedtoretrieveshapedatafromafeatureclasswithaPointGeometrytype,theshapedataisreturnedasaPointGeometryobject.WhilePointobjectsarerequiredtoconstructallothergeometryobjectswhenacursorisnotusedtoretrievedatafromafeatureclass,it’sthePointGeometryobjectthatisusedtoperformpointgeospatialoperations.

Let’sexploregettingPointGeometryobjectsfromadataaccessmoduleSearchCursorandusingthereturneddatarowstocreatebufferedpoints.Inourbusstopanalysis,thiswillreplacetheneedtousetheArcToolboxBuffertooltocreatethe400footbuffersaroundeachstop.ThescriptinthefollowingusesadictionarytocollectthebufferobjectsandthensearchesthecensusblocksusinganotherSearchCursor.ToaccesstheshapefieldusingtheSearchCursor()method,theSHAPE@tokenispassedasoneofthefields.Then,thescriptwilliteratethroughthebusstopsandfindallcensusblockswithwhicheachstopintersects:

#Generate400footbuffersaroundeachbusstop

importarcpy,csv

busStops=r"C:\Projects\PacktDB.gdb\SanFrancisco\Bus_Stops"

censusBlocks2010=r"C:\Projects\PacktDB.gdb\SanFrancisco\CensusBlocks2010"

sql="NAME='71IB'ANDBUS_SIGNAG='FerryPlaza'"

dataDic={}

witharcpy.da.SearchCursor(busStops,['NAME','STOPID','SHAPE@'],sql)as

cursor:

forrowincursor:

linename=row[0]

stopid=row[1]

shape=row[2]

dataDic[stopid]=shape.buffer(400),linename

NowthatthedatahasbeenretrievedandthebuffershavebeengeneratedusingthebuffermethodofthePointGeometryobjects,thebufferscanbecomparedagainstthecensusblockgeometryusingiterationandaSearchCursor.Therewillbetwogeospatialmethodsusedinthisanalysis:overlapandintersect.Theoverlapsmethodisabooleanoperation,returningavalueoftrueorfalsewhenonegeometryiscomparedagainstanother.Theintersectmethodisusedtogettheactualareaoftheintersectaswellasidentifyingthepopulationofeachblock.Usingtheintersectrequirestwoparameters:asecondgeometryobject,andanintegerindicatingwhichtypeofgeometrytoreturn(1forpoint,2forline,4forpolygon).Wewantthepolygonalareaofintersectreturnedtohaveanareaofintersectionavailablealongwiththepopulationdata:

#Intersectcensusblocksandbusstopbuffers

processedDataDic={}={}

forstopidindataDic.keys():

values=dataDic[stopid]

busStopBuffer=values[0]

linename=values[1]

blocksIntersected=[]

witharcpy.da.SearchCursor(censusBlocks2010,

['BLOCKID10','POP10','SHAPE@'])ascursor:

forrowincursor:

block=row[2]

population=row[1]

blockid=row[0]

ifbusStopBuffer.overlaps(block)==True:

interPoly=busStopBuffer.intersect(block,4)

data=row[0],row[1],interPoly,block

blocksIntersected.append(data)

processedDataDic[stopid]=values,blocksIntersected

Thisportionofthescriptiteratesthroughtheblocksandintersectsagainstthebufferedbusstops.Nowthatwecanidentifytheblocksthattouchthebufferaroundeachstopandthedataofinteresthasbeencollectedintothedictionary,itcanbeprocessedandtheaveragepopulationofalloftheblockstouchedbythebuffercanbecalculated:

#Createanaveragepopulationforeachbusstop

dataList=[]

forstopidinprocessedDataDic.keys():

allValues=processedDataDic[stopid]

popValues=[]

blocksIntersected=allValues[1]

forblocksinblocksIntersected:

popValues.append(blocks[1])

averagePop=sum(popValues)/len(popValues)

busStopLine=allValues[0][1]

busStopID=stopid

finalData=busStopLine,busStopID,averagePop

dataList.append(finalData)

Nowthatthedatahasbeencreatedandaddedtoalist,itcanbeoutputtedtoaspreadsheetusingthecreateCSVmodulewecreatedinChapter4,ComplexArcPyScriptsandGeneralizingFunctions:

#Generateaspreadsheetwiththeanalysisresults

defcreateCSV(data,csvname,mode='ab'):

withopen(csvname,mode)ascsvfile:

csvwriter=csv.writer(csvfile,delimiter=',')

csvwriter.writerow(data)

csvname="C:\Projects\Output\StationPopulations.csv"

headers='BusLineName','BusStopID','AveragePopulation'

createCSV(headers,csvname,'wb')

fordataindataList:

createCSV(data,csvname)

Thedatahasbeenprocessedandwrittentothespreadsheet.Thereisonemorestepthatwecantakewiththedataandthatistousetheareaoftheintersectiontocreateaproportionalpopulationvalueforeachbuffer.Let’sredotheprocessingofthedatatoincludetheproportionalareas:

dataList=[]

forstopidinprocessedDataDic.keys():

allValues=processedDataDic[stopid]

popValues=[]

blocksIntersected=allValues[1]

forblocksinblocksIntersected:

pop=blocks[1]

totalArea=blocks[-1].area

interArea=blocks[-2].area

finalPop=pop*(interArea/totalArea)

popValues.append(finalPop)

averagePop=round(sum(popValues)/len(popValues),2)

busStopLine=allValues[0][1]

busStopID=stopid

finalData=busStopLine,busStopID,averagePop

dataList.append(finalData)

NowthescriptistakingfulladvantageofthepowerofArcPygeometryobjects,andthescriptisrunningcompletelyinmemorywhichavoidsproducinganyintermediatedatasets.

SummaryInthischapter,wediscussedindetailtheuseofArcPygeometryobjects.Thesevariedobjectshavesimilarmethodsandare,infact,sub-classedfromthesamePythonclass.Theyareusefulforperformingin-memorygeospatialanalyses,whichavoidshavingtoreadandwritedatafromtheharddriveandalsoskipscreatinganyintermediatedata.

ArcPygeometryobjectswillbecomeanimportantpartofautomatinggeospatialworkflows.CombiningthemwithSearchCursorsmakesArcPymoreusefulthananyearlierimplementationofPythonscriptingtoolsforArcGIS.Next,wewillconverttherawscriptintoascripttoolthatcanbeexecuteddirectlyfromtheArcToolboxorapersonaltoolboxinageodatabase.

Chapter7.CreatingaScriptToolNowthatthebasicsofcreatingandexecutingArcPyscriptshavebeencovered,weneedtotakethenextstepandcreatere-useableScripttools.CreatingScripttoolswillallowforgreatercodereuse,andwillmakeiteasytocreatecustomtoolsforotherGISanalystsandcustomers.WithaPythonscriptbackendorcode,andafamiliarArcGIStoolfrontendoruserinterface,theparticularsofthecodearehiddenfromtheuser;itbecomesjustanothertool,albeitatoolthatcansavedaysandweeksofwork.

Thischapterwillcoverthefollowingtopics:

AddingparameterstoscriptstoacceptinputandproduceoutputasrequiredbytheuserCreatingacustomtoolfrontendandacustomtoolboxSettingtheparametersofthetoolfrontendtoallowittopassargumentstothecodebackend

AddingdynamicparameterstoascriptThescriptswehavegeneratedinpreviouschaptershaveallhadhard-codedinputs.Theinputvalueswerewritteninthescriptasstringsornumbersandassignedtovariables.WhiletheycanbeupdatedmanuallytoreplacetheinputandoutputfilepathsandSQLstatements,programmersshouldaimtocreatescriptsthatwillnotrequireeditingeachtimetheyareused.Instead,scriptsshouldbedesignedtobedynamicandacceptfilepathsandotherinputsasparametersorarguments,inmuchthesamemannerthatthefunctionswehavecreatedacceptparameters.

Pythonwasdesignedwiththisinmind,andthesysmodulehasamethodcalledsys.argvthatacceptsinputspassedtothescriptwhenitisexecuted.WhilethedesignersofArcPyanditspredecessorarcgisscriptingmoduleinitiallytookadvantageofthesys.argvmethod,intimetheydesignedanArcPymethodforacceptingscriptparameters.AseithermethodcanbeusedwhenwritingArcPyscripts,andbotharefoundinexamplescriptsontheweb,itisimportanttorecognizetheminutedifferencesbetweenthesys.argvmethodandarcpy.GetParameterAsText().Themajordifferencebetweenthetwomethodsisthatsys.argvacceptsthedynamicargumentsasalist.Membersofthelistareaccessedusinglistindexingandassignedtovariables.Arcpy.GetParameterAsText()isafunctionthatacceptsanindexnumberparameter.Theindexnumberpassedreflectstheorderoftheparameterwithinthetool’sfrontend;thefirstparameteriszero,thenextisone,andsoon.

NoteIftheorderoftheparametersisadjustedinthetoolfrontend,thisadjustmentmustbereflectedinthecodebackend.

Displayingscriptmessagesusingarcpy.AddMessageItisimportanttoreceivefeedbackfromscriptstoassesstheprogressofthescriptasitperformsananalysis.Asbasicasthiswouldseem,Pythonscriptsandprogramminglanguagesingeneraldonot,bydefault,provideanyfeedbackexceptforerrorsandtheterminationofthescript.Thiscanbeabitdiscouragingtothenoviceprogrammer,asallbuilt-infeedbackisnegative.

Toalleviatethislackoffeedback,theuseofprintstatementsallowsthescripttogivereportsontheprogressoftheanalysisasitruns.However,whenusingaScripttool,printstatementsdonothaveanyeffect.Theywillnotbedisplayedanywhere,andareignoredbythePythonexecutable.TodisplaymessagesinthescriptconsolewhenScripttoolsareexecuted,ArcPyhasaarcpy.AddMessage()method.

Arcpy.AddMessagestatementsareaddedtoscriptswhereverfeedbackisrequiredbytheprogrammer.Thefeedbackrequiredispassedtothemethodasaparameteranddisplayed;whetheritbealist,string,floatorinteger.Arcpy.AddMessagemakesiteasytocheckontheresultsofanalysiscalculations,toensurethatthecorrectinputsareusedandthatthecorrectoutputsareproduced.Asthisfeedbackfromthescriptcanbeapowerfuldebuggingtool,usearcpy.AddMessagewheneverthereisaneedforfeedbackfromtheScripttool.

NoteNotethatstatementspassedtoarcpy.AddMessagewillonlydisplaywhenthescriptisrunasaScripttool.

AddingdynamiccomponentstothescriptTostartmakingthescriptintoaScripttool,weshouldfirstcopythescriptthatwecreatedinChapter6,WorkingwithArcPyGeometryObjectsChapter6_1.py,asChapter7_1.pyinanewfoldercalledChapter7.Wecanthenstartreplacingthehard-codedvariableswithdynamicvariablesusingarcpy.GetParameterAsText.ThereisanotherArcPymethodcalledGetParameterthatacceptstheinputsasanobject,butforourpurposes,GetParameterAsTextisthemethodtouse.

Byaddingarcpy.GetParameterAsTextandarcpy.AddMessagetothescript,wewillhavetakenthefirststeptowardscreatingaScripttool.Caremustbetakentoensurethatthevariablescreatedfromthedynamicparametersareinthecorrectorder,asreorderingthemcanbetime-consuming.Oncetheparametersareaddedtothescriptandthehard-codedportionsofthescriptreplacedwithvariables,thescriptisreadytobecomeaScripttool.

First,moveallofthevariablesthatarehard-codedintothetopofthescript.Then,replacealloftheassignedvalueswitharcpy.GetParameterAsText,makingthemdynamicvalues.Eachparameterisreferredtousingzero-basedindexing;however,theyarepassedtoafunctionindividuallyinsteadofasamemberofalist:

#Chapter7.py

importarcpy,csv

busStops=arcpy.GetParameterAsText(0)

censusBlocks2010=arcpy.GetParameterAsText(1)

censusBlockField=arcpy.GetParameterAsText(2)

csvname=arcpy.GetParameterAsText(3)

headers=arcpy.GetParameterAsText(4).split(',')

sql=arcpy.GetParameterAsText(5)

keyfields=arcpy.GetParameterAsText(6).split(';')

dataDic={}

censusFields=['BLOCKID10',censusBlockField,'SHAPE@']

if"SHAPE@"notinkeyfields:

keyfields.append("SHAPE@")

arcpy.AddMessage(busStops)

arcpy.AddMessage(censusBlocks2010)

arcpy.AddMessage(censusBlockField)

arcpy.AddMessage(csvname)

arcpy.AddMessage(sql)

arcpy.AddMessage(keyfields)

Asyoucanseefromthevariablekeyfieldsandthevariableheaders,somefurtherprocessingmustbeappliedtocertainvariables,asnotallofthemaregoingtobeusedasstrings.Inthiscase,alistiscreatedfromthestringpassedtothevariablekeyfieldsbyusingthestringfunctionssplitandsplittingthestringoneverysemi-colon,whiletheheadersvariableiscreatedbysplittingonthecommas.Toothervariables,suchasthecensusBlockFieldvariableandthevariablekeyfields,theSHAPE@keywordisaddedbecauseitwillberequiredeachtimetheanalysisisrun.Ifaparticularfieldisrequiredforeachrunofthedata,suchastheBLOCKID10field,itcanremainhard-codedinthescript,oroptionallycouldbecomeitsownselectablefieldparameterintheScripttool.

Thevariableswillthenbeaddedtotheremainderofthescriptinthecorrectplaces,makingthescriptreadyfortheScripttooltobecomepartofacustomToolboxinageodatabaseorinArcToolbox.However,wemustfirstcreatethetoolpartoftheScripttoolforthevaluestobecollectedandpassedtothescript.

CreatingaScripttoolCreatingascripttoolisapowerfulcombinationofthepowerofArcPyandtheeaseofuseofthetoolsinArcToolbox.

Thefirststepistocreateacustomtoolboxtoholdthescripttool.Toachievethis,dothefollowing:

1. OpenupArcCatalogandrightclickintheSanFrancisco.gdbFileGeodatabase.2. SelectNewandthenToolboxfromthemenu.3. CallthetoolboxChapter8Tools.4. RightclickonChapter8Tools,selectAdd,andthenselectScript.

Thefollowingmenuwillappearallowingyoutosetupthescripttool.Addatitlewithnospacesandalabel,aswellasadescription.Iprefertorunscripttoolsintheforegroundtoseethemessagesitpasses,butitisnotnecessaryandcanbeannoyingwhenneedingtostartascriptandstillworkonothertasks.ClickNextoncethemenuhasbeenfilledout.

Thenextmenucontainsanentryfieldandafiledialogbutton,allowingtheusertofindthescripttowhichtheparameterscollectedwillbepassed.Usethefiledialogtonavigatetoandselectthescript,andmakesurethatRunPythonscriptinprocessischecked.

Now,pushNextoncethescripthasbeenidentified.

LabellinganddefiningparametersThenextdialogboxisthemostimportantone.Itiswheretheparameterstobepassedarelabeledandtheirdatatypesaredefined.Caremustbetakentochoosethecorrectdatatypeforeachparameterastherearemultipledatatypesthatcanbesuppliedforsomeoftheparameters.Also,propertiesforeachparameterwillbedefined;thisinformationwillcharacterizethedatatobecollectedandhelptomakeitpossibleforthedatatobeinthecorrectformataswellasthecorrectdatatype.

Startbyaddingthedisplaynameforeachparametertobecollected.Thedisplaynameshouldexplainthetypeofinputthatisrequired.Forinstance,thefirstparameterwillbethebusstop’sfeatureclass,sothedisplaynamecouldbeBusStopFeatureClass.

NoteMakesuretoaddthedisplaynamesintheorderthattheywillbepassedtovariablesinthescript.

AddingdatatypesNext,addintheDataTypesforeachparameter.Thisisintricatebecausetherewillbealargelistofdatatypestochoosefrom,andoftenthereareafewtypesthatwouldworkformanyparameters.Forinstance,ifashapefileparameteriscreated,itwouldallowtheusertoselectashapefileasexpected.However,itmightbebettertousetheFeatureClassdatatype,asthenbothshapefilesandfeatureclassescouldbeusedintheanalysis.

AddingtheBusStopfeatureclassasaparameterThefirstparameteristheBusStopfeatureclass,anditshouldbeaFeatureClassdatatype.ClickontheDataTypeFieldnexttothedisplaynameandadrop-downlistwillappear.Oncethedatatypeisselected,checkouttheparameterpropertiesbelowthelistofparameters.FortheBusStopfeatureclass,thedefaultswillbeacceptable,asthe

featureclassisrequired,isnotamulti-value,hasnodefaultorenvironmentsettings,andisnotobtainedfromanyotherparameter.

NoteSomeoftheparameterswillrequireanotherparametertobeselectedfirstastheyarederivedvaluesobtainedfromthefirstparameter.TheCensusBlockFieldparameterandtheSQLstatementparameterderivevaluesfromtheCensusBlockfeatureclassandBusStopfeatureclassparameters,respectively.

AddingtheCensusBlockfeatureclassasaparameterTheCensusBlockfeatureclassissimilartotheBusStopfeatureclass.ItwillbeaFeatureClassdatatype,allowingbothshapefilesandfeatureclassestobeselected,andthereisnoneedtoadjustthedefaultparameterproperties.Oncethedatatypehasbeenset,the

CensusBlockparameterisreadyforuse.

AddingtheCensusBlockfieldasaparameterTheCensusBlockfieldparameterhasanewtwist;itisobtainedfromtheCensusBlockfeatureclassparameter,andwillonlybepopulatedoncethatfirstparameterhasbeencreated.Tomakethispossible,theObtainedfromparameterpropertywillhavetobeset.SelectFieldasthedatatype,andthenclickontheblankareanexttotheObtainedfromparameterpropertyandselectCensus_Block_Feature_Class.ThiswillcreatealistofthefieldscontainedwithintheCensusBlockfeatureclass.

AddingtheoutputspreadsheetasaparameterAsthespreadsheetthatwillbeproducedfromtheanalysisrunbythescripttoolisaCommaSeparatedValue(CSV)file,selectTextFileastheDataType.Settingthe

Defaultparameterpropertytoafilenamecansavetime,andwillmaketherequiredfileextensioneasiertoidentify.Also,asthespreadsheetwillbeproducedbytheScripttoolasanoutput,theDirectionparameterpropertyshouldbeOutput.

AddingthespreadsheetfieldnamesasaparameterThefieldnameschosenasheadersfortheoutputspreadsheetshouldbeaStringdatatype,withthefieldnamesseparatedbycommasandnospaces.Thescriptusesthestringdatatype’ssplitmethodtoseparatethefieldnames.Passingacommatothesplitmethodseparatestheparameterbysplittingtheinputstringonthecommastocreatealistoffieldnames.Thelistoffieldnameswillbeusedasaheaderbythecsvmodulewhencreatingthespreadsheet.

AddingtheSQLStatementasaparameterTheSQLStatementparameterwillrequirethehelpfulSQLExpressionBuildermenuandshouldthereforebeaSQLExpressiondatatype.TheSQLExpressionBuilderwilluseafieldobtainedfromtheBusStopfeatureclass.SettheObtainedfromparameterpropertytotheBusStopfeatureclassbyclickingonthatpropertyandselectingBus_Stop_Feature_Classfromthedrop-downlist.

AddingthebusstopfieldsasaparameterThefinalparameteristhebusstopfieldsparameter,whichisaFielddatatypethatwillbeobtainedfromtheBusStopfeatureclass.ChangetheMultiValueparameterpropertyfromNotoYestoallowtheusertoselectmultiplefields.AlsoremembertosettheObtainedfromparameterpropertytoBus_Stop_Feature_ClasssothatthefieldsarepopulatedfromtheBusStopfeatureclassparameter.

Nowthatalltheparametershavebeendescribedandtheirpropertieshavebeenset,thescripttoolisready.ClickonFinishtoclosethemenu.

InspectingthefinalscriptOncealloftheparametershavebeencollected,thevariablesarethenusedtoreplacethehard-codedfieldlistsorotherstaticscriptelementswiththenewdynamicparameterscollectedfromthescripttool.Inthismanner,thescripthasbecomeavaluabletoolthatcanbeusedformultipledataanalyses,asthefieldstobeanalyzedarenowdynamic:

importarcpy,csv

busStops=arcpy.GetParameterAsText(0)

censusBlocks2010=arcpy.GetParameterAsText(1)

censusBlockField=arcpy.GetParameterAsText(2)

csvname=arcpy.GetParameterAsText(3)

headers=arcpy.GetParameterAsText(4).split(',')

sql=arcpy.GetParameterAsText(5)

keyfields=arcpy.GetParameterAsText(6).split(';')

dataDic={}

censusFields=['BLOCKID10',censusBlockField,'SHAPE@']

if"SHAPE@"notinkeyfields:

keyfields.append("SHAPE@")

arcpy.AddMessage(busStops)

arcpy.AddMessage(censusBlocks2010)

arcpy.AddMessage(censusBlockField)

arcpy.AddMessage(csvname)

arcpy.AddMessage(sql)

arcpy.AddMessage(keyfields)

x=0

witharcpy.da.SearchCursor(busStops,keyfields,sql)ascursor:

forrowincursor:

stopid=x

shape=row[-1]

dataDic[stopid]=[]

dataDic[stopid].append(shape.buffer(400))

dataDic[stopid].extend(row[:-1])

x+=1

processedDataDic={}

forstopidindataDic.keys():

values=dataDic[stopid]

busStopBuffer=values[0]

blocksIntersected=[]

witharcpy.da.SearchCursor(censusBlocks2010,censusFields)ascursor:

forrowincursor:

block=row[-1]

population=row[1]

blockid=row[0]

ifbusStopBuffer.overlaps(block)==True:

interPoly=busStopBuffer.intersect(block,4)

data=population,interPoly,block

blocksIntersected.append(data)

processedDataDic[stopid]=values,blocksIntersected

dataList=[]

forstopidinprocessedDataDic.keys():

allValues=processedDataDic[stopid]

popValues=[]

blocksIntersected=allValues[-1]

forblocksinblocksIntersected:

pop=blocks[0]

totalArea=blocks[-1].area

interArea=blocks[-2].area

finalPop=pop*(interArea/totalArea)

popValues.append(finalPop)

averagePop=round(sum(popValues)/len(popValues),2)

busStopLine=allValues[0][1]

busStopID=stopid

finalData=busStopLine,busStopID,averagePop

dataList.append(finalData)

defcreateCSV(data,csvname,mode='ab'):

withopen(csvname,mode)ascsvfile:

csvwriter=csv.writer(csvfile,delimiter=',')

csvwriter.writerow(data)

headers.insert(0,"ID")

createCSV(headers,csvname,'wb')

fordataindataList:

createCSV(data,csvname)

ThevariablexwasaddedtokeeptrackofthemembersofthedictionarydataDic,whichinthescriptinChapter6,WorkingwithArcPyGeometryObjectshadreliedontheSTOPIDfield.Toeliminatethisdependency,xwasintroduced.

RunningtheScriptToolNowthatthefrontendhasbeendesignedtoacceptparametersfromauser,andthebackendscriptisreadytoaccepttheparametersfromthefrontend,thetoolisreadytobeexecuted.DoubleclickontheScriptToolinthetoolboxtoopenthetooldialogbox.

SelecttheparametersaswithanyArcToolboxtool(forexampleusingthefiledialogtonavigateafiletreetotheBusStopfeatureclass).Oncetheparametershavebeenset,clickonOKtoexecutethescript.

Oneoptionaladjustmentwouldbetoaddanarcpy.AddMessagelinewheretheaveragepopulationiscalculated.Bydoingthis,theindividualblockpopulationwouldbedisplayedandthescriptconsolewouldgivefeedbackabouttheprogressofthescript.

InsertinthescriptjustbelowthelinewherethevariablefinalDataisdefined:

arcpy.AddMessage(finalData)

Thefeedbackprovidedbythislinewillmakeitobviousthatthescriptisworking,whichisusefulwhenthescriptexecutesalonganalysis.Whenperforminglonganalyses,itisgoodpracticetoprovidefeedbacktotheusersothattheycanseethatthescriptisworkingasexpected.Passnewlinecharacters(\n)asparameterstoarcpy.AddMessagewhenthereisalargeamountofdatabeingpassedtoarcpy.AddMessage.Thiswillbreakupthedataintodiscretechunksandmakeiteasiertoread.

SummaryInthischapter,welearnedhowtoconvertascriptintoapermanentandsharablescripttoolthatcanbeusedbyanArcGISuserwithnoprogrammingexperience.BycreatingafrontendusingthefamiliarArcGIStoolinterface,andthenpassingparameterstocustombuilttoolsthatemployArcPy,GISprogrammerscancombinetheeaseofuseoftheArcToolboxwiththepowerofPython.

Inthenextchapter,wewillexplorehowtouseArcPytocontroltheexportofmapsfrommapdocuments.Byadjustingmapelementssuchastitlesandlegends,wecancreatedynamicmapoutputstoreflectthenatureofthedataproducedbymapanalysis.InChapter9,MoreArcpy.MappingTechniqueswewilladdtheoutputofmapstoourscripttoolcreatedinthischapter.

Chapter8.IntroductiontoArcPy.MappingCreatingmapsisanart,onethatcanbelearnedthroughyearsofdedicatedstudyofcartography.Thevisualdisplayofinformationisbothexcitinganddifficult,andcanbearewardingpartofthedailyworkflowofgeospatialprofessionals.Oncethebasicshavebeenlearnedandmastered,cartographicoutputbecomesaconstantbattletoproducemoremapsatafasterpace.ArcPy,onceagain,hasapowerfulsolution:thearcpy.mappingmodule.

Byallowingfortheautomaticmanipulationofallmapcomponents,includingthemapwindow,thelayers,thelegend,andalltextelements,arcpy.mappingmakescreating,modifying,andoutputtingmultiplemapsfastandsimple.Mapbookcreation–anotherimportantskillforgeospatialprofessionals,isalsomadeeasyusingthemodule.Inthischapterwewilldiscusssomebasicfunctionalitiesavailablethrougharcpy.mappinganduseittooutputamapbookofbusstopsandtheirsurroundingcensusblocks.

Thischapterwillcoverthefollowingtopics:

InspectingandupdatingMapDocument(MXD)layerdatasourcesExportingMXDstoPDForotherimageformatsAdjustingmapdocumentelements

UsingArcPywithmapdocumentsRecognizingthelimitationsofthepreviousarcgisscriptingmodule,ESRIdesignedtheArcPymoduletonotonlyworkwithdatabutalsoincludedthearcpy.mappingmoduletoallowdirectinteractionwithmapdocuments(MXDs)andthelayerstheycontain.Thisnewmoduleopenedupamultitudeofmapautomationpossibilities.Ascriptmightaidinidentifyingbrokenlayerlinks,updatethedatasourceoftheselayers,andapplynewcolorschemestolayers.Anotherscriptmightuseamaptemplateandcreateasetofmaps,onefromeachfeatureclassinafeaturedataset.AthirdscriptcouldcreateamapbookfromanMXD,movingfromcelltocellinagridlayertooutputthepagesofthebook,orevencalculatingthecoordinatesonthefly.Dynamicallycreatedmaps,basedondatafromafreshanalysis,canbeoutputtedatthesametimethedataisproduced.Arcpy.mappingmovestheArcPymodulefromhelpfultoinstrumental,inanygeospatialworkflow.

Toinvestigatetheutilityofthearcpy.mappingmodule,we’llneedthehelpofanMXDtemplate.I’vepreparedamappackagecontainingthedataandMXDthatwewillusefortheexercisesinthischapter.ItincludesthedatafromourSanFranciscobusstop’sanalysis,whichwewillcontinueandextendtoincludemaps.

InspectingandreplacinglayersourcesThefirstandmostimportantarcpy.mappingmoduleuseistoidentifyandfixthebrokenlinksbetweenlayersinamapdocumentandtheirdatasources.LayersymbologyandGISdatastorageareseparated,meaningthatlayerdatasourcesareoftenmoved.Arcpy.mappingoffersaquicksolution,thoughimperfect.

Thissolutiondependsonanumberofmethodsincludedinthearcpy.mappingmodule.First,wewillneedtoidentifythebrokenlinks,andthenwewillfixthem.ToidentifythebrokenlinkswewillusetheListBrokenDataSources()methodincludedinarcpy.mapping.

TheListBrokenDataSources()methodrequiresanMXDpathtobepassedtotheMapDocument()methodofarcpy.mapping.Oncethemapdocumentobjecthasbeencreated,itispassedtotheListBrokenDataSources()method,andalistwillbegeneratedcontaininglayerobjects,oneforeachlayerwithabrokenlink.Thelayerobjectshaveanumberofpropertiesavailabletothem.Usingtheseproperties,let’sprintoutthenameanddatasourceofeachlayerusingthenameanddatasourcepropertiesofeachobject:

importarcpy

mxdPath='C:\Projects\MXDs\Chapter8\BrokenLinks.mxd'

mxdObject=arcpy.mapping.MapDocument(mxdPath)

brokenLinks=arcpy.mapping.ListBrokenDataSources(mxdObject)

forlinkinbrokenLinks:

printlink.name,link.dataSource

FixingthebrokenlinksNowthatwehaveidentifiedthebrokenlinks,thenextstepistofixthem.Inthiscase,itwasrevealedthatthedatasourcesshouldbeinafoldercalledData,buttheyarenotcontainedwithinthatfolder.Thescriptmustthenbesteppeduptoreplacethedatasourcesofeachlayer,sothattheypointattheactuallocationofthedatasource.

Therearemethodsincludedinbothlayerobjectsandmapdocumentobjectsthatcanaccomplishthisnextstep.IfallofthedatasourcesforanMXDhavebeenmoved,itisbettertousetheMXDobjectanditsmethodstofixthesources.IntheexampleMXD,thedatasourceshaveallbeenmovedintoanewfoldercalledNewData,sowewillemploythefindAndReplaceWorkspacePaths()methodtorepairthelinks:

oldPath=r'C:\Projects\MXDs\Data'

newPath=r'C:\Projects'

mxdObject.findAndReplaceWorkspacePaths(oldPath,newPath)

mxdObject.save()

Aslongasthedatasourcesarestillinthesameformat(suchthatshapefilesarestillshapefilesorfeatureclassesarestillfeatureclasses),thefindAndReplaceWorkspacePaths()methodwillwork.Ifthedatasourcetypeshavebeenchanged(suchthat,shapefilesareimportedintoafilegeodatabase),thereplaceWorkspaces()methodwillhavetobeusedinsteadasitrequiresworkspacetypeasaparameter:

oldPath=r'C:\Projects\MXDs\Data'

oldType='SHAPEFILE_WORKSPACE'

newPath=r'C:\Projects'

newType='FILEGDB_WORKSPACE'

mxdObject.replaceWorkspaces(oldPath,oldType,newPath,newType)

mxdObject.save()

FixingthelinksofindividuallayersIftheindividuallayersdonotshareadatasource,thelayerobjectswillneedtobeadjustedusingthefindAndReplaceWorkspacePath()methodavailabletolayers.Thismethodissimilartothemethodusedpreviously,butitwillonlyreplacethedatasourceofthelayerobjectitisappliedtoinsteadofallofthelayers.Whencombinedwithadictionary,thelayerdatasourcescanbeupdatedusingthelayernameproperty:

importarcpy

layerDic={'Bus_Stops':[r'C:\Projects\OldDataPath',r'C:\Projects'],

'stclines_streets':[r'C:\Projects\OtherPath',r'C:\Projects']}

mxdPath=r'C:\Projects\MXDs\Chapter8\BrokenLinks.mxd'

mxdObject=arcpy.mapping.MapDocument(mxdPath)

brokenLinks=arcpy.mapping.ListBrokenDataSources(mxdObject)

forlayerinbrokenLinks:

oldPath,newPath=layerDic[layer.name]

layer.findAndReplaceWorkspacePath(oldPath,newPath)

mxdObject.save()

Thesesolutionsworkwellforindividualmapdocumentsandlayers.TheycanalsobeextendedtofoldersfullofMXDsbyusingtheglob.glob()methodofthebuilt-inglobmodule(whichhelpstogeneratealistoffilesthatmatchacertainfileextension)andtheos.path.join()methodoftheosmodule:

importarcpy,glob,os

oldPath=r'C:\Projects\MXDs\Data'

newPath=r'C:\Projects'

folderPath=r'C:\Projects\MXDs\Chapter8'

mxdPathList=glob.glob(os.path.join(folderPath,'*.mxd'))

forpathinmxdPathList:

mxdObject=arcpy.mapping.MapDocument(mxdPath)

mxdObject.findAndReplaceWorkspacePaths(oldPath,newPath)

mxdObject.save()

ExportingtoPDFfromanMXDThenextmostimportantuseofarcpy.mappingistoautomaticallyexportMXDs.ThefollowingcodewillhighlighttheexportofPDFs,butnotethatthemodulealsosupportstheexportofJPEGsandotherimageformats.Usingarcpy.mappingforthisprocessisajoy,astheusualprocessofopeningandexportingtheMXDsinvolvesalotofwaitingforArcMaptostartandthemaptoload,whichcanbeatimesink:

importarcpy,glob,os

mxdFolder=r'C:\Projects\MXDs\Chapter8'

pdfFolder=r'C:\Projects\PDFs\Chapter8'

mxdPathList=glob.glob(os.path.join(mxdFolder,'*.mxd'))

formxdPathinmxdPathList:

mxdObject=arcpy.mapping.MapDocument(mxdPath)

arcpy.mapping.ExportToPDF(mxdObject,

os.path.join(pdfFolder,

basepath(

mxdPath.replace('mxd','pdf')

)))

NoteNotethattheoutputfoldermustexistforthiscodetoruncorrectly.Whilethereareosmodulemethodstocheckwhetherapathexists(os.path.exists)andtocreateafolder(os.mkdir),thatisnotincludedinthiscodesnippetandthearcpy.mapping.ExportToPDF()methodwillthrowanexceptioniftheinputoroutputpathsdonotexist.

Thisexamplecodeisveryusefulandcanbeconvertedintoafunctionthatwouldacceptthefolderpathasaparameter.Thefunctioncouldthenbeaddedtoascripttool,asdiscussedinthelastchapter.

AdjustingmapdocumentelementsArcpy.mappingincludesimportantmethodsthatwillfacilitatetheautomationofmapdocumentmanipulation.TheseincludetheabilitytoaddnewlayersorturnlayersonandoffwithinMXDs,theabilitytoadjustthescaleofthedataframeormoveadataframetofocusonaspecificregion,andtheabilitytoadjusttextcomponentsofthemap(suchastitlesorsubtitles).Thesemethodswillbeaddressedaswecontinueourbusstopanalysis.

OpenuptheMXDcalledMapAdjust.mxd.Thisrepresentsourbasemapdocument,withlayersandelementsthatwewilladjusttoourneeds.Itcontainslayersthatwehavegeneratedfromouranalysis,andthebaselayersthatfilloutthemap.Therearealsoanumberoftextelementsthatwillbeautomaticallyreplacedbythescripttofitthespecificneedsofeachmap.However,itdoesnotdoagoodjobofrepresentingtheresultsoftheanalysisasthecensusblocksthatintersectthebusstopbuffersoverlap,makingithardtointerpretthecartography.

Thescriptwillreplacethedatasourceofthecensusblocklayerandthebusstoplayertomakeitpossibletoonlyproduceonemapforeachbusstop,andthecensusblocksthatareintersectedwitheachbuffersurroundingthestops.

Tomakethispossible,wewillhavetocreatetwoemptyfeatureclasses:one,withalloftheattributesofthecensusblocks,andtheother,withtheattributesofthebusstops.Thiswillallowthedatasourcetobereplacedwiththedataproducedbytheanalysis.

OpenuptheSanFrancisco.gdbFileGeodatabaseandrightclickontheChapter8Resultsfeaturedataset.SelectNewfromthedrop-downmenuandthenselectFeatureClass.NamethefirstfeatureclassSelectedCensusBlocksandmakeitapolygon.Selectthedefaultskeywordonthenextmenu,andthenonthefollowingmenu,pushtheimportbutton.SelecttheCensusBlocksfeatureclassfromtheSanFranciscofeaturedataset;thiswillloadthefieldsintothenewfeatureclass.DothesamethingforasecondfeatureclasscalledSelectedBusStops,butmakesurethatitisapointgeometrytypeandimportthe

schemafromtheBusStopsfeatureclass.RepeatthesameprocessforathirdfeatureclasscalledSelectedStopBuffers,butmakesurethatitisapointgeometrytypeandimporttheschemafromtheBuffersfeatureclass.

Oncethefeatureclasseshavebeencreated,itisnowpossibletousethemtoloadtheresultsoftheanalysis.Wewillberedoingtheanalysisinmemoryandwritingouttheresultstothenewlycreatedfeatureclasses,sothattheentirecensusblockwillbecaptured,insteadofonlytheportionthatintersectswiththebuffer,asitwillbetterillustratetheresultsoftheanalysis.

TheinitialstateoftheMapAdjust.mxdmapdocumentfeaturesanumberoffeatureclasseswithwhichwearenowfamiliar:thedownloadedfeatureclassBus_Stops,thegeneratedfeatureclassBuffers,theintersectedandclippedCensusBlocks,andfourfeatureclassesusedforcartographicpurposes,namelytheStreetsfeatureclass,theParksfeatureclass,aNeighborhoodsfeatureclass,andanoutlineofSanFrancisco.Therearetwodataframes,onewiththedefaultnameLayersandanothercalledInset,thatareusedtocreatethesmallinsetthatwillshowthepositionoftheLayersdataframeasitmovesaroundSanFrancisco.ThesmallrectanglethatdepictstheextentoftheLayersdataframeisanExtentframecreatedinthepropertiesoftheInsetdataframe.

Hereisanexportedviewoftheinitialstateofthemapdocument:

Theideahere,istousetheinitialresultsofouranalysistogeneratethesymbologyofthepopulationlayeraswellasthebusstoplayerandthebufferlayer.Oncetheyhavebeensetandsaved,theycanbeusedasabasisfortheindividualmappagesthatwewillbeproducingfromthisbasicmapdocument.

NoteNotethetextelementsthatmakeupthetitleandsubtitle,aswellasthelegendandattributiontextatthebottomoftherightpane.Theseelementsareavailableforadjustmentalongwiththelayersanddatasourcesthatmakeupthemapdocumentbyusingthearcpy.mapping.ListElements()method.

AutomatedmapdocumentadjustmentNowthatweunderstandtheinitialconfigurationofthemapdocument,wewillintroduceascriptthatwillautomatetheadjustment.Thisscriptwillincludeanumberofconceptsthatwehavecoveredinthischapterandearlierchapters,andwillalsointroducesomenewmethodsformapdocumentadjustmentsthatwewilldetailinthefollowing:

importarcpy,os

dirpath=os.path.dirname

basepath=os.path.basename

Bus_Stops=r"C:\Projects\SanFrancisco.gdb\Bus_Stops"

selectedBusStop=

r'C:\Projects\SanFrancisco.gdb\Chapter8Results\SelectedBusStop'

selectedStopBuffer=

r'C:\Projects\SanFrancisco.gdb\Chapter8Results\SelectedStopBuffer'

CensusBlocks2010=r"C:\Projects\SanFrancisco.gdb\CensusBlocks2010"

selectedBlock=

r'C:\Projects\SanFrancisco.gdb\Chapter8Results\SelectedCensusData'

pdfFolder=r'C:\Projects\PDFs\Chapter8\Map_{0}'

bufferDist=400

sql="(NAME='71IB'ANDBUS_SIGNAG='FerryPlaza')"

mxdObject=arcpy.mapping.MapDocument("CURRENT")

dataFrame=arcpy.mapping.ListDataFrames(mxdObject,"Layers")[0]

elements=arcpy.mapping.ListLayoutElements(mxdObject)

forelinelements:

ifel.type=="TEXT_ELEMENT":

ifel.text=='TitleElement':

titleText=el

elifel.text=='SubtitleElement':

subTitleText=el

arcpy.MakeFeatureLayer_management(CensusBlocks2010,'blocks_lyr')

layersList=arcpy.mapping.ListLayers(mxdObject,"",dataFrame)

layerStops=layersList[0]

layerCensus=layersList[1]

layerBuffer=layersList[2]

layerBlocks=layersList[3]

iflayerBlocks.dataSource!=selectedBlock:

layerBlocks.replaceDataSource(dirpath(dirpath(layerBlocks.dataSource)),

'FILEGDB_WORKSPACE',basepath(selectedBlock))

iflayerStops.dataSource!=selectedBusStop:

layerStops.replaceDataSource(dirpath(dirpath(layerStops.dataSource)),

'FILEGDB_WORKSPACE',basepath(selectedBusStop))

iflayerBuffer.dataSource!=selectedStopBuffer:

layerBuffer.replaceDataSource(dirpath(dirpath(layerBuffer.dataSource)),

'FILEGDB_WORKSPACE',basepath(selectedStopBuffer))

layerStops.visible=True

layerBuffer.visible=True

layerCensus.visible=False

witharcpy.da.SearchCursor(Bus_Stops,['SHAPE@','STOPID','NAME',

'BUS_SIGNAG','OID@','SHAPE@XY'],sql)

ascursor:

forrowincursor:

stopPointGeometry=row[0]

stopBuffer=stopPointGeometry.buffer(bufferDist)

witharcpy.da.UpdateCursor(layerBlocks,['OID@'])asdcursor:

fordrowindcursor:

dcursor.deleteRow()

arcpy.SelectLayerByLocation_management('blocks_lyr','intersect',

stopBuffer,"","NEW_SELECTION")

witharcpy.da.SearchCursor('blocks_lyr',['SHAPE@','POP10','OID@'])

asbcursor:

inCursor=arcpy.da.InsertCursor(selectedBlock,

['SHAPE@','POP10'])

fordrowinbcursor:

data=drow[0],drow[1]

inCursor.insertRow(data)

delinCursor

witharcpy.da.UpdateCursor(selectedBusStop,['OID@'])asdcursor:

fordrowindcursor:

dcursor.deleteRow()

inBusStopCursor=arcpy.da.InsertCursor(selectedBusStop,['SHAPE@'])

data=[row[0]]

inBusStopCursor.insertRow(data)

delinBusStopCursor

witharcpy.da.UpdateCursor(selectedStopBuffer,['OID@'])asdcursor:

fordrowindcursor:

dcursor.deleteRow()

inBufferCursor=arcpy.da.InsertCursor(selectedStopBuffer,

['SHAPE@'])

data=[stopBuffer]

inBufferCursor.insertRow(data)

delinBufferCursor

layerStops.name="Stop#{0}".format(row[1])

arcpy.RefreshActiveView()

dataFrame.extent=arcpy.Extent(row[-1][0]-1200,row[-1][1]-1200,

row[-1][0]+1200,row[-1][1]-1200)

subTitleText.text="Route{0}".format(row[2])

titleText.text="BusStop{0}".format(row[1])

outPath=pdfFolder.format(str(row[1])+"_"+str(row[-2]))+

'.pdf'

printoutPath

arcpy.mapping.ExportToPDF(mxdObject,outPath)

titleText.text='TitleElement'

subTitleText.text='SubtitleElement'

arcpy.RefreshActiveView()

Wow!That’salotofcode.Let’sreviewitsectionbysectiontoaddresswhateachpartofthescriptisdoing.

ThiscodewillberuninthePythonWindowoftheMXD,somakesuretoopentheMXD.Onceitis,openthePythonWindowandrightclickinit,andthenselectLoadfromtheright-clickmenu.Usingthefilenavigationbrowser,findthescriptcalledChapter8_6_AdjustmapCURRENT.pyandselectitbyclickingonit.PushOKanditwillloadinthePythonWindow.PushingEnterwillexecutethescript,orusethescrollbartoperusetheloadedlines.

ThevariablesWithinthescript,anumberofvariablesarefirstcreatedtoholdthestringfilepaths,theintegerbufferdistance,andthesqlstatementusedtoidentifythebuslineofinterest:

importarcpy,os

Bus_Stops=r"C:\Projects\SanFrancisco.gdb\Bus_Stops"

selectedBusStop=

r'C:\Projects\SanFrancisco.gdb\Chapter8Results\SelectedBusStop'

selectedStopBuffer=

r'C:\Projects\SanFrancisco.gdb\Chapter8Results\SelectedStopBuffer'

CensusBlocks2010=r"C:\Projects\SanFrancisco.gdb\CensusBlocks2010"

selectedBlock=

r'C:\Projects\SanFrancisco.gdb\Chapter8Results\SelectedCensusData'

pdfFolder=r'C:\Projects\PDFs\Chapter8\Map_{0}'

bufferDist=400

sql="(NAME='71IB'ANDBUS_SIGNAG='FerryPlaza')"

Thesewillbeusedlatertoallowustosearchthelayersandperformanalysisonthem.

ThemapdocumentobjectandthetextelementsBecausethiscodewillbeexecutedinanopenmapdocument,wedon’thavetopassanMXDfilepathtothearcpy.mapping.MapDocument()method.Instead,wewillusethekeywordCURRENTtoindicatethatwearereferencingtheopenmapdocument:

mxdObject=arcpy.mapping.MapDocument("CURRENT")

dataFrame=arcpy.mapping.ListDataFrames(mxdObject,"Layers")[0]

elements=arcpy.mapping.ListLayoutElements(mxdObject)

forelinelements:

ifel.type=="TEXT_ELEMENT":

ifel.text=='TitleElement':

titleText=el

elifel.text=='SubtitleElement':

subTitleText=el

Oncethemapdocumentobjecthasbeencreated,theLayersdataframeisselectedfromalistofdataframesusingtheListDataFrames()methodandpassedtothevariablecalleddataFrame.

Next,thelayoutelementsarepassedasalisttotheelementsvariableusingtheListLayoutElements()method.Thelayoutelementsincludethevariouselementsofthemapdocumentlayoutview:thelegend,theneatlines,thenortharrow,thescalebar,andthetextelementsusedastitlesanddescriptions.Unfortunately,thereisnoniceordertothelistreturned,astheirpositionthroughoutthelayoutisundetermined.Accesstothetextelements,whichwewouldliketoassigntoavariableforlateruse,mustbeidentifiedusingtwopropertiesoftheelementobjects:thetypeandthetext.Wewanttoadjustthetitleandsubtitleelements,soaforloopisusedtosearchthroughthelistofelementsandthepropertiesareusedtofindtheelementsofinterest.

ThelayerobjectsTheMakeFeatureLayertool,partoftheDataManagementtoolset,isusedtocopydatafromdiskintomemoryasalayer.ArcGISrequiresthegenerationoflayerstoperformselectionsandoperationsondata,insteadofoperatingonthefeatureclassesdirectly.Byusinglayerstoperformtheseoperations,thesourcefeatureclassesareprotected.

TheMakeFeatureLayertoolisaccessedusingArcPy’sMakeFeatureLayer_management()method.WhenusingthistoolinthePythonWindow,theresultisaddedtothemapdocumentasalayerthatwillbevisibleintheTableofContents.WhenthetoolisnotusedinthePythonWindowinArcMap,theresultinglayerisonlygeneratedinmemoryandisnotaddedtothemapdocument.

Intheportionofthefollowingcode,alayercalledblocks_lyrisgeneratedinmemorybypassingthefilepathofthecensusblocksfeatureclass.ThelayerobjectscontainedwithintheinitialMXDarethenaccessedusingtheListLayers()methodofthearcpy.mapping()module.TheyarereturnedintheorderthattheyarelistedintheTableofContentsofthemapdocumentandareassignedtovariablesusinglistindexing,includingthenewlycreatedblocks_lyr:

arcpy.MakeFeatureLayer_management(CensusBlocks2010,'blocks_lyr')

layersList=arcpy.mapping.ListLayers(mxdObject,"",dataFrame)

layerStops=layersList[0]

layerCensus=layersList[1]

layerBuffer=layersList[2]

layerBlocks=layersList[3]

ReplacingthedatasourcesNowthatwehaveassignedthelayerobjectstovariables,wewillcheckwhethertheirdatasourcesarethecorrectfeatureclassesthatweuseformapproduction.UsingthedataSourcepropertyofeachlayerobject,theyarecomparedtothefilepathvariablesthatwewanttouseasdatasources:

iflayerBlocks.dataSource!=selectedBlock:

layerBlocks.replaceDataSource(dirpath(dirpath(layerBlocks.dataSource)),

'FILEGDB_WORKSPACE',basepath(selectedBlock))

iflayerStops.dataSource!=selectedBusStop:

layerStops.replaceDataSource(dirpath(dirpath(layerStops.dataSource)),

'FILEGDB_WORKSPACE',basepath(selectedBusStop))

iflayerBuffer.dataSource!=selectedStopBuffer:

layerBuffer.replaceDataSource(dirpath(dirpath(layerBuffer.dataSource)),

'FILEGDB_WORKSPACE',basepath(selectedStopBuffer))

Ifstatementsareusedtocheckwhetherthedatasourcesarecorrect.Ifnot,theyarereplacedwiththecorrectdatasourcesusingthereplaceDataSource()layermethod.Thismethodrequiresthreeparameters:theworkspace(inthiscase,theFileGeodatabase),theworkspacetype,andthenameofthenewfeatureclassdatasource,whichmustbeinthesameworkspaceforthereplaceDataSource()methodtowork(thoughitdoesnotneedtobeinthesamefeaturedataset).

AdjustinglayervisibilityThelayerobjectshaveapropertythatallowsustoadjusttheirvisibility.SettingthisBooleanpropertytoTrueorFalsewilladjustthelayer’svisibilityon(True)oroff(False):

layerStops.visible=True

layerBuffer.visible=True

layerCensus.visible=False

WewantthelayervariablelayerCensus,whichisthenewblocks_lyrobject,tobeturnedoff,soitissettoFalse,butthebusstopsandbufferlayerobjectsneedtobevisible,sotheyaresettoTrue.

GeneratingabufferfromthebusstopsfeatureclassAllofthevariableshavebeengeneratedorassigned,sothenextstepistouseaSearchCursortosearchthroughtheselectedbusstops.Foreachbusstop,bufferobjectswillbegeneratedtofindcensusblocksthatintersectwiththeseindividualbusstops:

witharcpy.da.SearchCursor(Bus_Stops,['SHAPE@','STOPID','NAME',

'BUS_SIGNAG','OID@','SHAPE@XY'],sql)

ascursor:

forrowincursor:

stopPointGeometry=row[0]

stopBuffer=stopPointGeometry.buffer(bufferDist)

witharcpy.da.UpdateCursor(layerBlocks,['OID@'])as

dcursor:

fordrowindcursor:

dcursor.deleteRow()

ForeachrowofdataretrievedfromtheBusStopsfeatureclass,anumberofattributesarereturned,containedinatuple.Thefirstofthese,row[0],isaPointGeometryobject.ThisobjecthasabuffermethodthatisusedtogenerateabufferPolygonobjectinmemory,whichisthenassignedtothestopBuffervariable.Oncethebufferobjectiscreated,thedataaccessUpdateCursor’sdeleteRow()methodisusedtoerasetherowsinthecensusblockslayer.Oncetherowshavebeendeleted,thelayercanthenberepopulatedwithnewlyselectedcensusblocksthatwillbeidentifiedinthenextsection.

IntersectingthebusstopbufferandcensusblocksToidentifythecensusblocksintersectingwiththebufferaroundeachbusstop,theArcToolboxtoolSelectLayerByLocationisinvokedusingtheArcPymethodSelectLayerByLocation_management():

arcpy.SelectLayerByLocation_management('blocks_lyr','intersect',

stopBuffer,"","NEW_SELECTION")

witharcpy.da.SearchCursor('blocks_lyr',['SHAPE@',

'POP10','OID@'])asbcursor:

inCursor=arcpy.da.InsertCursor(selectedBlock,['SHAPE@',

'POP10'])

fordrowinbcursor:

data=drow[0],drow[1]

inCursor.insertRow(data)

delinCursor

Thismethodrequiresthein-memoryblocks_lyrlayerobjectandthenewlycreatedbufferobjectassignedtothevariablestopBuffer.Italsorequiresthetypeofselectionintersectandanotherparameterthatcontrolswhethertheselectionwillbeaddedtoanexistingselectionorwillbeanewselection.Inthiscase,wewantanewselection,asonlythecensusblocksthatintersectthecurrentbusstopareneeded.

Oncethecensusblockshavebeenselectedandidentified,theshapedataandpopulationdataispassedtothefeatureclassrepresentedbythevariableselectedBlockusinganInsertCursor.TheInsertCursormustbedeletedusingthedelkeyword,asonlyoneInsertCursororUpdateCursorcanbeinmemoryatatime.

PopulatingtheselectedbusstopandbufferfeatureclassesInasimilarmanner,thenextstepistopopulatethebusstopandbufferfeatureclassesthatwillbeusedinthemapproduction.ThebusstopsfeatureclassisfirstmadeblankusingthedeleteRow()method,andthentheselectedbusstopshapefielddataisinsertedintothefeatureclass.Thesamestepsarethentakenwiththebusstopbuffersfeatureclassandthebuffergeometryobject:

witharcpy.da.UpdateCursor(selectedBusStop,['OID@'])asdcursor:

fordrowindcursor:

dcursor.deleteRow()

inBusStopCursor=arcpy.da.InsertCursor(selectedBusStop,['SHAPE@'])

data=[row[0]]

inBusStopCursor.insertRow(data)

delinBusStopCursor

witharcpy.da.UpdateCursor(selectedStopBuffer,['OID@'])asdcursor:

fordrowindcursor:

dcursor.deleteRow()

inBufferCursor=arcpy.da.InsertCursor(selectedStopBuffer,['SHAPE@'])

data=[stopBuffer]

inBufferCursor.insertRow(data)

delinBufferCursor

UpdatingthetextelementsNowthatthedatahasbeengeneratedandwrittentothefeatureclassescreatedtoholdthem,thenextstepistoupdatethelayoutelements.Thisincludeslayerpropertiesthatwillaffectthelegend,theextentofthedataframe,andthetextelements:

layerStops.name="Stop#{0}".format(row[1])

dataFrame.extent=arcpy.Extent(row[-1][0]-1200,row[-1][1]-1200,

row[-1][0]+1200,row[-1][1]-1200)

subTitleText.text="Route{0}".format(row[2])

titleText.text="BusStop{0}".format(row[1])

arcpy.RefreshActiveView()

Thenameofthebusstopslayerisadjustedusingitsnamepropertytoreflectthecurrentbusstop.Thedataframeextentisadjustedbycreatinganarcpy.Extentobjectandpassingitfourparameters:Xmin,Ymin,Xmax,Ymax.TogeneratethesevaluesIhaveusedthesomewhatarbitraryvalueof1200feettocreateasquarearoundthebusstop.Thetextelementsareupdatedusingtheirtextproperty.Finally,theRefreshActiveView()methodisusedtoensurethatthemapdocumentwindowiscorrectlyupdatedtothenewextent.

ExportingtheadjustedmaptoPDFThefinalstepistopassthenewlyadjustedmapdocumentobjecttoArcPy’sExportToPDFmethod.Thismethodrequirestwoparameters,themapdocumentobjectandastringthatrepresentsthefilepathofthePDF:

outPath=pdfFolder.format(str(row[1])+"_"+str(row[-2]))+'.pdf'

arcpy.mapping.ExportToPDF(mxdObject,outPath)

titleText.text='TitleElement'

subTitleText.text='SubtitleElement'

arcpy.RefreshActiveView()

ThePDFfilepathstringisgeneratedfromthepdfFolderstringtemplateandtheIDofthebusstop,alongwiththeobjectIDandthefileextension.pdf.OncethatandthemapdocumentobjectrepresentedbythevariablemxdObjectarepassedtotheExportToPDFmethod,thePDFwillbegenerated.Thetextelementsarethenresetandtheviewisrefreshedtoensurethatthemapdocumentwillbereadyforthenexttimethescriptisused.

RunningthescriptinthePythonWindowOpenupthemapdocumentcalledMapAdjust.mxdifitisnotopenalready.OpenthePythonWindowandrightclickinthewindow.SelectLoadfromthemenu.Whenthefiledialogopens,findthescriptcalledChapter8_6_AdjustmapCURRENT.pyandselectit,makingsurethatthefilepathswithinitarecorrect.PushOKanditwillloadinthePythonWindow.PushEnteroncethescriptisloadedtorunthescript.Itcantakeafewsecondsormoreforittobeobviousthatthescriptisrunning.

NoteNotethatthePythonWindowisnotagreatplacetoexecuteArcPyscriptsinmostcases,asitissomewhatlimitedwhencomparedtoIDEs.UsingittoloadandexecuteascriptthatperformsthesemapdocumentadjustmentsisoneofthebestusesofthePythonWindow.

Oncethescriptisrunning,theadjustmentstothemapdocumentwillbegintoappearandrepeat.Thisisafascinatingprocess,astheeffectsofrunningthescriptarevisibleinamannerthatisnotreadilyavailablewhenrunningPythonscripts.OncethePDFsbegintobegenerated,openoneuptoviewtheoutput.Thescriptwillgenerateamapforeachbusstopontheselectedbusline,sofeelfreetoshutdownthemapdocumentaftergeneratingasetnumberofthePDFs.

Hereisanexampleoftheoutput:

Themapsgeneratedbythescriptshoweachbusstopatthecenter,surroundedbythebufferandthesymbolizedcensusblockswithwhichthebufferintersects.Thetitle,subtitleandthelegendhavebeenadjustedtoindicatethebusstopdepictedinthemap.WithArcPy,wearenowincontrolofboththepartsofgeospatialanalysis:theanalysisitself,andthecartographicproductiondepictingtheresultoftheoutput.

SummaryInthischapterarcpy.mappingwasintroducedandusedtocontroltheelementsofmapdocumentsthatneedtobeadjustedtocreatecustommaps.Byjoininggeospatialanalysisandmapproductiontogether,weareclosertoutilizingthefullpowerofArcPy.

Inthenextchapter,wewillgofurtherwitharcpy.mappingandcreateascripttoolthatcanbeaddedtoArcToolbox,whichwillruntheanalysisaswellasgeneratemapsfromtheresultingdata.WewillalsorefinethescriptandintroduceDataDrivenPagestodiscusshowthatpowerfultoolcanbeusedinanArcPyscript.

Chapter9.MoreArcPy.MappingTechniquesTheabilitytocontrolmapdocumentcartography,whilealsorunninggeospatialanalyses,increasesthepowerandusefulnessofArcPy.Thepropertiesandmethodsofarcpy.mappingcanbeutilizedtomanipulatelayerobjects,mapscalesanddataframeextents,oreventosetdefinitionqueries.Bycombiningautomatedgeospatialanalysiswithdynamicmapproduction,scriptedmappingsystemsaremadepossible.Thischapterwillcoverthefollowingtopics:

Arcpy.mappingLayerobjectsLayerobjectdefinitionqueriesandextentsArcpy.mappingDataFrameobjectsCreatingdynamicallyscaledmaps

Usingarcpy.mappingtocontrolLayerobjectsArcpy.mappingLayerobjectsareusedtocontrolthepropertiesoflayerswithinmapdocumentdataframes.Turninglayervisibilityonandoff,addingnewlayers,andadjustinglayerordercanallbeaccomplishedusingLayerobjectproperties.

CreatingLayerobjectsinvolvespassingparameterstothearcpy.mapping.ListLayers()method.AsdiscussedinChapter8,IntroductiontoArcPy.Mapping,whenreferencinganarcpy.mapping.MapDocumentobject,thelayerswithinthemapdocumentcanbeaccessedusingzero-basedindexing.ThiscodewillprintthelistofLayerobjectscontainedwithinthedataframecalledLayersinanMXD:

importarcpy

mxdPath=r'C:\Projects\MXDs\Chapter9\MapDocument1.mxd'

mxdObject=arcpy.mapping.MapDocument(mxdPath)

dataFrame=arcpy.mapping.ListDataFrames(mxdObject,"Layers")[0]

layersList=arcpy.mapping.ListLayers(mxdObject,"",dataFrame)

printlayersList

ThelayerswithinthedataframecalledLayers,havebeenassignedtothevariablelayersListusingtheListLayers()method.EachlayerinlayersListcanbeaccessedusingzero-basedindexing.Oncethelayershavebeenaccessedwithinthelistandeitherassignedtoavariableorplacedinsideaforloop,thepropertiesandmethodsoftheLayerobjectscanbeutilized.

NoteThesecondparameteroftheListLayersmethodisemptyhere,butdoesnothavetobe.ItisawildcardparameterthatwilllimitthereturnedLayerobjectstothosethatmatchthepatternofthewildcard.Forinstance,*StopswouldreturnalllayerswiththenameStopsattheend.Multipleasteriskscanbeusedtofindlayerswiththewordatthebeginning,middle,orendofthelayername.

LayerobjectmethodsandpropertiesLayerobjectpropertiesandmethodscaneitherbereadonly,meaningtheycanbecheckedbutnotadjusted,ortheyarereadandwrite,meaningtheycanbeadjustedwithinthescript.Let’sexploreanumberofthesepropertiesandmethods,andseehowtheycanbeusedtocontrolthelookandfeelofthemapsproducedfromthemapdocument,aswellasthedatafromthescriptanalysis.

DefinitionqueriesAnimportantpropertyofLayerobjectsistheabilitytodynamicallysetdefinitionqueries.AdefinitionqueryisaSQLstatementwhereclausethatlimitsthedataavailablefordisplay,query,orotherdataoperations(buffers,intersections,etc.)toonlytherowsthatmatchthewhereclause.DefinitionqueriescouldbesetinanMXDbyopeningalayer’spropertiesmenuandusingtheDefinitionQuerytab,buthereweareconcernedwithhowtoaddthemprogrammatically.Followingisanexampleofhowtodothis:

layersList=arcpy.mapping.ListLayers(mxdObject,"",dataFrame)

busStops=layersList[0]

busStops.definitionQuery="NAME='71IB'ANDBUS_SIGNAG='FerryPlaza'"

ThisvaluablepropertycanbeutilizedtoreformatthecodefromChapter8,IntroductiontoArcPy.Mapping.RememberthecomplicatedsecondportionoftheChapter8_6.pyscript,whereeachbusstopalongthe71Inboundlineisselectedanditsgeometryiswrittentoanotherfeatureclass?Instead,wecanuseLayerobjectsanddefinitionqueriestoperformthesametypeofgeometryoperation.Let’sexaminehowthefirstpartofthatoperation(selectingthebusstopgeometryandcreatingabufferaroundit)lookswhenadefinitionqueryisused:

importarcpy

bufferDist=400

mxdPath=r'C:\Projects\MXDs\Chapter9\MapDocument1.mxd'

mxdObject=arcpy.mapping.MapDocument(mxdPath)

dataFrame=arcpy.mapping.ListDataFrames(mxdObject,"Layers")[0]

layersList=arcpy.mapping.ListLayers(mxdObject,"",dataFrame)

busStops=layersList[0]

defQuery="NAME='71IB'ANDBUS_SIGNAG='FerryPlaza'"

busStops.definitionQuery=defQuery

idList=[]

witharcpy.da.SearchCursor(busStops,['OID@'])ascursor:

forrowincursor:

idList.append(row[0])

foroidinidList:

newQuery="OBJECTID={0}".format(oid)

printnewQuery

busStops.definitionQuery=newQuery

witharcpy.da.SearchCursor(busStops,

['SHAPE@','STOPID','NAME','BUS_SIGNAG','OID@','SHAPE@XY'])ascursor:

forrowincursor:

stopPointGeometry=row[0]

stopBuffer=stopPointGeometry.buffer(bufferDist)

Inthisexample,thedefinitionqueryisusedtolimitthepotentialresultsfromtheSearchCursortothebusstopspecifiedbythequery.However,thisisoverlycumbersomeandthedefinitionquerydoesn’taddmuch,asfirstanotherSearchCursorisneededtoextracttheObjectIDinformationfromthebusStopslayer.ThiscomplicatesthecodewhenonlyoneSearchCursorisnecessary.

Definitionqueriesshouldbeusedtoselecttheblocksthatintersectwiththebuffer,asthis

willeliminatetheneedtousethecomplicatedSearchCursorandInsertCursorsetupthatwasemployedinChapter8,IntroductiontoArcPy.Mapping.Let’sreformulatethecodesothatdefinitionqueriesareproperlyusedonthecensusblockLayerobject.

ThefirststepistoaddsomecodethatwillgeneratetheSQLstatementthatwillbeusedasthedefinitionquery:

importarcpy

bufferDist=400

mxdPath=r'C:\Projects\MXDs\Chapter9\MapDocument1.mxd'

mxdObject=arcpy.mapping.MapDocument(mxdPath)

dataFrame=arcpy.mapping.ListDataFrames(mxdObject,

"Layers")[0]

layersList=arcpy.mapping.ListLayers(mxdObject,"",dataFrame)

busStops=layersList[0]

censusBlocks=layersList[3]

sql="NAME='71IB'ANDBUS_SIGNAG='FerryPlaza'"

witharcpy.da.SearchCursor(busStops,['SHAPE@','STOPID','NAME',

'BUS_SIGNAG','OID@'],sql)ascursor:

forrowincursor:

busQuery='OBJECTID={0}'.format(row[-1])

busStops.definitionQuery=busQuery

stopPointGeometry=row[0]

stopBuffer=stopPointGeometry.Buffer(bufferDist)

arcpy.SelectLayerByLocation_management(censusBlocks,'intersect',stopBuffer,

"","NEW_SELECTION")

blockList=[]

witharcpy.da.SearchCursor(censusBlocks,

['OID@'])asbcursor:

forbrowinbcursor:

blockList.append(brow[0])

newQuery='OBJECTIDIN('forCOUNTER,oidinenumerate(blockList):

ifCOUNTER<len(blockList)-1:

newQuery+=str(oid)+','

else:

newQuery+=str(oid)+')'

printnewQuery

Inthissection,thecodeassignsthecensusblockslayerintheMXDtothevariablecensusBlocks.ThebusstopsSearchCursoristhencreated,andthe400footbufferisgeneratedforeachrowtoselectthecensusblockssurroundingthebusstop.Oncethecorrectblockshavebeenselected,asecondSearchCursorisusedonthecensusBlocksLayerobjecttofindtheObjectID(usingtheOID@token)oftheselectedblocks.TheObjectIDsarethenappendedtothelistcalledblockList.

ThislististheniteratedinaforlooptogenerateastringSQLstatement.UsingtheinitialstringassignedtothevariablenewQuery,theforloopwilladdtheObjectIDsofeachselectblocktothestringtocreateavalidSQLstatement.Theforloopusesthefunctionenumeratetocountthenumberofloopsthattheforloopperforms;thisallowsforanif/thenstatementtobeused.Theif/thenstatementdetermineswhatcomesaftertheObjectIDinthestring,aseachObjectIDmustbeseparatedbyacomma,exceptforthefinalObjectID,whichmustbefollowedbytheclosingparenthesis.Theforloopproduces

aSQLstatementsimilartothisexample:

OBJECTIDIN(910,1664,1812,1813,2725,6382)

Theprintstatementattheendisusedtodemonstratetheresultsofthissectionofthecode,andalsotogivethatwarmfuzzyfeelingthatcomesfromseeingtheresultsofthecodeworking.OncewearesurethatthecodeisgeneratingvalidSQLstatements(closedparenthesisandcommaseparatedObjectIDs),thenextstepistoassignthedefinitionquerytothecensusBlocksLayerobjectandusetheresulttogenerateamapofthearea.

ControllingthedataframewindowextentandscaleInChapter8,IntroductiontoArcPy.Mappingwestartedtoexplorethepropertiesandmethodsofthedataframe.Usingthearcpy.Extentobject,wewereabletosettheextentofthedataframetoanextentthatwashard-codedintothescript.However,thisdoesnotalwayscapturetheentireextentoflargecensusblocks.Usingacombinationofdefinitionqueriesandthedataframeextentandscaleproperties,wecanavoidtheseunwantedresults.

Therearetwodataframeobjectmethodsusedtoshiftthedataframewindowtotheareaofinterest,inthiscasetheselectedcensusblocks.Thefirst,whichwearenotusinghere,isdataFrame.zoomToSelectedFeatures.Thesecond,istoassignthedataframe’sextentpropertytotheextentofthecensusblocklayerafterthedefinitionqueryhasbeenassignedtoit.

Ipreferthesecondmethod,asitwillworkevenwhenthereisnoselectedcensusblocks.Also,asthemapsthatareproducedbythisscriptshouldnotshowtheselectionoftheblocks,wewillhavetoaddcodetoexplicitlycleartheselectiononcethecorrectcensusblockshavebeenidentified:

censusBlocks.definitionQuery=newQuery

dataFrame.extent=censusBlocks.getExtent()

arcpy.SelectLayerByAttribute_management(censusBlocks,

"CLEAR_SELECTION")

Thedefinitionqueryhasmadeiteasytomovethedataframewindowtotheareaofinterest,astheextentrectangle(orenvelope)ofthelayerisnowonlyaroundthespecifiedblocksandthedataFrameextentpropertycanbesettotheextentrectangle.However,thisisnotalwayscartographicallydesirableasitseemsbettertomovethedataframewindowbackfromtheextentrectangle.Todothat,we’llaccessthedataframetheobject’sscaleproperty.

Thescalepropertycanbesettobeamultiplierofthecurrentscaletoavoidhard-codinganyspecificdistanceswhenadjustingthedataframeextent.Whenusingthescaleproperty,itisimportanttoremembertousethearcpy.RefreshActiveView()method,asitwillrefreshthedataframewindowtothenewscale.

dataFrame.scale=dataFrame.scale*1.1

arcpy.RefreshActiveView()

Asthedataframeextentwassetinthefewlinesbeforethis,thecurrentscalerepresentstheenvelopeoftheselectedcensusblocks.Toadjustit,assessthepropertyandapplyamultiplier.Inthiscase,themultiplieris1.1,butitcouldbeanyvalue.Thismakestheresultingmaplookbetterbygivingtheanalysisresultssomebackgroundcontext.

AddingaLayerobjectThelaststepbeforeexportingoutthemapsistoaddthe400footbufferscreatedaboveasalayertothedataframeobject.Toaccomplishthis,weneedtocreateasymbolizedlayeraheadoftimeandcopyitssymbologytoensureitlooksasdesired.ThiswillbeaddedtotheMXDasaplaceholderlayer,andassignedtothebufferLayervariableinthescript.

1. OpenupanMXDandaddthebusstopfeatureclass.2. RuntheBufferToolintheProximitytoolsetintheAnalysistoolsetofthe

ArcToolbox,addingthebusstopfeatureclassastheinputandsettingthebuffersizeto400feet.Afterthetoolhasrun,openthepropertiesofthebufferlayerandsymbolizethelayerasdesired.

3. Oncethelayerhasbeensymbolized,right-clickonthelayerandselectSaveAsLayerFile.

4. SavethelayerinafolderandclosetheMXD.5. OpenuptheMapDocument1.mxdmapdocumentandaddthelayerusingtheAdd

Databutton.6. Makesuretochangethenameto400FootBufferandtoaddittothelegendabove

thePopulationsection.7. Inthescript,assignthebufferlayertothevariablebufferLayer.8. Lowerinthescript,inthebusstopSearchCursor,addtheselinesbelowwherethe

bufferisgeneratedaroundthebusstopgeometry:

arcpy.CopyFeatures_management(stopBuffer,

r"C:\Projects\Output\400Buffer.shp")

bufferLayer.replaceDataSource(r"C:\Projects\Output","SHAPEFILE_WORKSPAC

E","400Buffer")

ThesetwolinescopythebuffergeneratedtodiskasashapefileandthenreplacethedatasourceofthebufferLayerLayerobjectwiththenewlycreatedbuffer.Notethatthenameoftheshapefiledoesnotincludethe.shpextension;theSHAPEFILE_WORKSPACEparametermakesthisunnecessary.

NoteTomakesurethateachnewbuffershapefilecanbewrittenoveranexistingshapefile,addthefollowinglinebelowtheimportarcpylinetomakesurethatfilescanbeoverwritten:

arcpy.env.overwriteOutput=1

ExportingthemapsThefinalstepofthisscriptistoexportthemapsoftheareasurroundingeachbusstop.Todothis,wewillborrowsomecodefromthescriptChapter8_6_AdjustMap.pyandaddthewholescripttoafilecalledChapter9.py.Thiscodewillidentifyandadjustthetitleandsubtitleelements,makingitpossibletocustomizeeachresultingPDF:

importarcpy

arcpy.env.overwriteOutput=1

bufferDist=400

pdfFolder=r'C:\Projects\PDFs\Chapter9\Map_{0}'

mxdPath=r'C:\Projects\MXDs\Chapter9\MapDocument1.mxd'

mxdObject=arcpy.mapping.MapDocument(mxdPath)

dataFrame=arcpy.mapping.ListDataFrames(mxdObject,"Layers")[0]

elements=arcpy.mapping.ListLayoutElements(mxdObject)

forelinelements:

ifel.type=="TEXT_ELEMENT":

ifel.text=='TitleElement':

titleText=el

elifel.text=='SubtitleElement':

subTitleText=el

layersList=arcpy.mapping.ListLayers(mxdObject,

"",dataFrame)

busStops=layersList[0]

bufferLayer=layersList[2]

censusBlocks=layersList[4]

sql="NAME='71IB'ANDBUS_SIGNAG='FerryPlaza'"

witharcpy.da.SearchCursor(busStops,['SHAPE@',

'STOPID',

'NAME',

'BUS_SIGNAG',

'OID@'],sql)ascursor:

forrowincursor:

busQuery='OBJECTID={0}'.format(row[-1])

busStops.definitionQuery=busQuery

stopPointGeometry=row[0]

stopBuffer=stopPointGeometry.buffer(bufferDist)

arcpy.CopyFeatures_management(stopBuffer,r"C:\Projects\Output\400Buffer.shp

")

bufferLayer.replaceDataSource(r"C:\Projects\Output",

"SHAPEFILE_WORKSPACE",

"400Buffer")

arcpy.SelectLayerByLocation_management(censusBlocks,

'intersect',

stopBuffer,

"",

"NEW_SELECTION")

blockList=[]

witharcpy.da.SearchCursor(censusBlocks,

['OID@'])asbcursor:

forbrowinbcursor:

blockList.append(brow[0])

newQuery='OBJECTIDIN('

forCOUNTER,oidinenumerate(blockList):

ifCOUNTER<len(blockList)-1:

newQuery+=str(oid)+','

else:

newQuery+=str(oid)+')'

printnewQuery

censusBlocks.definitionQuery=newQuery

dataFrame.extent=censusBlocks.getExtent()

arcpy.SelectLayerByAttribute_management(censusBlocks,

"CLEAR_SELECTION")

dataFrame.scale=dataFrame.scale*1.1

arcpy.RefreshActiveView()

subTitleText.text="Route{0}".format(row[2])

titleText.text="BusStop{0}".format(row[1])

outPath=pdfFolder.format(str(row[1]))+'.pdf'

printoutPath

arcpy.mapping.ExportToPDF(mxdObject,outPath)

titleText.text='TitleElement'

subTitleText.text='SubtitleElement'

censusBlocks.definitionQuery=''

busStops.definitionQuery=''

SummaryInthischapter,wecoveredtheuseoflayerdefinitionqueries,dataframeextentsandscales,andlayersourcereplacementtoeasetheproductionofmaps.Byusingdefinitionqueries,thelayerscanbemodifiedtonewextents,makingiteasiertozoomintothelayerextentandtosetthescaleofthedataframe.Thedefinitionqueriesalsolimitwhichmembersofalayeraredisplayedwithinthedataframe.Layersourcereplacementwasusedasacartographiccontrol,allowingustopre-generatethestyleofalayerandadjustthedatathatitrepresenteddynamically.

Inthenextchapter,wewillcombinethelessonsfromthelastthreechapters,allowingustocreateascripttoolthatwillrunanalysisandproducespreadsheetsandmapsfromtheanalysisresults.

Chapter10.AdvancedGeometryObjectMethodsInthischapter,wewilldiscussadvancedGeometryobjectmethods,previouslydiscussedinChapter6,WorkingwithArcPyGeometryObjects.ThegoalofthisbookistogiveanintroductiontoArcPyanditsmodules,whilealsodemonstratinghowtoapplythesetoolswhencreatingenduringGISworkflows.Performingananalysisonceisgood,butdoingitoverandover,withtheclickofabutton,isbetter.Makingtheanalysisresultssharableinanindustrystandardformatisalsodesirable.IntheArcGISworld,thebestwaytodothisiswithArcPyandscripttoolsthattakeadvantageofGeometryobjectmethods.

Thischapterwillcoverthefollowingtopics:

AddingcommonfunctionstoamoduleinthePythonpathMakingtheanalysismoreadvancedbyaddingpointgenerationAdvancedPolygonobjectmethodsUsingtheXLWTtocreateExcelspreadsheets

CreatingaPythonmoduleAnimportantsteptowardscreatingreusablecodeistopackageitscomponentfunctionsintoamodulethatcanbecalledfromthePythonpathbyanyscript.Tostart,weneedtocreateafolderinthesite-packagesfolderwherePythonmodulesareplacedwhendownloadedandextractedusingthePythonmoduleprocess,orwhenrunningthesetup.pyscriptincludedwithsharedmodules.

Modulespackagetogetherfunctionsinoneormorescriptsintoafolderthatcanbesharedwithothers(thoughtheyoftendependonothermodulestorun).Wehaveusedsomeofthebuilt-inmodulessuchasthecsvmoduleandthird-partymodulessuchasArcPy.Let’sexploretheirconstructiontogetafeelofhowamoduleispackagedforuseandsharing.

NoteManymodulesarenotplacedwithinthesite-packagesfolder,buttheyrequirethePythonpathtobemodifiedtomakethemimportable.Placingmoduleswithinthesite-packagesfoldereliminatesthisrequirement.

Openupthesite-packagesfolderinWindowsExplorerbynavigatingtoC:\Python27\ArcGIS10.2\Lib\site-packages(orC:\Python27\Lib\site-packagesifyou’reusingthestandardPython2.7installation)folder.Onceinthefolder,createanewfoldercalledcommon,asshowninthefollowingscreenshot:

The__init__.pyfileWithinthisfolder,aspecialfileneedstobeaddedtoletPythonrecognizethefolderasamodule.Thisfile,called__init__.py,takesadvantageofthespecialpropertyofPythoncalledmagicobjectsorattributesthatarebuiltintoPython.Thesemagicobjectsusetheleadingandtrailingdoubleunderscoretoavoidanyconfusionwithcustomfunctions.

NoteNotethatthesearedoubleunderscores;singleunderscoresareusuallyusedforso-calledprivatefunctionswithincustomPythonclasses.

The__init__.pyfileisusedtoindicatethatthefolderisamodule(makingitimportableusingtheimportkeyword),andtoinitiatethemodulebycallinganymodulesthatitmayinturnrelyon.However,thereisnorequirementtoaddimportcommandstothe__init__.pyfile;itcanbeanemptyfileandwillstillperformthemodulerecognitionfunctionalitythatwerequire.

1. OpenupIDLEorAptanaoryourfavoriteIDE,andinthefoldercalledcommon,addanewPythonfileandcallit__init__.py.Thisfilewillremainemptyfornow.

2. Nowthatwehaveinitiatedthemodule,weneedtocreateascriptthatwillholdourcommonfunctions.Let’scallituseful.pybecausethesefunctionswillbemostusefulforthisanalysisandothers.

3. Thenextstepistotransferfunctionsthatwehadcreatedinearlierchapters.Thesevaluablefunctionsarelockedintothosescripts,sobyaddingthemtouseful.py,wewillmakethemavailabletoallotherscriptswecraft.

NoteOneimportantfunctionistheformatSQLMultiplefromChapter4,ComplexArcPy

ScriptsandGeneralizingFunctions,whichgeneratesSQLstatementsusingatemplateandalistofdata.Byaddingittouseful.py,wewillbeabletocallthefunctionanytimeaSQLstatementisrequired.

4. OpenthescriptChapter4Modified2.pyandcopythefunction,andthenpasteitintouseful.py.Ithasnodependencies,soitdoesnothavetobemodified.

AnotherusefulfunctionfromthatscriptistheformatIntersectfunctionthatgeneratesastringoffilepathsthatareusedwhenrunningtheArcToolboxIntersecttool.WhilewehavereacheddeeperintoArcPysincethatfunctionwasdesigned,andnolongerneedtocalltheIntersecttoolinourbusstopanalysis,itdoesnotmeanthatwewillneverneedtocallitinthefuture.Itisstillusefulandshouldbeaddedtouseful.py.

ThelastfunctionthatwecanraidisthecreateCSV()function.CopyandpasteitfromChapter4Modified.pyintouseful.py.However,toavoidtheneedtoimporttheCSVmoduleseparately,wewillneedtomodifythefunctionslightly.Hereishowitshouldlook:

defcreateCSV(data,csvname,mode='ab'):

'createsacsvfile'

importcsv

withopen(csvname,mode)ascsvfile:

csvwriter=csv.writer(csvfile,delimiter=',')

csvwriter.writerow(data)

delcsv

Byimportingandthendeletingthecsvmodule,weareabletouseittogeneratethecsvfileandthenremovethemodulefrommemoryusingthedelkeyword.

Nowthatwehavethefunctionswewillbereusingsavedintheuseful.pyscript,insidethecommonmodule,let’sexplorehowtocallthemusingPython’simportmethod.OpenupaPythonexecutable,usingeitherPython.exeorIDLE,orthebuilt-interminalinAptana.Atthetriplechevronprompt(>>>),writethefollowingline:

>>>fromcommon.usefulimportcreateCSV>>>

Ifthesecondtriplechevron-shapedpromptappears,thefunctionwascorrectlyimportedfromthemodule.Toimportthefunctionsinthismoduleinascript,usethesameimportstructureandlistthefunctionsdesired,separatingthemusingacomma:

fromcommon.usefulimportcreateCSV,formatSQLMultiple

Thefunctionsinthescriptuseful.pywerecalledusingPythondotnotation.Thisismadepossiblebecausethe__init__.pyfileindicatestoPythonthatthefoldercommonisnowamodule,andthatitshouldexpectamethodcalledusefultobepresent,withthefunctionscreateCSVandformatSQLMultipleinsideit.

AddingadvancedanalysiscomponentsThebusstopanalysiswehaveusedtointroduceArcPycanbefurtherextendedtogeneratemorerefinedresults.Tobetterestimatethetruenumberofpeoplethateachbusstopserves,let’saddafunctionthatwillgeneraterandompointswithintheblocksconsidered,whileeliminatingparksandotherareasthatdonotcontainhousing.

Todothis,weneedtointroduceanewdatasetfromtheSanFranciscogeodatabase,theRPD_Parksfeatureclass.Byusingthisfeatureclasstoreducetheareaconsideredforouranalysis,wecangenerateamorerealisticassessmentoftheserviceareapopulationforeachbusstop.

WhileusingtheArcToolboxErasetooltoerasethearearepresentedintheRPD_Parkspolygonswouldbeausualstepwhenrunningaspatialanalysis,therearedrawbackstothisoption.ThefirstisthattheErasetoolisonlyavailablewiththeArcGISforDesktopAdvancedlicenselevel,makingitavailableonlytocertainusers.Theseconddrawbackisthatthetoolproducesanintermediatedataset,somethingtobeavoidedwhereverpossible.

UsingArcPywillgiveustheabilitytoavoidbothofthesedrawbacks.WecancreateascriptthatwillgeneraterandompointsonlywithinthefractionofthecensusblockpolygonsthatdonotintersectwiththeRPD_Parksfeatureclass.Todothis,wewillreachdeeperintothemethodsoftheArcPyPolygonobject.

AdvancedPolygonobjectmethodsInChapter6,WorkingwithArcPyGeometryObjectswestartedexploringtheArcPyGeometryobjectsandhowtousetheirmethodstoperformin-memoryspatialanalysis.TheBufferandIntersectmethodsoftheseobjectswereintroducedandusedtogenerateanalysisresults.Next,wewilldiscussmoreofthesemethodsandshowhowtheycanhelpimprovein-memoryspatialanalysis.

ThePolygonobjecthasamethodcalledDifferencethatallowsustofindtheareaofnon-intersectionwhentwopolygonsintersect.Passingacensusblockpolygonandaparkpolygonasparameterswillreturn(asapolygonobject)thefractionofthefirstparameterwherenooverlapoccurs.AnotherimportantmethodiscalledOverlaps,whichiscalledtotestwhethertwoGeometryobjects(points,lines,orpolygons)intersect.Ifthereisanoverlap,theOverlapsmethodwillreturnTrue,whilereturningFalseifthereisnooverlapbetweenthetwoobjects.Unionisalsoanimportantmethodthatwillbeusedwithinthischapter,itallowsfortwoGeometryobjectstobeunionedintooneobject.

Let’sexploretheseimportantmethods.Tofindthenon-intersectareaoftwopolygonobjects,thefollowingfunctioncombinestheOverlapsandDifferencemethods:

defnonIntersect(poly1,poly2):

'returnsareaofnon-intersectbetweentwopolygons'

ifpoly1.overlaps(poly2)==True:

returnpoly1.difference(poly2)

ThefunctionnonIntersectacceptstwoPolygonobjectsasparameters.Thefirstparameter,poly1,isthepolygonofintersect(thecensusblockpolygon)andthesecondparameter,poly2,isthepolygontobecheckedforoverlap.TheifconditionalusestheOverlapsmethodandreturnsTrueifthereisanoverlapbetweenthetwoparameters.Ifthereisanyoverlap,thedifference()methodreturnsthenon-intersectareaasapolygonobject.However,thisfunctionshouldbeextendedtocoversituationswheretheOverlaps()methodreturnsFalse:

defnonIntersect(poly1,poly2):

'returnsareaofnon-intersectbetweentwopolygons'

ifpoly1.overlaps(poly2)==True:

returnpoly1.difference(poly2)

else:

returnpoly1

ThefunctionwillnowreturnthefirstparameterwhentheOverlapsmethodreturnsFalse,indicatingthatthereisnooverlapbetweenthetwopolygonobjects.Thisfunctionisnowcompleteandavailabletobeusedinananalysis.BecausenonIntersect()isafunctionthatcanbeusedinotherspatialanalyses,copyitandaddittouseful.py.

GeneratingrandompointstorepresentpopulationThenextsteptoimprovethebusstopanalysisistogeneratepointstorepresentthepopulationofeachcensusblock.Whilerandompointswillnotprovideaperfectrepresentationofthepopulation,itwillserveasagoodmodelofthepopulationandallowustoavoidareaaveragingtofindtheroughpopulationofeachcensusblockservedbyabusstop.TheCreateRandomPointstoolintheArcToolboxDataManagementtoolsetmakesitsimpletogeneratethepoints.

TheCreateRandomPointstoolacceptsanumberofrequiredandoptionalparameters.Asthetoolgeneratesafeatureclass,therequiredparametersaretheworkspacewherethefeatureclasswillbeplacedandthenameofthefeatureclass.Theoptionalparametersofinterestaretheconstrainingfeatureclassandthenumberofpointstobegenerated.Aswearelookingtoavoidcreatingnewfeatureclassesintheintermediatestepsofouranalysis,wecanutilizethein_memoryworkspace,whichallowsfeatureclassestobegeneratedinmemory,meaningtheyarenotwrittentotheharddrive.

Becausethereisaneedtogenerateaspecificnumberofrandompointsforeachcensusblock,weshouldcreateafunctionthatwillacceptaconstrainingpolygonandpopulationfigurethatrepresentseachcensusblock.Thein_memoryworkspacewon’tworkforeverysituation,however,sowe’llprovidetheworkspaceparameterwithadefaultvalue:

defgeneratePoints(fc,pop,constrant,workspace='in_memory'):

'generaterandompoints'

importos,arcpy

arcpy.CreateRandomPoints_management(workspace,fc,

constrant,"",pop,"")

returnos.path.join(workspace,fc)

Thefunctionwillcreatethefeatureclassintheworkspacedesiredandwillreturnthepath(joinedusingtheosmodule)tothefeatureclassforuseintherestofthescript.Thisfunctionisalsoreusableandshouldbecopiedintouseful.py.

UsingthefunctionswithinascriptNowthatwehavecreatedthefunctionsthatwillhelpustorunamoreadvancedspatialanalysis,let’saddthemtoascriptalongwithsomeSearchCursorstoiteratethroughthedata:

#Importthenecessarymodules

importarcpy,os

fromcommon.usefulimportnonIntersect,generatePoints,createCSV

#Addanoverwritestatement

arcpy.env.overwriteOutput=True

#Definethedatainputs

busStops=r'C:\Projects\SanFrancisco.gdb\SanFrancisco\Bus_Stops'

parks=r'C:\Projects\SanFrancisco.gdb\SanFrancisco\RPD_Parks'

censusBlocks=

r'C:\Projects\SanFrancisco.gdb\SanFrancisco\CensusBlocks2010'

csvName=r'C:\Projects\Output\Chapter10Analysis.csv'

#Createthespreadsheetinmemoryandaddfieldheaders

headers='LineName','StopID','TotalPopulationServed'

createCSV(headers,csvName,mode='wb')

#Copythecensusblockdataintoafeaturelayer

arcpy.MakeFeatureLayer_management(censusBlocks,'census_lyr')

#CopytheparkdatageometriesintoalistandunionthemallparkGeoms=

arcpy.CopyFeatures_management(parks,arcpy.Geometry())

parkUnion=parkGeoms[0]

forparkinparkGeoms[1:]:

parkUnion=parkUnion.union(park)

#Createasearchcursortoiteratethebusstopdata

sql="NAME='71IB'ANDBUS_SIGNAG='FerryPlaza'"

witharcpy.da.SearchCursor(busStops,['NAME','STOPID','SHAPE@'],sql)as

cursor:

forrowincursor:

lineName=row[0]

stopID=row[1]

stop=row[2]

busBuf=stop.buffer(400)

#Selectcensusblocksthatintersectthebusbuffer

arcpy.SelectLayerByLocation_management("census_lyr","intersect",

busBuf,'','NEW_SELECTION')

#UseasecondCursortofindtheselectedpopulation

totalPopulation=0

witharcpy.da.SearchCursor("census_lyr",['SHAPE@','POP10',

'BLOCKID10'])asncursor:

fornrowinncursor:

block=nrow[0]

checkedBlock=nonIntersect(block,parkUnion)

blockName=nrow[2]

population=nrow[1]

ifpopulation!=0:

points=generatePoints("PopPoints",

population,checkedBlock)

pointsGeoms=

arcpy.CopyFeatures_management(points,arcpy.Geometry())

pointsUnion=pointsGeoms[0]

forpointinpointsGeoms[1:]:

pointsUnion=pointsUnion.union(point)

pointsInBuffer=busBuf.intersect(pointsUnion,1)

intersectedPoints=pointsInBuffer.pointCount

totalPopulation+=intersectedPoints

#Addthetallieddatatothespreadsheet

data=lineName,stopID,totalPopulation

print'datawritten',data

createCSV(data,csvName)

#Startthespreadsheettoseetheresults

os.startfile(csvName)

Let’sreviewthecode,sectionbysection,asthatisalottotakeinatfirst.

Theimportportioniswherewecalltheusualmodules,arcpyandos,alongwithourcustomfunctionsinthecommonmodule:

importarcpy,os

fromcommon.usefulimportnonIntersect

fromcommon.usefulimportgeneratePoints

fromcommon.usefulimportformatSQLMultiple

fromcommon.usefulimportnonIntersectcreateCSV

Asdiscussedpreviously,thefunctionsinthecommonmodule’susefulmethodarecalledusingthePythondotnotationandthefrom…import…importationstyle,makingthemavailabledirectly.Manyfunctionscanbeimportedononeline,separatedbycommas,orindividuallyasshownhere.

Thenextline,whichsetstheArcPyEnvironmentoverwritepropertytoTrue,isveryimportantbecauseitallowsustooverwritetheresultsoftheCreaterandompointsoperation.Iftheresultswerenotoverwritten,thefunctionresults,whichotherwisewoulduseallavailablememoryandcausethescripttofail:

arcpy.env.overwriteOutput=True

NoteItisimportanttobecarefulwiththisoverwritesettingbecauseitwillallowforanyfeatureclasstobeoverwritten.Allofouroutputisinmemoryandonlygeneratedfortheanalysis,sothereislittleneedtoworryhere,buttakecaretomakesurethatnothingimportantisoverwrittenwhenrunningascript.

Thenextportionisthesetofvariablesthatwillbeusedinthisscript,andwillinitiatethespreadsheetthatwillbeusedtocollecttheresultsoftheanalysis:

busStops=r'C:\PacktDB.gdb\SanFrancisco\Bus_Stops'

parks=r'C:\PacktDB.gdb\SanFrancisco\RPD_Parks'

censusBlocks=r'C:\PacktDB.gdb\SanFrancisco\CensusBlocks2010'

csvName=r'C:\Projects\Output\Chapter10Analysis.csv'

headers='LineName','StopID','TotalPopulationServed'

createCSV(headers,csvName,mode='wb')

ThefilepathsassignedtovariablesherecouldbereplacedwithArcPyparametersifweweretoturnthisintoascripttool,butfornow,thehard-codedpathsarefine.Belowthevariables,theresultsspreadsheetiscreatedandthecolumnfieldheadersareadded.

Itisworthnotingthatthespreadsheetiscreatedusingthewbmode.Thismodeofbinaryfileopening,knownaswb(writebinary),isusedforcreatinganewfile.ItmustbeexplicitlypassedintothecreateCSV()functionasthedefaultmodeparameterisab(appendbinary),whichwillcreateanewfileifitdoesnotexist,oraddtoonethatalreadyexists(athirdbinarymodeisrborreadbinary,whichisusedforopeninganexistingfile).

Thenextfewlinesmakedatainthefeatureclassesavailableinmemory.ThecensusblockdataisconvertedintoaFeatureLayer,whiletheRPD_ParksdataisreadintomemoryasalistofPolygonobjectsthatisthenunionedintoasingle,unifiedPolygonobjectcalledparkUnion:

arcpy.MakeFeatureLayer_management(censusBlocks,'census_lyr')parkGeoms=

arcpy.CopyFeatures_management(parks,arcpy.Geometry())

parkUnion=parkGeoms[0]

forparkinparkGeoms[1:]:

parkUnion=parkUnion.union(park)

ByusingtheCopyFeaturestoolintheDataManagementtoolset,theparkGeomsvariableispassedalistofthegeometriesforeachrowofdataintheRPD_Parksfeatureclass.However,wedon’twanttohavetoiteratethroughtheparkgeometriestocomparethemtoeachcensusblock,sotheUnionmethodisinvokedtocreateonePolygonobjectfromtheentirelist.ByassigningthefirstmemberofthelisttotheparkUnionvariable,andtheniteratingthroughtheparkGeomslisttouniontheothergeometriesonebyone,theresultisonePolygonobjectthatrepresentsallparkswithintheRPD_Parksdataset.

Onceallofthemoduleshavebeenimportedandthevariableshavebeenassigned,wecanentertheforloopofthedataaccessSearchCursortobegintheanalysis.However,wedon’twanttorunthisforallofthebusstops,sowewilluseaSQLstatementwhereclause,tolimittheanalysistoasinglebusline:

sql="NAME='71IB'ANDBUS_SIGNAG='FerryPlaza'"

witharcpy.da.SearchCursor(busStops,['NAME','STOPID','SHAPE@'],sql)as

cursor:

forrowincursor:

lineName=row[0]

stopID=row[1]

stop=row[2]

busBuf=stop.buffer(400)

arcpy.SelectLayerByLocation_management("census_lyr","intersect,busBuf,'','N

EW_SELECTION')

totalPopulation=0

Thefirstportionoftheiterationinvolvesenteringtheforloopandassigningthevaluesofeachrowtoavariable.APolygonobjectbufferof400feetiscreatedaroundthePointGeometryobjectreturnedbytheSearchCursor.ThisbufferisthenusedtointersectwiththecensusblocksFeatureLayertofindandselectallofthecensusblocksthatintersectthebuffer.Totallythepopulationservedbyeachbuffer,thevariabletotalPopulationiscreated.

Oncetheselectionhasbeenperformed,asecondSearchCursorcanbeusedtoiteratethroughtheselectedblockstoretrievetheirpopulationvaluesandPolygonobjectsforrandompointgeneration:

witharcpy.da.SearchCursor("census_lyr",['SHAPE@','POP10',

'BLOCKID10'])asncursor:

fornrowinncursor:

block=nrow[0]

checkedBlock=nonIntersect(block,parkUnion)

blockName=nrow[2]

population=nrow[1]

Inthisiteration,onceeachcensusblockhasbeenretrieved(intheformofaPolygonobject),theblockisthencheckedagainsttheunionedparkgeometryusingthenonIntersectfunctioncreatedpreviously.Thisensuresthatthepointswillonlybecreatedwithinareasthatarenotparks,thatis,morelikelytorepresentwherepeoplewouldlive.Thepopulationvaluesarealsoretrieved.

Oncetheconstrainingpolygon(forexamplethecensusblock)hasbeenevaluatedandanypotentialparkportionhasbeenremoved,andthepopulationvalueisavailable,therandompointscanbegeneratedusingthegeneratePoints()function:

ifpopulation!=0:

points=generatePoints("PopPoints",population,checkedBlock)

pointsGeoms=arcpy.CopyFeatures_management(points,arcpy.Geometry())

pointsUnion=pointsGeoms[0]

forpointinpointsGeoms[1:]:

pointsUnion=pointsUnion.union(point)

pointsInBuffer=busBuf.intersect(pointsUnion,1)

intersectedPoints=pointsInBuffer.pointCount

totalPopulation+=intersectedPoints

ThegeneratePoints()functionrequiresthreeparameters.Thefirstisthenameofthefeatureclasstobegenerated;thiswillbeoverwritteneachtimeitisgenerated,thusavoidingtheoveruseofmemorybycreatinganin_memoryfeatureclassforeachcensusblock.TheothertwoparametersarethepopulationvalueandtheconstrainingPolygonobject.

Oncethesehavebeenpassedtothefunction,itreturnsafilepathtothenewlycreatedfeatureclassandassignsthefilepathtothevariablepoints.ThegeometriesinpointsarethenextractedusingtheCopyFeaturestoolandassignedtothevariablepoints.TheUnionmethodisagainusedtocreateasingle,unifiedpopulationPointGeometryobject

thatwillbeintersectedwiththebusstopbuffer.Oncethisintersectionhasbeenrun,theresultinggeometriesareassignedtothepointsInBuffervariableandthepointCountmethodisusedtofindthenumberofpointsthatweregeneratedwithinthebufferedarea.Thisisourestimateofpopulationwithinthecensusblock,andthisvalueisaddedtothetotalPopulationvariabletoeventuallyyieldthetotalestimatedpopulationwithin400feetofthebusstop.

ThefinallinesofthescriptdemonstratehowthedataiscollectedintoatupleandpassedtothecreateCSV()moduletobewrittentoourfinalspreadsheet:

data=lineName,stopID,totalPopulation

print'datawritten',data

createCSV(data,csvName)

os.startfile(csvName)

Thelastline,os.startfile(csvName),usesthestartfilemethodoftheosmoduletoautomaticallyopenthespreadsheetoncetheanalysisiscompleted.Inthiscase,thespreadsheetC:\Projects\Output\Chapter10Analysis.csvhasbeenpopulatedwiththeresultsoftheanalysisandisopenedtodisplaytheseresults.However,theusermayhavetoindicatethatthelinesarecommaseparatedvaluestoopenthescript.

Insteadofcreatingacommaseparatedvalue,wecantakeadvantageofanotherPythonmodulethatisinstalledwhenArcGIS10.2andArcPyisinstalled.Thismodule,calledXLWT,isusedtogenerateExcelspreadsheets,andalongwiththeExcelspreadsheetreadingmoduleXLRD,isoneofthemostusefulmodulesavailabletousersofPython.

CreatinganXLSusingXLWTXLWTisapowerfulmodulethatallowsforamultitudeofstylingoptions.However,forourpurposeswecanignorethoseoptionsandcreateafunctionthatwillgenerateaspreadsheetwiththeresultsofourspatialanalysis.Thisfunctioncanofcoursebeaddedtocommon.useful:

defgenerateXLS(indatas,sheetName,fileName):

importxlwt

workbook=xlwt.Workbook()

sheet=workbook.add_sheet(sheetName)

forYCOUNTER,datainenumerate(indatas):

forXCOUNTER,valueinenumerate(data):

sheet.write(YCOUNTER,XCOUNTER,value)

workbook.save(fileName)

Thisfunctionrequiresthreeparameters,indatas-alistcontainingrowsofiterabledata,astringsheetname,andastringfilenamethatendswiththe.xlsextension.

Tousethisfunction,addittocommon.useful.Onceithasbeenadded,copyandrenametheolderanalysisscriptsothatitcanbeadjusted:

importarcpy,os

fromcommon.usefulimportnonIntersect,generatePoints,generateXLS

arcpy.env.overwriteOutput=True

busStops=r'C:\Projects\PacktDB.gdb\SanFrancisco\Bus_Stops'

parks=r'C:\Projects\PacktDB.gdb\SanFrancisco\RPD_Parks'

censusBlocks=r'C:\Projects\PacktDB.gdb\SanFrancisco\CensusBlocks2010'

xlsName=r'C:\Projects\Output\Chapter10Analysis.xls'

headers='LineName','StopID','TotalPopulationServed'

indatas=[headers]

arcpy.MakeFeatureLayer_management(censusBlocks,'census_lyr')parkGeoms=

arcpy.CopyFeatures_management(parks,arcpy.Geometry())

parkUnion=parkGeoms[0]

forparkinparkGeoms[1:]:

parkUnion=parkUnion.union(park)

sql="NAME='71IB'ANDBUS_SIGNAG='FerryPlaza'"

witharcpy.da.SearchCursor(busStops,['NAME','STOPID',

'SHAPE@'],sql)ascursor:

forrowincursor:

lineName=row[0]

stopID=row[1]

stop=row[2]

busBuf=stop.buffer(400)

arcpy.SelectLayerByLocation_management("census_lyr","intersect",busBuf,'','

NEW_SELECTION')

totalPopulation=0

witharcpy.da.SearchCursor("census_lyr",['SHAPE@','POP10',

'BLOCKID10'])asncursor:

fornrowinncursor:

block=nrow[0]

checkedBlock=nonIntersect(block,parkUnion)

blockName=nrow[2]

population=nrow[1]

ifpopulation!=0:

points=

generatePoints("PopPoints",population,checkedBlock)

pointsGeoms=arcpy.CopyFeatures_management(points,arcpy.Geometry())

pointsUnion=pointsGeoms[0]

forpointinpointsGeoms[1:]:

pointsUnion=pointsUnion.union(point)

pointsInBuffer=busBuf.intersect(pointsUnion,1)

intersectedPoints=pointsInBuffer.pointCount

totalPopulation+=intersectedPoints

data=lineName,stopID,totalPopulation

indatas.append(data)

generateXLS(indatas,"Results",xlsName)

os.startfile(xlsName)

WecannowgenerateExcelspreadsheetsjustaseasilyaswehavegeneratedCSVfileswhileemployingareusablefunction.Wenowhavetheabilitytoperformrepeatablespatialanalysisfastandcanproduceresultsinindustrystandardformats.

SummaryInthischapter,wehaveexploredhowtocreatemodulesandreusablefunctionsthatwillsavescriptingtimeinthefuturebyallowingustoavoidrewritingtheseusefulfunctions.WefurtherexploredthemethodsavailablethroughArcPyGeometryobjects,includingtheIntersect,Overlaps,andUnionmethods.Wecreatedaspatialanalysisthatwritesnofeatureclassestodisk,makingitsothattheanalysistimeisreducedandunnecessaryfilesareavoided.Finally,weexploredhowtogenerateExcelspreadsheetsusingtheXLWTmodulesothatanalysisresultscanbesharedinindustrystandardformats.

Inthenextchapter,wewillexplorehowtouseArcPytointeractwiththeArcGISfordesktopextensionssuchasNetworkAnalystandSpatialAnalyst.Byincorporatingtheirfunctionalitywithinascript,wefurtherincreaseourabilitytocreatefastandrepeatablespatialanalysisworkflows.

Chapter11.NetworkAnalystandSpatialAnalystwithArcPyUseoftheArcGISforDesktopextensionsalsobenefitsfromthepowerofPythonandArcPy.TheabilitytomodelroutesusingastreetsdatasetorabusroutesdatasetusingArcPywillhelpusconvertentireworkflowsintoscripttools.BothNetworkAnalystsandSpatialAnalystshaveaccessmodulesbuiltintoArcPyforimprovedcontroloftheiravailabletools,methods,andproperties.

Thischapterwillcoverthefollowingtopics:

CreatingasimplenetworkdatasetCheckingouttheextensionsTheArcPyNetworkAnalystmoduleTheArcPySpatialAnalystmodule

TheNetworkAnalystextensionTheESRI’sNetworkAnalystextensionisapowerfultooltoenableroutingandnetworkconnectivityfunctionalitywithinArcGIS.Theextension,whenusedforstreetrouting,allowsuserstofindthequickestpathbetweentwopointsalongaroadnetwork.Theroutecanbeconstrainedbyanumberoffactors,suchastrafficorleftturns,tobettermodelroadtravel.Similaranalysiscanberunusingothertypesofnetworks,suchaswaterpipenetworksorelectricalnetworks.

UsingNetworkAnalystTousetheNetworkAnalystextension,theArcGISforDesktopAdvancedlicenseisrequired.InArcCatalogorArcMap,clickontheCustomizemenuandselectExtensions.OncetheExtensionsmenuisopen,clickonthecheckboxnexttoturnontheNetworkAnalystExtension.

CreatingaFeatureDatasetThefirststeptousinganetworkdatasetistocreateonewithinafeaturedataset.Todoso,wewillgenerateafeaturedatasettoholdthedataofinterest.Right-clickontheFilegeodatabasethathousestheBusStopdataandselectNew,andthenselectFeatureDatasetfromtheNewmenu.NameitChapter11ResultsandclickonNext.

Next,selecttheSpatialReferenceSystem(SRS).Inthiscase,wewillbeusingtheSRSofthelocalStatePlanezoneforSanFrancisco.Itisaprojectedcoordinatesystem,soselectthatfolder,andthenclickontheStatePlanefolder.Onceitisopened,selectthefoldercalledNAD1983(USFeet).Fromtheavailablereferencesystems,selecttheone

calledNAD1983StatePlaneCaliforniaIIIFIPS0403(USFeet).ClickonNexttogotothenextmenu.

NoteThissystemisalsoknownas2227inWellKnownID(WKID)orEuropeanPetroleumSurveyGroup(EPSG)systems.Moreinformationaboutthesecodesisavailableathttp://spatialreference.org,awebsiteusedtofindthethousandsofspatialreferencesystemsusedthroughouttheworld.

ClickontheVerticalCoordinateSystemsfolderandthenselecttheNorthAmericafolder.SelecttheNorthAmericanVerticalDatumof1988infeet(NAVD1988USsurveyfeet).Thiswillmakeitpossibletohavetheverticalandhorizontallinearunitsinthesamemeasurementsystem.ClickonNexttogotothenextmenu.

Thetolerancesonthenextpagearealsoveryimportant,butwewillnotcoverthemindetailhere.AcceptthedefaultsandclickonFinishtofinalizetheFeatureDataset.

ImportingthedatasetsImportthebusstops,streets,andbusroutesfeatureclassesintotheChapter11ResultsFeatureDataset.Right-clickonthedatasetandselectImport,andthenFeatureClass(Single).AddthefeatureclassesonebyonetogivethemanewnamethatwillkeepthemseparatedfromtheversionscontainedwithintheSanFranciscoFeatureDataset.ImportingthemwillmakesurethattheyareinthecorrectSRSandthatanetworkdatasetcanbecreated.

CreatingtheNetworkDatasetNowthatwehaveadatacontainer,wecancreateanetworkdatasetfromthestreetsfeatureclass.Right-clickontheChapter11ResultsfeaturedatasetandselectNew,andthenchooseNetworkDataset.

CalltheNetworkDatasetStreet_NetworkandclickonNext.SelecttheStreetsfeatureclassastheclassthatwillparticipateinthenetworkdatasetandclickonNexttomovetothenextmenu.SelectGlobalTurnstomodelturnswithinthenetwork.Inthenextmenu,usethedefaultconnectivitysettings.Then,accepttheUsingZCoordinateValuesfromGeometrysetting.Acceptthedefaultcostrestrictionanddrivingdirectionssettings,andfinallyclickonFinishtogeneratethenetworkdataset.Then,buildthenetworkdatasetusingthefinalmenu.Thenetworkdatasetisreadytobeused.

AccessingtheNetworkDatasetusingArcPyNowthatthenecessarysetuphasbeencompleted,thestreet_networknetworkdatasetcanbeaddedtoascriptforuseingeneratingroutes.Becausethisisasimpleanalysis,theonlyimpedancevaluetobeusedwillbethelengthofthestreetsegments.ThroughtheuseofaSearchCursor,PointGeometryobjectsfromthebusstopscanbeaccessedandaddedaslocationstobesearched:

importarcpy

arcpy.CheckOutExtension("Network")

busStops=r'C:\Projects\PacktDB.gdb\Chapter11Results\BusStops'

networkDataset=r'C:\Projects\PacktDB.gdb\Chapter11Results\street_network'

networkLayer="streetRoute"

impedance="Length"

routeFile="C:\Projects\Layer\{0}.lyr".format(networkLayer)

arcpy.MakeRouteLayer_na(networkDataset,

networkLayer,impedance)

print'layercreated'

sql="NAME='71IB'ANDBUS_SIGNAG='FerryPlaza'"

witharcpy.da.SearchCursor(busStops,['SHAPE@','STOPID'],sql)ascursor:

forrowincursor:

stopShape=row[0]

printrow[1]

arcpy.AddLocations_na(networkLayer,'Stops',stopShape,"","")

arcpy.Solve_na(networkLayer,"SKIP")

arcpy.SaveToLayerFile_management(networkLayer,routeLayerFile,"RELATIVE")

print'finished'

BreakingdownthescriptLet’sdissectthescript,whichoncefinished,willgeneratealayerfilecontainingtheaddedStops,andtheRoutesalongstreetstobestgetfromtheoriginstoptothedestinationstop.

ThescriptbeginsbyimportingthearcPymodule.ThenextlineallowsustousetheNetworkAnalystextension:

arcpy.CheckOutExtension("Network")

Usingthearcpy.CheckOutExtension()methodtoinvoketheNetworkAnalystextensioninvolvespassingthecorrectkeywordtothemethodasaparameter.Onceithasbeeninvoked,thetoolsoftheextensioncanbecalledandexecutedinthescript.

Assigningthebusstopsfeatureclassandthestreet_networknetworkdatasettovariables,theycanthenbepassedtoArcPy’sMakeRouteLayer_na()method,alongwithavariablerepresentingtheimpedancevalue:

arcpy.MakeRouteLayer_na(networkDataset,

networkLayer,impedance)

TheMakeRouteLayer_natoolproducesaRouteLayerinmemory.Thisblanklayerneedstobepopulatedwithstopstoproducetheroute(s)betweenthem.Forthispurpose,weneedaSearchCursortoaccessthePointGeometryobjectsandaSQLstatementthatwilllimitthereturnedresultstothelineofinterest:

sql="NAME='71IB'ANDBUS_SIGNAG='FerryPlaza'"

witharcpy.da.SearchCursor(busStops,['SHAPE@','STOPID'],sql)ascursor:

forrowincursor:

stopShape=row[0]

printrow[1]

arcpy.AddLocations_na(networkLayer,'Stops',stopShape,"","")

TheSearchCursorwillallowtheStopssublayerofthelayerproducedbytheMakeRouteLayertooltobepopulatedwhenusedinconjunctionwiththeAddLocationstool.Oncepopulated,theRouteLayercanbepassedtotheSolvetooltofindtheroutesbetweenthepointsofinterest.Again,theroutesaresolvedbasedonfindingthelowestimpedancebetweenthetwopoints.Inthisexample,theonlyimpedanceisthesegmentlength,butitcouldbetrafficorelevationorotherrestrictiontypes,ifthatdataisavailable:

arcpy.Solve_na(networkLayer,"SKIP")

arcpy.SaveToLayerFile_management(networkLayer,routeLayerFile,"RELATIVE")

ThefinalresultisalayerfilethatiswrittentodiskusingtheSaveToLayerFiletool.

TheNetworkAnalystmoduleInanefforttomaketheuseoftheNetworkAnalystextensionmorePythonic,thenewerNetworkAnalyst(na)moduleadjustshowthemethodsthatcorrespondtotheArcToolboxNetworkAnalysttoolsareaccessed.InsteadofcallingthetoolsdirectlyfromArcPy,thetoolsarenowmethodsofthenamodule.RemovingtheinitialsoftheNetworkAnalysttoolsetalsoreducesconfusionandmakesiteasiertorememberthenameofthemethod.Seethedifferenceasfollows:

importarcpy

arcpy.CheckOutExtension("Network")

busStops=r'C:\Projects\SanFrancisco.gdb\SanFrancisco\Bus_Stops

networkDataset=

r'C:\Projects\SanFrancisco.gdb\Chapter11Results\street_network'

networkLayer="streetRoute"

impedance="Length"

routeLayerFile="C:\Projects\Layer\

{0}_2.lyr".format(networkLayer)arcpy.na.MakeRouteLayer(networkDataset,

networkLayer,impedance)

print'layercreated'

sql="NAME='71IB'ANDBUS_SIGNAG='FerryPlaza'"

witharcpy.da.SearchCursor(busStops,['SHAPE@','STOPID'],sql)ascursor:

forrowincursor:

stopShape=row[0]

printrow[1]

arcpy.na.AddLocations(networkLayer,'Stops',stopShape,"","")

arcpy.na.Solve(networkLayer,"SKIP")

arcpy.management.SaveToLayerFile(networkLayer,routeLayerFile,"RELATIVE")

print'finished'

Thetoolwillproducethesamelayeroutputastheoriginalscript,butthereorganizationoftheNetworkAnalysttoolsintothenamodulehasmadethecodemorelogical.Forinstance,itmakesmoresensetocallSolveusingarcpy.na.Solve(),insteadofarcpy.Solve_na(),asitreinforcesthatSolveisamethodoftheNetworkAnalyst(na)module.AsArcPycontinuestobedeveloped,IexpectmorePythoniccodereorganizationtooccur.

AccessingtheSpatialAnalystExtensionTheSpatialAnalystExtensionisveryimportanttoperformanalysisonbothrasterandvectordatasets,butitisgenerallyusedtoperformsurfaceanalysisandrastermath.TheseoperationsaremadeeveneasierbytheuseofArcPy,asallofthetoolsavailableintheSpatialAnalystToolboxareexposedwiththeSpatialAnalystaccessmodule.ThisincludestheRasterCalculatortools,makingmapalgebraeasybyusingthetoolsandoperatorsinsimpleexpressions.

AddingelevationtothebusstopsTheelevationraster“sf_elevation”hasbeendownloadedfromNOAAandaddedtotheFileGeodatabase.However,itcoverstheentireBayArea,andweshouldwriteascripttoonlyextractanareaofthecityofSanFranciscoasitwillreducethetimeneededtorunourscripts.We’lluseaSQLstatementasthewhereclausetolimittheresultstotheSouthofMarket(SoMa)neighborhood.Todoso,let’stakeadvantageofaSearchCursorandtheSpatialAnalystaccessmodule’sExtractbyPolygonproperty:

importarcpy

arcpy.CheckOutExtension("Spatial")

busStops=r'C:\Projects\PacktDB.gdb\SanFrancisco\Bus_Stops'

sanFranciscoHoods=

r'C:\Projects\PacktDB.gdb\SanFrancisco\SFFind_Neighborhoods'

sfElevation=r'C:\Projects\PacktDB.gdb\sf_elevation'

somaGeometry=[]

sql="name='SouthofMarket'"

witharcpy.da.SearchCursor(sanFranciscoHoods,['SHAPE@XY'],sql,None,True)

ascursor:

forrowincursor:

X=row[0][0]

Y=row[0][1]

somaGeometry.append(arcpy.Point(X,Y))

somaElev=arcpy.sa.ExtractByPolygon(sfElevation,somaGeometry,"INSIDE")

somaOutPath=sfElevation.replace('sf_elevation','SOMA_elev')

somaElev.save(somaOutPath)

print'extractionfinished'

TheExtractByPolygon()methodisabitmisleading,asitdoesnotacceptaPolygonobjectasaparameter.Instead,itrequiresalistofPointobjectsthatrepresenttheverticesoftheareathatwewanttoextract.AstheSearchCursorisiteratingthroughtheneighborhoodsdataset,aPolygonobjectisreturnedbythecursor.Fortunately,theSearchCursorhasafinalparameter,whichwehavenotyetexplored,thatallowsustoextracttheindividualpointsorverticesthatmakeuptheSomaneighborhoodpolygon.BysettingtheSearchCursor’soptionalExplodetoPointsparameter(whichconvertsPolygonobjectsintocoordinatepairsforeachvertex)toTrue,PointobjectscanbegeneratedbypassingtheXYvaluesofeachreturnedvertextothearcpy.Pointmethod.ThesePoint()objectsareappendedtothesomaGeometrylistandthenpassedtotheSpatialAnalystaccessmodule’sExtractByPolygonmethod.

NotePassingaPolygonObjectinsteadofPointObjectswillreturnanerror.

UsingMapAlgebratogenerateelevationinfeetWenowhavearastertousetoextractelevationvalues.However,boththeoriginalrasterandthegeneratedSoManeighborhoodrastercontainelevationvaluesinmeters,anditwouldbebettertoconvertthemtofeettokeepthemconsistentwiththeprojectionofthebusstops.Let’suserastermathandtheTimes()methodtoconvertthevaluesfrommeterstofeet:

somaOutPath=sfElevation.replace('sf_elevation','SOMA_elev')

outTimes=arcpy.sa.Times(somaOutPath,3.28084)

somaFeetOutPath=sfElevation.replace('sf_elevation','SOMA_feet')

outTimes.save(somaFeetOutPath)

TheTimes()methodgeneratesanewrastertogleantheelevationvaluesweneedforthebusstopsofinterest.

AddinginthebusstopsandgettingelevationvaluesNowthatwehavegeneratedarasterthatwecanusetofindelevationvaluesinfeet,weneedtoaddanewarcpy.sa()methodtogeneratethepoints.TheExtractValuesToPoints()methodwillgenerateanewbusstopsfeatureclasswithanewfieldthatholdstheelevationvalues:

witharcpy.da.SearchCursor(sanFranciscoHoods,['SHAPE@'],sql)ascursor:

forrowincursor:

somaPoly=row[0]

arcpy.MakeFeatureLayer_management(busStops,'soma_stops')

arcpy.SelectLayerByLocation_management("soma_stops","INTERSECT",somaPoly)

outStops=r'C:\Projects\PacktDB.gdb\Chapter11Results\SoMaStops'

arcpy.sa.ExtractValuesToPoints("soma_stops",

somaOutFeet,outStops,"INTERPOLATE","VALUE_ONLY")

print'pointsgenerated'

ThefinalresultWeproducedasubsetfeatureclassofthebusstopsthathastheelevationvaluesaddedasafield.Thisprocesscouldberepeatedfortheentirecity,oneneighborhoodatatime,oritcouldbeperformedwiththeoriginalelevationrasterontheentirebusstopsfeatureclasstogenerateavalueforeachstop:

importarcpy

arcpy.CheckOutExtension("Spatial")

arcpy.env.overwriteOutput=True

busStops=r'C:\Projects\PacktDB.gdb\SanFrancisco\Bus_Stops'

sanFranciscoHoods=

r'C:\Projects\SanFrancisco.gdb\SanFrancisco\SFFind_Neighborhoods'

sfElevation=r'C:\Projects\SanFrancisco.gdb\sf_elevation'

somaGeometry=[]

sql="name='SouthofMarket'"

witharcpy.da.SearchCursor(sanFranciscoHoods,['SHAPE@XY'],sql,None,True)

ascursor:

forrowincursor:

somaGeometry.append(arcpy.Point(row[0][0],row[0][1]))

somaElev=arcpy.sa.ExtractByPolygon(sfElevation,somaGeometry,"INSIDE")

somaOutput=sfElevation.replace('sf_elevation','SOMA_elev')

somaElev.save(somaOutput)

print'extractionfinished'

somaOutput=sfElevation.replace('sf_elevation','SOMA_elev')

outTimes=arcpy.sa.Times(somaOutput,3.28084)

somaOutFeet=sfElevation.replace('sf_elevation','SOMA_feet')

outTimes.save(somaOutFeet)

print'conversioncomplete'

witharcpy.da.SearchCursor(sanFranciscoHoods,['SHAPE@'],sql)ascursor:

forrowincursor:

somaPoly=row[0]

arcpy.MakeFeatureLayer_management(busStops,'soma_stops')

arcpy.SelectLayerByLocation_management("soma_stops","INTERSECT",somaPoly)

outStops=r'C:\Projects\SanFrancisco.gdb\Chapter11Results\SoMaStops'

arcpy.sa.ExtractValuesToPoints("soma_stops",

somaOutFeet,outStops,"INTERPOLATE","VALUE_ONLY")

print'pointsgenerated'

ThisscriptdemonstrateswellthevalueofaccessingtheadvancedextensionsinArcPyandcombiningthemwithSearchCursorsandGeometryobjects.ThescriptcouldbetakenevenfurtherbyaddingaSearchCursortolookthroughtheoutstopsdatasetandexportingtheresultstoaspreadsheet,orevenaddinganewfieldtotheoriginalbusstopsdatasettopopulatewiththeelevationvalues.ItcouldevenbeusedasimpedancevaluestobeenteredintoaNetworkAnalystextensionanalysis—afuncodingtaskthatIhopeyouwillattempt.

SummaryInthischapter,wecoveredthebasicsofusingcommonArcGISforDesktopAdvancedextensionswithinArcPy,withafocusontheNetworkAnalystaccessmoduleandtheSpatialAnalystaccessmodule.WeexploredhowtogenerateanetworkandhowtocreatenetworkpathsusingArcPy.WealsoexploredhowtoaccessSpatialAnalysttoolsandusetheminconjunctionwithSearchCursorstoworkwithrastersandvectorsforspatialanalysis.

Inthenextchapter,wewillexploresomefinalpiecestotheArcPypuzzlethatwillallowthecreationofadvancedscriptsandscripttools.

Chapter12.TheEndoftheBeginningThisbookisalmostdone,butthereissomuchmoretoknowaboutwritingcodeinPythonandArcPy.Unfortunately,Ican’tfititallintoonebook,butthatalsomeansthatyougettohavefunexploringallofthemethodsandpropertiesofArcPy.Asaconclusiontothebook,wewillcoversomeotherimportanttopicsthatcancropupwhenwritingArcPyscripts.Combinedwiththelessonsfromearlierchapters,Ihopeyou’llsoonbeusingArcPyatwork,atschool,orjustforfun(whynot?).

Thischapterwillcoverthefollowingtopics:

Workingwithfieldinformation–types,aliases,domains,spatialtypes,andmoreAccessinginformationdescribingaFeatureClassAutomaticallygeneratingaFeatureClassandpopulatingitwithfieldsAutomaticallycreatingFileGeodatabasesandFeatureDatasetsCreatingaScripttoolthatwillrunthebusstopanalysisandgenerateresultsinanautomaticallygeneratedFileGeodatabase,FeatureDataset,andFeatureClass

GettingfieldinformationfromfeatureclassesWhencreatingscripttools,orjustrunningascript,therecanbetimesthatextractingfieldinformationfromafeatureclass(orshapefile)isnecessary.Thisinformationcanincludefieldnamesandaliases,fieldtypeandlength,scale,domains,orsubtypes.Theseareallpropertiesavailablethroughthearcpy.ListFieldsmethod.We’llexplorethemanyproperties,howtoextractthem,andhowtousetheminascript.

ByorganizingtheArcPymethodsintoafunction,thedataisorganizedinaformthatweprefer,insteadofrelyingonthedefaultorganizationusedbythedesignersofArcPy.It’simportanttorememberthatscriptsyoucreateshouldreflectyourneeds,andcreatingthesefunctionwrappersisonestepforwardtowardspolishingtherawArcPytoolstoworkinyourworkflows.

AccessingtheListFields’propertiesTheListFieldstoolisavailableasanArcPymethod.Arcpy.ListFieldsacceptsonlyoneparameter,afeatureclass,orshapefile.Oncetheparameterhasbeenpassed,aseriesofimportantpropertiesareavailableusingdotnotation.Totakefurtheradvantageoftheseproperties,wewillcreatefunctionsthatmakeiteasytogettheinformationwewant,intheformatwerequire.

ListcomprehensionsWithinthesefieldinformationfunctions,wewilltakeadvantageofaPythondatastructureknownaslistcomprehensions.Theysimplifytheforloopstructuretomakeiteasiertopopulatealistwiththevaluesrequired(thefieldinformationinthiscase).

Tocreatealistcomprehension,aforloopisgeneratedinsideasetofbrackets,andthelistispopulatedwiththegeneratedvalues.Hereisanexampleofalistcomprehensionthatcreatesalistwiththesquarevaluesofthenumbersfrom1to10,asruninthePythoninterpreter:

>>>originalList=range(1,11)

>>>printoriginalList

[1,2,3,4,5,6,7,8,9,10]

>>>newList=[x**2forxinoriginalList]

>>>printnewList

[1,4,9,16,25,36,49,64,81,100]

Listcomprehensionsareusedbecausetheyarefasterandeasiertowrite,thoughitmaytakesometimetogetusedtothesyntax.Experimentwiththemtobetterunderstandtheiruseandlimitations,andalsoconsultsomeofthemanyresourcesavailableonline.

CreatingthefieldinformationfunctionsEachofthefunctionswillbeaseparateentity,buttheywillallhaveasimilarstructure.Oneparameterwillbeacceptedbyeachfunction,thefeatureclassofinterest.ArcPywillbeimported,andlaterdeletedfrommemory,tomakesurethattheListFields()methodcanbecalledwithoutanerror.OncethefeatureclassispassedtotheListFields()method,thevaluesdesiredwillpopulatealistinsidealistcomprehension.Onceithasbeenpopulated,itisreturnedfromthefunctionusingthereturnkeyword.

Hereisthesetoffunctionsforthefieldnames:

defreturnfieldnames(fc):

importarcpy

fieldnames=[f.nameforfinarcpy.ListFields(fc)]

delarcpy

returnfieldnames

defreturnfieldalias(fc):

importarcpy

fieldalias=[f.aliasNameforfinarcpy.ListFields(fc)]

delarcpy

returnfieldalias

defreturnfieldbasename(fc):

importarcpy

fieldtypes=[f.baseNameforfinarcpy.ListFields(fc)]

delarcpy

returnfieldtypes

Thesenamefunctionsareusefulwhencreatinganewfeatureclassbasedonanotherfeatureclass.Sometimesthereisaneedtopreservetheexactnamesandaliasesfromtheoriginalfeatureclass,andusingthesefunctionswillmakethispossible.Whendoingthis,thereisaneedtoprovideotherfieldinformationaswell.Herearethefunctionsrelatedtofieldtypes,lengths,precision,andscale:

defreturnfieldtypes(fc):

importarcpy

fieldtypes=[f.typeforfinarcpy.ListFields(fc)]

delarcpy

returnfieldtypes

defreturnfieldlength(fc):

importarcpy

fieldlengths=[f.lengthforfinarcpy.ListFields(fc)]

delarcpy

returnfieldlengths

defreturnfieldprecision(fc):

importarcpy

fieldprecise=[f.precisionforfinarcpy.ListFields(fc)]

delarcpy

returnfieldprecise

defreturnfieldscale(fc):

importarcpy

fieldscales=[f.scaleforfinarcpy.ListFields(fc)]

delarcpy

returnfieldscales

Thereisevenapropertyusedtorequestdomaininformation:

defreturnfielddomain(fc):

importarcpy

fielddomains=[f.domainforfinarcpy.ListFields(fc)]

delarcpy

returnfielddomains

Thesefunctionsallsharethestructurediscussedearlier,andhavetheadvantageofbeingsimpletouseandeasytosearchthroughout.Becausefieldsinafeatureclasshaveaspecificorder,eachlistreturnedbythefunctionswillhaveanordertotheinformationreturned,accessiblebyaspecificindexnumber.

Thefieldsubtypesarealsoavailablethroughthedataaccessmodule.Becausetheyarerelatedtothefields,theyarereturnedasadictionary:

defreturnfieldsubtypes(fc):

importarcpy

fieldsubdic={}

subtypes=arcpy.da.ListSubtypes(fc)

forstcode,stdictinsubtypes.iteritems():

forstkeyinstdict.iterkeys():

ifstkey=='FieldValues':

fields=stdict[stkey]

forfield,fieldvalsinfields.iteritems():

sub=fieldvals[0]

desc=fieldvals[1]

fieldsubdic[field]=sub,desc

delarcpy

returnfieldsubdic

NoteAddingthesefunctionstotheuseful.pyscriptinthecommonmodulewillmakethemavailabletoanyscriptorscripttool.Usetheimportkeywordtoaddthemtoanynewscript.Theyareself-containedfunctionsthatonlyrequirethefilepathtothefeatureclassofinterest.

QueryingfeatureclassinformationSomeimportantpiecesofinformationaboutanincomingfeatureclasscannotbeaccessedusingtheListFields()method.Instead,anumberofdifferentmethodswillbeusedtofindtheGeometrytype,orSpatialReference,orthefieldsubtypeofeachfeatureclass.SomeofthesearediscoveredusingArcPy’sDescribemethod,builttoprovide

FortheGeometrytype,wewillusetheshapeTypepropertyoftheDescribe()method:

defreturngeometrytype(fc):

importarcpy

arcInfo=arcpy.Describe(fc)

geomtype=arcInfo.shapeType

delarcpy

returnstr(geomtype)

ThenameoftheShapefield(whichusuallydefaultstoShape)canalsoberequestedusingtheDescribemethodandreturnsastringdatatype:

defreturngeometryname(fc):

importarcpy

arcInfo=arcpy.Describe(fc)

geomname=arcInfo.shapeFieldName

delarcpy

returnstr(geomname)

Thefeatureclassspatial_referenceisalsoavailablethroughtheDescribemethod.Thedataisreturnedasaspatial_referenceobject:

defreturnspatialreference(fc):

importarcpy

spatial_reference=arcpy.Describe(fc).spatialReference

delarcpy

returnspatial_reference

Aspatial_referenceobjecthasanumberofimportantproperties.Theprojectionnameandprojectioncodeareamongtheimportant

defreturnprojectioncode(fc):

importarcpy

spatial_reference=arcpy.Describe(fc).spatialReference

proj_code=spatial_reference.projectionCode

delarcpy

returnproj_code

defreturnprojectionname(fc):

importarcpy

spatial_reference=arcpy.Describe(fc).spatialReference

proj_name=spatial_reference.name

delarcpy

returnproj_name

Manyotherpropertiesandmethodscanbesimilarlyutilizedtomakethemavailablewithinscriptsorscripttools.ExploretheArcGIShelpdocumentsforfurtherinsightsintothepropertiesavailablethroughtheDescribemethod.

GeneratingFileGeodatabasesandfeatureclassesFileGeodatabasesdonothavetoexistbeforeascriptisrun;instead,theycanbegeneratedwhenascriptisexecutedusingtheCreateFileGDBtool,whichisalsoanArcPymethod.OncetheFileGeodatabasehasbeencreated,FeatureDatasetscanbeadded.

GeneratingtheFileGeodatabaseisveryeasy.Theonlyparametersarethefolderstoplaceitinside,andthenameoftheGeodatabase:

importarcpy

folderPath=r"C:\Projects"

gdbName="ArcPy.gdb"

arcpy.CreateFileGDB_management(folderPath,gdbName)

TheFeatureDatasetsaremoredifficulttocreate,asthereisanoptionalspatialreferenceparameterthatrequiresaSpatialReferenceobjecttobegenerated.WhiletheSpatialReferenceobjectisoptional,itishighlyrecommended.

ThereareafewoptionstogeneratetheSpatialReferenceobject.OneofthemusesthereturnspecialReference()functiondefinedearlier;bypassingafeatureclasstothefunction,aSpatialReferenceobjectiscreated.Anothermethodwouldbetopassafilepathtoaprojectionfile.prjastheoptionalthirdparameter.AthirdmethodistogenerateaSpatialReferenceobjectbyusingthearcpy.SpatialReferencemethodandpassingitaprojectioncodeoraprojectionstring:

spatialReference=arcpy.SpatialReference(2227)

Howeveritisgenerated,itisthenpassedtothearcpy.CreateFeatureDatasetmethodalongwiththefilepathoftheFileGeodatabaseandthenameoftheFeatureDataset:

spatialReference=arcpy.SpatialReference(2227)

fileGDB=r"{0}\{1}".format(folderPath,gdbName)

featureDataset="Chapter12Results"

arcpy.CreateFeatureDataset_management(fileGDB,featureDataset,

spatialReference)

GeneratingafeatureclassNowthataFileGeodatabaseandaFeatureDatasethavebeencreated,let’sgenerateaFeatureClassinsidetheFeatureDataset.Thisisdoneusingthearcpy.CreateFeatureClassmethod.Thismethodhasanumberofoptionalparameters,includingaFeatureClasstouseasatemplateandaSpatialReference.Forthisexample,thereisnoneedtousetheSpatialReferenceparameterasitisbeingwrittentoaFeatureDataset,whichdictatestheSpatialReferenceused.ThetemplateparameterwillcopythefieldsofthetemplateFeatureClass,butfornow,wewillonlycreatetheShapefield:

featureClass="BufferArea"

geometryType="POLYGON"

featurePath=r"{0}\{1}".format(fileGDB,featureDataset)

arcpy.CreateFeatureclass_management(featurePath,featureClass,

geometryType)

ThecreatedFeatureClasswillneedsomefieldswiththeattributeinformationthatwillbepopulatedlater.Thefieldshaveanumberofparametersthatdependonthefieldtype,includinglength,precision,andalias,amongothers:

fieldName="STOPID"

fieldAlias="BusStopIdentifier"

fieldType="LONG"

fieldPrecision=9

featureClassPath=r"{0}\{1}".format(featurePath,featureClass)

arcpy.AddField_management(featureClassPath,fieldName,fieldType,

fieldPrecision,

"","",fieldAlias)

Let’saddasecondfieldtoholdtheaveragedpopulationvaluesproducedbytheBusStopanalysis:

fieldName2="AVEPOP"

fieldAlias2="AverageCensusPopulation"

fieldType2="FLOAT"

featureClassPath=r"{0}\{1}".format(featurePath,featureClass)

arcpy.AddField_management(featureClassPath,fieldName2,fieldType2,"","",

"",fieldAlias2)

TheFileGeodatabase,FeatureDataset,andFeatureClassfieldshavenowbeengenerated.Let’sextendthescriptintoascripttoolbyaddingtheBusStopanalysisfunctions,whilewritingtheresultstothegeneratedFeatureClass.Creating,ascripttoolthatpopulatesafeatureclass.

ThisscripttoolwillborrowfromtheideasoutlinedinChapter10,AdvancedGeometryObjectMethodsandwillcreateaunionofthePolygonGeometryobjectsthatintersectwiththebufferedbusstopstopopulatetheShapefield,alongwiththebusstopIDandtheaveragedpopulationfortheblocksintersectedwitheachbuffer.

OpenthescriptChapter12_3.pyandexploreitscontents.Coupledwiththecodesnippetsmentionedearlierandtheuseofarcpy.GetParameterAsTexttogetdatafromthescript

tool,thedatageneratedwillbewritteninafeatureclassbythefollowingcode:

arcpy.AddMessage("BeginningAnalysis")

insertCursor=arcpy.da.InsertCursor(featureClassPath,['SHAPE@',fieldName,

fieldName2])

arcpy.MakeFeatureLayer_management(censusBlocks2010,"census_lyr")

witharcpy.da.SearchCursor(busStops,['SHAPE@',busStopField],sql)as

cursor:

forrowincursor:

stop=row[0]

stopID=row[1]

busBuffer=stop.buffer(400)

arcpy.SelectLayerByLocation_management("census_lyr","intersect",busBuffer,'

','NEW_SELECTION')

censusShapes=[]

censusPopList=[]

witharcpy.da.SearchCursor("census_lyr",

['SHAPE@',censusBlockPopField])asncursor:

fornrowinncursor:

censusShapes.append(nrow[0])

censusPopList.append(nrow[1])

censusUnion=censusShapes[0]

forblockincensusShapes[1:]:

censusUnion=censusUnion.union(block)

censusPop=sum(censusPopList)/len(censusPopList)

finalData=(censusUnion,stopID,censusPopulation)

insertCursor.insertRow(finalData)

arcpy.AddMessage("AnalysisComplete")

Thescriptcombinesmanyoftheideasthathavebeenintroducedthroughoutthebooktoallowtheusertorunacompleteworkflowthatgeneratesafeatureclasscontainingtheresultsoftheanalysis.ByaddingonlythefieldsofinterestandpopulatingthemwiththeunionedPolygonobjects,thescripteliminatesmostofthecruft,normallycreatedwhenrunningaspatialanalysis,andproducesaresultsdatasetthatcanbeviewedinArcMap.

SettingupthescripttoolparametersHereishowtheparametersofthescripttoollookwhensetup:

Thelistofparametersislong,soIamusingtwoimagestoportraythem.Itisimportanttochoosethecorrectdatatypeforeachparameterasitwillcontrolthedialoggeneratedtoretrievethedata.

ThebusstopIDfieldandthePopulationfieldarebothobtainedfromtheirrespectivefeatureclasses.TheFileGeodatabasenameisastringandthecodewillappend.gdbtotheendoftheinputstringifitisnotenteredinitially,tomakesurethatitcanbecorrectlygenerated.Itshouldnotalreadyexist;itwillnotbegeneratedifitdoes(ifdesired,thiscanbechangedbysettingthearcpy.env.overwriteOutputpropertytoTrueaftertheimportstatement).

Oncetheparametershavebeenset,andthetoolhasanameanddescription,saveitandthenopenthetool.Itshouldlooklikethisonceithasbeenfilledout:

ClickonOKtorunthetool.OpenArcMapandaddtheresults,alongwiththeSanFranciscopolygonandtheInbound71featureclassfromChapter4,ComplexArcPyScriptsandGeneralizingFunctions.Theresultswilllooksimilartothis,afterabitofcartographicsymbolizing:

Thefinalresultwillhaveonerowperbusstopselected,alongwiththeaveragedpopulationandthebusstopIDvalue.Insteadofusingaspreadsheetasanoutput,thefeatureclasswillallowtomakemapsorproducefurtherspatialanalysis.Producingcustomdatausingcustomscripttoolsputsyouinthedriver’sseatwhenperforminggeospatialanalysesandmakesyourtools,andyou,avaluableassettoanyteam.

EnvironmentalsettingsTheArcPymoduleallowsforthecontrolofglobalsettingsthatcontrolsinputandoutputprocessesusingArcPy’senvclass.Thesesettingswillhaveaneffectontheaccuracyofdataproducedusinggeospatialanalysistools.ResolutionandtolerancesettingsforX,Y,Z,andMcoordinatescanbecontrolled,alongwithoutputextent,rastercellsize,analysisworkspace,andmanyothersettings.

ToaccesstheenvironmentalsettingsusingArcPy,theclassenvisimportedfromarcpy:

>>>fromarcpyimportenv

Itcanalsobecalledusingdotnotationshownasfollows.Settingtheworkspaceremovestheneedtopassafilepathtoanysubsequentmethodscalledontheworkspace.HereisanexampleofsettingtheworkspaceandcallingtheListDatasets()methodwithoutpassingafilepathasaparameter:

>>>importarcpy

>>>arcpy.env.workspace=r"C:\Projects\SanFrancisco.gdb"

>>>arcpy.ListDatasets()

[u'SanFrancisco',u'Chapter3Results',u'Chapter4Results',

u'Chapter5Results',u'Chapter7Results',u'Chapter11Results']

ResolutionandtolerancesettingsTheresolutionandtolerancesettingscontroltheaccuracyoftheoutputofanydataproducedbyatoolinArcToolboxorwhenrunningascriptusingArcPy.Thesecan(andshould)besetforFeatureDatasetsinFileGeodatabasesorEnterpriseGeodatabases,butitisimportanttosetthemforanalysisruninthememoryorwhenusingshapefiles,orifthegeospatialanalysisrequiresgreateraccuracythanusedbythoseGeodatabases.

Settingtheresolutionsandtolerancesrequireanunderstandingoftheaccuracyrequiredforyourprojects.Thesesettingswilllimittheabilitytosnaptoalineorfindpointsthatintersectwithaline.Thelinearunitwillneedtoreflectthecoordinatesystemofchoice:

importarcpy

arcpy.env.MResolution=0.0005

arcpy.env.MTolerance=0.005

arcpy.env.ZResolution="0.0025Feet"

arcpy.env.ZTolerance="0.001Feet"

arcpy.env.XYResolution="0.00025Feet"

arcpy.env.XYTolerance="0.0005Feet"

Otherimportantenvironmentalsettingsinclude:

TheExtentsetting,whichlimitstheextentofanydataproducedfromananalysisbysettingarectangleofinterestusinganExtentobject,orastringwithspacedelimitedcoordinates(Xmin,Ymin,Xmax,Ymax)inthecurrentcoordinatesystem.TheMasksetting,whichlimitsrasteranalysistoareasthatintersectwithafeatureclassorarasterpassedasastringfilepathparametertothesetting.TheCellSizesetting,whichcontrolsthecellsizeofthedataproducedusingrasteranalysis.

TaketimeandexplorethepowerfulArcPyEnvironmentalSettingstoreducethetimeneededtowritecodeandensurehigh-qualitydataproduction.

SummaryThischapterandthisbookhavedemonstratedsomeofthemanywaysthatArcPycanbeusedtoautomategeospatialanalysis.Byapplyingthelessons,andbybeingcreativewiththemanymethodsandpropertiesofArcPy,repetitiveandslowgeospatialprocessescanbescriptedandmadeintocustomtoolsthatwillsavealotoftime.

IhopethatyouenjoyedlearningthebasicsofscriptingwithArcPyandPython.Ireallyhopethatyou’veevencometoliketheideaofprogramming,asitispowerfulandempowering.Thereismuchmoretomaster,butIthinkyouwillfindthatthemorescriptingyoudo,theeasieritistounderstand.

ThebestresourceforfurtherunderstandingofArcPyistheArcGISHelpDocuments,availablethroughtheHelpmenuinArcCatalogorArcMap.Thedocumentationisalsoavailableathttp://resources.arcgis.com/en/help/main/10.2/index.html.WorkingonenteringthecorrectquestionintoGooglecanbeveryhelpfulaswell.ProgrammingforumssuchasStackExchange(http://gis.stackexchange.com/)orESRI’sGeoNet(https://geonet.esri.com/welcome)arevaluableresourcestoaskallkindsofprogrammingquestions.Thereisananswerforalmosteveryquestionyoumayhave(butneverbeafraidtoaskquestionsyourself!).

Havefuncreatingsolutionsandtools,andgoodluckinallyourfuturegeospatialprogrammingchallenges!

IndexA

adjustedmapexporting,toPDF/ExportingtheadjustedmaptoPDF

analysiscomponentsadding/AddingadvancedanalysiscomponentsPolygonobjectmethods/AdvancedPolygonobjectmethodsrandompoints,generatingtorepresentpopulation/Generatingrandompointstorepresentpopulationfunctionsused,withinscript/UsingthefunctionswithinascriptXLScreating,XLWTused/CreatinganXLSusingXLWT

analysisresultstallying/Tallyingtheanalysisresults

ApplicationProgrammingInterface(API)/WrappermodulesAptanaStudio3/AptanaStudio3

URL/AptanaStudio3ARCMacroLanguage(AML)

about/ModelBuilderArcPy

about/OverviewofPythonused,withmapdocuments/UsingArcPywithmapdocumentsused,foraccessingnetworkdataset/AccessingtheNetworkDatasetusingArcPy

arcpy.AddMessageused,fordisplayingscriptmessages/Displayingscriptmessagesusingarcpy.AddMessage

arcpy.mappingused,forcontrollingLayerobjects/Usingarcpy.mappingtocontrolLayerobjects

arcpy.Pointfunction/UsinganInsertCursorarcpy.SpatialReference()method/ThedataaccessmoduleArcPygeometryobjectclasses

about/ArcPygeometryobjectclasses,ArcPyPointobjects,ArcPyArrayobjects,ArcPyPolylineobjectsPolygonobjects/ArcPyPolygonobjects,Polygonobjectbuffers,OtherPolygonobjectmethodsgeometryobjects/ArcPygeometryobjectsPointGeometryobjects/ArcPyPointGeometryobjects

ArcPymoduleabout/OverviewofPython,TheArcPymodule

attributefieldinteractions/Attributefieldinteractionsautomatedmapdocumentadjustment

about/Automatedmapdocumentadjustmentvariables/Thevariablesmapdocumentobject/Themapdocumentobjectandthetextelementstextelements/Themapdocumentobjectandthetextelementslayervisibility,adjusting/Adjustinglayervisibilitybuffer,generatingfrombusstopsfeatureclass/Generatingabufferfromthebusstopsfeatureclassbusstopbufferandcensusblocks,intersecting/Intersectingthebusstopbufferandcensusblockstextelements,updating/Updatingthetextelements

Bbrokenlinks

fixing/Fixingthebrokenlinksbuffer

generating,frombusstopsfeatureclass/Generatingabufferfromthebusstopsfeatureclass

Buffertoolmodeling/ModelingtheSelectandBuffertools

built-infunctionsstr/Commonlyusedbuilt-infunctionsint/Commonlyusedbuilt-infunctionsfloat/Commonlyusedbuilt-infunctions

built-inmodulesURL/Commonlyusedstandardlibrarymodules

busstopbufferblockandcensusblock,intersecting/Intersectingthebusstopbufferandcensusblocks

busstopclassandbufferfeatureclass,populating/Populatingtheselectedbusstopandbufferfeatureclasses

BusStopfeatureclassadding,asparameter/AddingtheBusStopfeatureclassasaparameter

busstopfieldsadding,asparameter/Addingthebusstopfieldsasaparameter

Ccensusblock

andbusstopbufferblock,intersecting/Intersectingthebusstopbufferandcensusblocks

CensusBlockfeatureclassadding,asparameter/AddingtheCensusBlockfeatureclassasaparameter

CensusBlockfieldadding,asparameter/AddingtheCensusBlockfieldasaparameter

CommaSeparatedValue(CSV)/Addingtheoutputspreadsheetasaparametercomments

about/CommentsCSVmodule

adding,toscript/AddingtheCSVmoduletothescriptcursor

used,foraccessingdata/Accessingthedata:Usingacursor

Ddata

accessing,cursorused/Accessingthedata:Usingacursordataaccessmodule

about/Thedataaccessmoduleattributefieldinteractions/Attributefieldinteractionsupdatecursor/Updatecursorsshapefield,updating/Updatingtheshapefieldpointlocation,adjusting/Adjustingapointlocationrow,deletingwithupdatecursor/DeletingarowusinganUpdateCursorinsertcursor,using/UsinganInsertCursor

dataframewindowextentcontrolling/Controllingthedataframewindowextentandscale

datasetsimporting/Importingthedatasets

datasourcesreplacing/Replacingthedatasources

datatypesabout/Datatypesstrings/Stringsintegers/Integersfloats/Floatslists/Liststuples/Tuplesdictionaries/Dictionariesiterabledatatypes/Iterabledatatypesadding/Addingdatatypes

definitionqueryabout/Definitionqueries

defkeywordabout/Functions

deleteRowmethod/DeletingarowusinganUpdateCursordictionaries/Dictionariesdynamiccomponents

adding,toscript/Addingdynamiccomponentstothescriptdynamicparameters

adding,toscript/Addingdynamicparameterstoascript

Eenvironmentalsettings

about/Environmentalsettingsresolutionsetting/Resolutionandtolerancesettingstolerancesetting/ResolutionandtolerancesettingsExtentsetting/ResolutionandtolerancesettingsMasksetting/ResolutionandtolerancesettingsCellSizesetting/Resolutionandtolerancesettings

Ffeatureclasses

fieldinformation,obtainingfrom/GettingfieldinformationfromfeatureclassesListFieldstool/AccessingtheListFields’propertiesListcomprehensions/Listcomprehensionsfieldinformationfunctions,creating/Creatingthefieldinformationfunctionsfeatureclassinformation,querying/QueryingfeatureclassinformationFileGeodatabases,generating/GeneratingFileGeodatabasesandfeatureclassesgenerating/GeneratingFileGeodatabasesandfeatureclasses,Generatingafeatureclassscripttoolparameters,settingup/Settingupthescripttoolparametersenvironmentalsettings/Environmentalsettings

featureclassinformationquerying/Queryingfeatureclassinformation

FeatureDatasetcreating/CreatingaFeatureDataset

fieldinformationobtaining,fromfeatureclasses/Gettingfieldinformationfromfeatureclasses

fieldinformationfunctionscreating/Creatingthefieldinformationfunctions

filepaths,inPythonabout/FilepathsinPython

finalscriptabout/Thefinalscriptinspecting/Inspectingthefinalscript,RunningtheScriptTool

floats/Floatsforloops/Forloopsfunctions

about/Functionsused,withinscript/Usingthefunctionswithinascript

Ggeometryobjects/ArcPygeometryobjectsGIS

about/Datatypes

Hhard-codedinputs

about/Addingdynamicparameterstoascript

IIDEs

about/IntegratedDevelopmentEnvironments(IDEs),IDEsummaryIDLE/IDLEPythonWin/PythonWinAptanaStudio3/AptanaStudio3automaticallygeneratedscript/Theautomaticallygeneratedscript

IDLEabout/IDLE

If/Elif/Elsestatementsabout/If/Elif/Elsestatements

importstatements/Importstatementsindentation

about/Indentationindividuallayers

fixing/Fixingthelinksofindividuallayers,ExportingtoPDFfromanMXDinsertcursor

using/UsinganInsertCursorintegers/IntegersIntersecttool

adding/AddingtheIntersecttooliterabledatatypes/Iterabledatatypes

Kkeywordmethod/Adjustingapointlocationkeywords

about/Keywords

LLayerobject

adding/AddingaLayerobjectLayerobjects

controlling,arcpy.mappingused/Usingarcpy.mappingtocontrolLayerobjectsmethods/Layerobjectmethodsandpropertiesproperties/Layerobjectmethodsandproperties

layerobjectsabout/Thelayerobjects

Layersabout/Usingarcpy.mappingtocontrolLayerobjects

layersourcesinspecting/Inspectingandreplacinglayersourcesreplacing/Inspectingandreplacinglayersourcesbrokenlinks,fixing/Fixingthebrokenlinksindividuallayers,fixing/Fixingthelinksofindividuallayers,ExportingtoPDFfromanMXDmapdocumentelements,adjusting/Adjustingmapdocumentelements

layervisibilityadjusting/Adjustinglayervisibility

Listcomprehensions/ListcomprehensionsListFieldstool/AccessingtheListFields’propertieslists/Lists

Mmapdocumentelements

adjusting/Adjustingmapdocumentelementsmapdocuments

ArcPy,usingwith/UsingArcPywithmapdocumentsmaps

exporting/Exportingthemapsmodel

creating/CreatingamodelandexportingtoPythonexporting,toPython/CreatingamodelandexportingtoPythonSelecttool,modeling/ModelingtheSelectandBuffertoolsBuffertool,modeling/ModelingtheSelectandBuffertoolsIntersecttool,adding/AddingtheIntersecttoolanalysisresults,tallying/Tallyingtheanalysisresultsexporting/Exportingthemodelandadjustingthescript

ModelBuilderabout/ModelBuilder

moduleadding,sysmoduleused/UsingPython’ssysmoduletoaddamodule

modulesresiding/Wheremodulesreside

Nnamespaces

about/Namespacesnamingvariables

using,bestpractices/VariablesNetworkAnalyst

using/UsingNetworkAnalystFeatureDataset,creating/CreatingaFeatureDatasetdatasets,importing/Importingthedatasetsnetworkdataset,creating/CreatingtheNetworkDatasetnetworkdatasetaccess,ArcPyused/AccessingtheNetworkDatasetusingArcPy

NetworkAnalystextensionabout/TheNetworkAnalystextension

NetworkAnalystmoduleabout/TheNetworkAnalystmodule

networkdatasetcreating/CreatingtheNetworkDataset

OOperatingSystem(OS)module

about/TheOperatingSystem(OS)moduleoutputspreadsheet

adding,asparameter/Addingtheoutputspreadsheetasaparameter

Pparameter

CensusBlockfeatureclass,addingas/AddingtheCensusBlockfeatureclassasaparameterCensusBlockfield,addingas/AddingtheCensusBlockfieldasaparameteroutputspreadsheet,addingas/Addingtheoutputspreadsheetasaparameterspreadsheetfieldnames,addingas/AddingthespreadsheetfieldnamesasaparameterSQLStatement,addingas/AddingtheSQLStatementasaparameterbusstopfields,addingas/Addingthebusstopfieldsasaparameter

parametersdatatypes,adding/AddingdatatypesBusStopfeatureclass,addingas/AddingtheBusStopfeatureclassasaparameter

PDFadjustedmap,exportingto/ExportingtheadjustedmaptoPDF

PointGeometryobjects/ArcPyPointGeometryobjectspointlocation

adjusting/Adjustingapointlocationpolygongeometry

inserting/InsertingapolygongeometryPolygonobjectmethods/AdvancedPolygonobjectmethodsPolygonobjects/ArcPyPolygonobjects,Polygonobjectbuffers,OtherPolygonobjectmethodspolyLinegeometry

inserting/Insertingapolylinegeometryprogrammingjargon

about/ForloopsPython

about/OverviewofPythonPython,asprogramminglanguage

about/Pythonasaprogramminglanguageinterpretedlanguage/Interpretedlanguagestandard(built-in)library/Standard(built-in)librarygluelanguage/Thegluelanguagewrappermodules/Wrappermodules

Python,basicsabout/ThebasicsofPythonimportstatements/Importstatementsvariables/Variablesforloops/ForloopsIf/Elif/Elsestatements/If/Elif/Elsestatementswhilestatement/Whilestatements

comments/CommentsPythonfolderstructure

about/Pythonfolderstructuremodules,residing/Wheremodulesresidesysmoduleused,foraddingmodule/UsingPython’ssysmoduletoaddamodule

Pythonfunctionsabout/Pythonfunctions–Avoidrepeatingcodedefining/Technicaldefinitionoffunctionswriting/Afirstfunctionwithparameters/Functionswithparametersused,forreplacingrepetitivecode/Usingfunctionstoreplacerepetitivecodegeneralization/Moregeneralizationofthefunctions

Pythoninterpreterabout/WhatisthePythoninterpreter?location/WhereisthePythoninterpreterlocated?using/WhichPythoninterpretershouldbeused?locating/Howdoesthecomputerknowwheretheinterpreteris?

Pythonmodulecreating/CreatingaPythonmodule__init__.pyfile/The__init__.pyfile

PythonModules,forGISAnalysisabout/ImportantPythonModulesforGISAnalysis

Pythonscriptabout/WhatisaPythonscript?executing/HowPythonexecutesascript

PythonSystem(SYS)moduleabout/ThePythonSystem(SYS)module

PythonWin/PythonWinURL/PythonWin

PythonWindowscript,runningin/RunningthescriptinthePythonWindow

Rrandompoints

generating,torepresentpopulation/Generatingrandompointstorepresentpopulation

replace()method/Updatecursorsrow

deleting,withupdatecursor/DeletingarowusinganUpdateCursor

Sscaleproperties

controlling/Controllingthedataframewindowextentandscalescript

adjusting/Exportingthemodelandadjustingthescript,AdjustingtheScriptCSVmodule,addingto/AddingtheCSVmoduletothescriptdynamicparameters,addingto/Addingdynamicparameterstoascriptdynamiccomponents,addingto/Addingdynamiccomponentstothescriptrunning,inPythonWindow/RunningthescriptinthePythonWindowbreaking/Breakingdownthescript

ScriptAnalysis,ArcPyToolscontinuing/Continuingthescriptanalysis:theArcPytoolsIntersectTool/TheIntersecttoolandstringmanipulationStringManipulation/TheIntersecttoolandstringmanipulation

scriptmessagesdisplaying,arcpy.AddMessageused/Displayingscriptmessagesusingarcpy.AddMessage

scripttoolcreating/CreatingaScripttoolparameters,defining/Labellinganddefiningparametersparameters,labelling/Labellinganddefiningparameters

scripttoolparameterssettingup/Settingupthescripttoolparameters

Selecttoolmodeling/ModelingtheSelectandBuffertools

shapefieldupdating/Updatingtheshapefield

SpatialAnalystExtensionaccessing/AccessingtheSpatialAnalystExtensionelevation,addingtobusstops/AddingelevationtothebusstopsMapAlgebraused,forgeneratingelevationinfeet/UsingMapAlgebratogenerateelevationinfeetbusstops,adding/Addinginthebusstopsandgettingelevationvalues,Thefinalresultelevationvalues,obtaining/Addinginthebusstopsandgettingelevationvalues,Thefinalresult

spreadsheetfieldnamesadding,asparameter/Addingthespreadsheetfieldnamesasaparameter

SQLStatementadding,asparameter/AddingtheSQLStatementasaparameter

standardlibrarymodulesdatetime/Commonlyusedstandardlibrarymodulesmath/Commonlyusedstandardlibrarymodules

string/Commonlyusedstandardlibrarymodulesstringaddition/Thestringmanipulationmethod1–stringadditionstringformatting/Thestringmanipulationmethod2–stringformatting#1,Thestringmanipulationmethod3–stringformatting#2StringManipulation

stringaddition/Thestringmanipulationmethod1–stringadditionstringformatting/Thestringmanipulationmethod2–stringformatting#1,Thestringmanipulationmethod3–stringformatting#2

strings/Stringssubroutines

about/TechnicaldefinitionoffunctionsSys.path.append()method

about/Thesys.path.append()methodsysmodule

used,foraddingmodule/UsingPython’ssysmoduletoaddamodule

Ttextelements

updating/Updatingthetextelementsadjustedmap,exportingtoPDF/ExportingtheadjustedmaptoPDF

Tkinterabout/IDLE

tuples/Tuples

Uupdatecursor

about/Updatecursorsused,fordeletingrow/DeletingarowusinganUpdateCursor

updateRow()method/Updatecursors,Adjustingapointlocation

Wwhilestatement/Whilestatements

XXLRDmodule

about/TheXLRDandXLWTmodulesXLS

creating,XLWTused/CreatinganXLSusingXLWTXLWTmodule

about/TheXLRDandXLWTmodules

Zzero-basedindexing

about/Zero-basedindexing