high performance batch geocoding with arcgis€¦ · geocoding tools. geoprocessing service....
TRANSCRIPT
Victor Bhattacharyya
High Performance Batch Geocoding with ArcGIS
Goal: Get the best possible batch geocoding performance from your system, be it Desktop or Enterprise
• Batch geocoding overview
• Desktop geocoding recommendations
• ArcGIS Enterprise geocoding recommendations
• Working with Clients for Enterprise
Outline
• Batch geocoding overview
• Desktop geocoding recommendations
• ArcGIS Enterprise geocoding recommendations
• Working with Clients for Enterprise
Outline
Batch GeocodingBatch Geocoding Overview
• Turns addresses into points on the map
Addresses: your dataBatch Geocoding Overview
• Addresses hard to control as a data type
• Addresses change
• Reference data changes
• Approaches must fit any need for scale
Geocoding in ArcGIS ProBatch Geocoding Overview
• Pro ships with default locators- World Geocoding Service (requires internet &
credits)- XY provider for coordinate and grid handling
• Locate pane defines available locators- Add more to your project
Locator options in ArcGIS ProBatch Geocoding Overview
• Build your own locators using the CreateLocator tool
• Use Esri-provided locators- World Geocoding Service- World Geocoder for ArcGIS- Street Map Premium locators
• Batch geocoding overview
• Desktop geocoding recommendations
• ArcGIS Enterprise geocoding recommendations
• Working with Clients for Enterprise
Outline
Desktop Geocoding RecommendationsBest practices for local geocoding
• Use the latest version of ArcGIS Pro (2.3)
• Use new locators
- Most recent release of SMP locators
- Build your own locators with the CreateLocator tool in ArcGIS Pro 2.3
• Set the number of threads on your locator to “Auto”
- This uses 1 less than the number of cores on your machine
• Put the locator and input table on a solid state drive
• Use the Geocode Table pane in ArcGIS Pro
• Automate the process with the GeocodeAddresses GP Tool
Automating Desktop Geocoding: Geocode Addresses Best practices for local geocoding
• Batch geocoding overview
• Desktop geocoding recommendations
• ArcGIS Enterprise geocoding recommendations
• Working with Clients for Enterprise
Outline
ArcGIS Enterprise Recommended Batch Geocoding SystemArchitecture
• Get the most performance out of your enterprise geocoding services
• Manage backend resources
• Perform batch geocoding behind your organizational firewall
• Has chunking and retry logic built-in
ArcGIS Portal
GIS Servers
Geocode Server
World Locator ServiceSMP Locator Service(s)
GeocodingToolsGeoprocessing Service
Hosting Server
System RecommendationsBest practices for Enterprise geocoding
• Use the latest version of Asynchronous Batch Geocoding enabled clients- Geocode Locations from Table Mapviewer tool -> Web- Geocode Table and Geocode Locations from Table in ArcGIS Pro 2.3 -> Desktop- arcpy.geocoding.GeoocdeLocationsFromTable() and GeocodeFile API -> Scripting
• ArcGIS Server- Multiple cores (the more the better)- Enough memory for those cores (locator size + 500MB per core)- SSD Drive- Fast processors
How the system works
GIS Servers
Geocode Server
World Locator ServiceSMP Locator Service(s)Custom Locator Service
Main Process
Hosting Server
Asynchronous Batch Geocoding
Input Table
1. Chunks requests2. Makes geocodeAddresses API calls to the geocoding service3. Uses retry logic to retry failed requests4. Receives geocoding results5. Stitches together results
How the system works
GIS Servers
Geocode Server
World Locator ServiceSMP Locator Service(s)Custom Locator Service
Geocoding ToolsGeoprocessing Service
Hosting Server
Asynchronous Batch Geocoding
Input Table
1. Chunks requests2. Makes geocodeAddresses API calls to the geocoding service3. Uses retry logic to retry failed requests4. Receives geocoding results5. Stitches together results
How the system works
GIS Servers
Geocode Server
World Locator ServiceSMP Locator Service(s)Custom Locator Service
Geocoding ToolsGeoprocessing Service
Hosting Server
Asynchronous Batch Geocoding
Input Table
“numBatchThreads”: 4
1. Chunks requests2. Makes geocodeAddresses API calls to the geocoding service3. Uses retry logic to retry failed requests4. Receives geocoding results5. Stitches together results
How the system works
GIS Servers
Geocode Server Geocoding ToolsGeoprocessing Service
Hosting Server
Asynchronous Batch Geocoding
Input Table
Output Feature Layer
“numBatchThreads”: 4
World Locator ServiceSMP Locator Service(s)Custom Locator Service
1. Chunks requests2. Makes geocodeAddresses API calls to the geocoding service3. Uses retry logic to retry failed requests4. Receives geocoding results5. Stitches together results6. Makes output available for download
NumBatchThreads: A Closer LookArchitecture
“numBatchThreads”: 4
Geocoding Services
GeocodingTools Geoprocessing Service
NumBatchThreads with Multiple UsersArchitecture
“numBatchThreads”: 2
Geocoding Services
GeocodingTools Geoprocessing Service
Configure Portal Batch Geocoding
• How to Publish locators and add them as Utility Services
• Set the number of threads per geocoder via the Sharing API
• Scale the GP Service based on server resources via Server Manager
• Learn more at the Configure Portal to Geocode Addresses online help
https://bit.ly/2UspGqa; https://bit.ly/2CbVBnt; https://bit.ly/2C65UJF; https://bit.ly/2NR4WWv;
Configure Threads
Configuration
Configuration Demo
• Batch geocoding overview
• Desktop geocoding recommendations
• ArcGIS Enterprise geocoding recommendations
• Working with Clients for Enterprise
Outline
Clients for geocoding data on the Portal
Demo in Pro
Demo in MapViewer
Automating Enterprise Geocoding: Geocode Locations from Table Python
API for geocoding data on a local machine
New Approach: GeocodeFile
GIS Servers
Geocode Server
World Locator ServiceSMP Locator Service(s)Custom Locator Service
Hosting Server
• Input needs to be uploaded to the server as a zipped gdb, or a zipped CSV or XLSX file
• Output CSV, XLSX or Feature Class is available for download from the server when the job is completed
• Geocode File API available in ArcGIS Enterprise 10.7
Geocoding Output
Client uploads table to server
Client downloadsgeocoding results
- Discover available geocoding services- Authenticate requests
GeocodingTools GP Service
ArcGIS Server
ArcGIS Portal
1
2
4
Chunking and geocoding3
Best practices for Enterprise geocoding
Automating Enterprise GeocodingGeocode File API
Automating Enterprise GeocodingGeocode File API (Part 2)
Clients for geocoding data on a Big Data File Share
(BDFS Input)
GeoAnalytics: Geocode Locations from Table Data in HDFS -> Geocode Locations from Table GeoAnalytics tool
• If you have the GeoAnalytics Extension enabled in ArcGIS Enterprise this tool appears in the map viewer
• Has chunking and retry logic built-in
• Takes input table from a BDFS in ArcGIS Enterprise
GeoAnalytics: Geocoding via the Python APIData in HDFS -> Geocode Locations from Table GeoAnalytics tool
• GeoAnalytics tool exposed via the “geoanalytics” module in the Python API
• Has chunking and retry logic built-in
• Takes input table from a BDFS in ArcGIS Enterprise
Configuring Geocode Locations GeoAnalytics ToolData in HDFS -> Geocode Locations from Table GeoAnalytics tool
• Add the geocoding service as a utility service in your organization
• Set the “percentageMaxAllowedComputeCores” and “percentageMaxAllowedComputeMemory” system properties
• Read more at: http://enterprise.arcgis.com/en/server/latest/get-started/windows/geoanalytics-settings.htm
https://<machine_name>/server/admin/system/properties/update
Complete answersand select “Submit”
Scroll down to find the feedback section
Select the session you attended
Download the Esri Events app and find your event
Please Take Our Survey on the App
Questions?
