high performance batch geocoding with arcgis · pdf file 2019-08-08 · geocoding...

Click here to load reader

Post on 06-Jul-2020

1 views

Category:

Documents

0 download

Embed Size (px)

TRANSCRIPT

  • High Performance Batch Geocoding with

    ArcGIS Victor Bhattacharyya

  • Goal: Get the best possible

    batch geocoding

    performance from your

    system, be it Desktop or

    Enterprise

  • • Batch geocoding overview

    • Desktop geocoding recommendations

    • ArcGIS Enterprise geocoding recommendations

    • Working with Clients for Enterprise

    Outline

  • • Batch geocoding overview

    • Desktop geocoding recommendations

    • ArcGIS Enterprise geocoding recommendations

    • Working with Clients for Enterprise

    Outline

  • Batch Geocoding: Turns a spreadsheet of addresses into points on the map

    • Your spreadsheet or table

    • An ArcGIS Application such as ArcGIS Pro, Map Viewer or API and SDKs

    • And a locator

    CSV, XLS

    FGDB Table

    Portal Table

    Big Data File Share ArcGIS Map Viewer

    ArcGIS Pro

    Geoanalytics Extension

    World Geocoding Service

    StreetMap Premium

    World Geocoder for ArcGIS

    Create Your Own

  • • Batch geocoding overview

    • Desktop geocoding recommendations

    • ArcGIS Enterprise geocoding recommendations

    • Working with Clients for Enterprise

    Outline

  • Desktop Geocoding Recommendations Best practices for local geocoding

    • Use the latest version of ArcGIS Pro (2.4)

    • Use new locators

    - Newest StreetMap Premium locators or locators built with the

    CreateLocator tool in ArcGIS Pro 2.4

    • Set the number of threads on your locator to “Auto”

    - This uses 1 less than the number of cores on your machine

    • Put the locator and input table on a solid state drive

  • Desktop Geocoding Recommendations: Use Geocode Table ArcGIS Pro 2.4

    • Guided workflow steps you through batch

    geocoding

    • More batch geocoding options

    - Location Type

    - Category filtering

    - Country filtering

    • Works with local locators and Portal geocoding

    services (including the World Geocoding Service)

    • Geocode Addresses can be used for automation

  • Automating Desktop Geocoding: Geocode Addresses Best practices for local geocoding

    • Run Geocode Addresses GP Tool in ArcGIS Pro

    • Right-click -> “Copy Python command”

    • Paste into your favorite IDE

  • • Batch geocoding overview

    • Desktop geocoding recommendations

    • ArcGIS Enterprise geocoding recommendations

    • Working with Clients for Enterprise

    Outline

  • System Recommendations Best practices for Enterprise geocoding

    • ArcGIS Server

    - Multiple cores (the more the better)

    - Enough memory for those cores (locator size + 500MB per core)

    - SSD Drive

    - Fast processors

    - Physical hardware (not a VM)

  • ArcGIS Enterprise Batch Geocoding System Architecture

    • Perform batch geocoding behind your

    organizational firewall

    • Get the most performance out of your

    enterprise geocoding services

    • Manage backend resources

    ArcGIS Portal

    GIS Servers

    Geocode Server

    World Locator Service

    SMP Locator Service(s)

    GeocodingTools Geoprocessing Service

    Hosting

    Server

  • How the system works

    GIS Servers

    Geocode Server Geocoding Tools

    Geoprocessing Service

    Hosting

    Server

    Behind the scenes

    Input Table

    Output Feature Layer

    World Locator Service

    SMP Locator Service(s)

    Custom Locator Service

    1. GP Service takes input table and chunks requests into batches of 1000

    2. GP Service uses geocodeAddresses API to call the Geocode Server geocoding service

    3. GP Service implements retry logic to retry failed requests

    4. GP Service receives geocoding results & stitches together results

    5. GP Service makes output available for download by clients

  • Configuring the system for your Organization’s use

    • Publish a locator to Portal via ArcGIS Pro

    • Add the locator as a Utility Service so

    your organization can work with them

    • Scale up the Geocoding Service to meet

    Enterprise needs in Server Manager

    - More cores = more geocoding throughput

    • Configure batch geocoding resources

    available to clients

    - numBatchThreads configured per locator

    ArcGIS Enterprise Large Batch Geocoding

    https://bit.ly/2UspGqa; https://bit.ly/2KF3h7M; https://bit.ly/2C65UJF; https://bit.ly/2WYNHWg

    Configure Threads

    https://bit.ly/2UspGqa https://bit.ly/2UspGqa https://bit.ly/2UspGqa https://bit.ly/2UspGqa https://bit.ly/2UspGqa https://bit.ly/2KF3h7M https://bit.ly/2C65UJF https://bit.ly/2WYNHWg

  • Configuring the system: A closer look

    • Scale up the Geocoding Service to meet

    Enterprise needs in Server Manager

    - More cores = more geocoding throughput

    ArcGIS Enterprise Large Batch Geocoding

    GeocodingTools Geoprocessing Service

    Geocoding Service

    GeocodingTools Geoprocessing Service

    Geocoding Service Instances

    Set Geocoding Service Min instances = 4 Max instances = 4

    https:///server/manager

    https://bit.ly/2UspGqa

  • Configuring the system: A closer look

    • Configure batch geocoding resources

    available to clients

    - numBatchThreads configured per locator

    ArcGIS Enterprise Large Batch Geocoding

    Configure Threads

    Geocoding Service Instances

    GeocodingTools Geoprocessing Service

    Set numBatchThreads = 4

    https:///portal/sharing/rest/portals/self/update

    https://bit.ly/2UspGqa

  • Configuration

    Demos

    https:///portal/sharing/rest/portals/self/update

  • • Batch geocoding overview

    • Desktop geocoding recommendations

    • ArcGIS Enterprise geocoding recommendations

    • Working with Clients for Enterprise

    Outline

  • Clients for geocoding data using Portal

  • Clients that use Enterprise Batch Geocoding Best practices for Enterprise geocoding

    • Use the latest version of Asynchronous Batch Geocoding enabled clients

    - Geocode Table in ArcGIS Pro 2.4 -> Desktop

    - Geocode Locations from Table Mapviewer tool -> Web

    - arcpy.geocoding.GeocodeLocationsFromTable() and arcpy.geocoding.GeocodeFile() -> Scripting

  • Demo in Pro

  • Demo in

    MapViewer

  • Automating Enterprise Geocoding: Geocode Locations from Table Python

  • Automating Enterprise Geocoding: Geocode File Python

  • Clients for geocoding data on a Big Data File Share

    (BDFS Input)

  • GeoAnalytics: Geocode Locations from Table Data in HDFS -> Geocode Locations from Table GeoAnalytics tool

    • If you have the GeoAnalytics Extension enabled

    in ArcGIS Enterprise this tool appears in the map

    viewer

    • Has chunking and retry logic built-in

    • Takes input table from a BDFS in ArcGIS

    Enterprise

  • GeoAnalytics: Geocoding via the Python API Data in HDFS -> Geocode Locations from Table GeoAnalytics tool

    • GeoAnalytics tool exposed via the

    “geoanalytics” module in the Python API

    • Has chunking and retry logic built-in

    • Takes input table from a BDFS in

    ArcGIS Enterprise

  • Configuring Geocode Locations GeoAnalytics Tool Data in HDFS -> Geocode Locations from Table GeoAnalytics tool

    • Add the geocoding service as a utility service

    in your organization

    • Set the

    “percentageMaxAllowedComputeCores” and

    “percentageMaxAllowedComputeMemory”

    system properties

    • Read more at:

    http://enterprise.arcgis.com/en/server/latest/g

    et-started/windows/geoanalytics-settings.htm

    https:///server/admin/system/properties/update

    http://enterprise.arcgis.com/en/server/latest/get-started/windows/geoanalytics-settings.htm

  • Please Share Your Feedback in the App

    Download the Esri

    Events app and find

    your event

    Select the session

    you attended

    Scroll down to

    “Survey”

    Log in to access the

    survey

    Complete the survey

    and select “Submit”

  • Questions?