challenges and success of migrating to an...

6/30/2015

1

Challenges and Success of Migrating to an Enterprise Database in York County, PA

PA GIS Conference ‐ June 16, 2015

Wade Gobrecht – York County Planning Commission

Andrew Ross ‐ GeographIT

Source: Can You Draw All 50 US States?www.thrillist.com

6/30/2015

2

Source: Map of USA | United States Population Density Mapwww.mapofusa.net

Planning Commission

Assessment

911/EMA

Ag Preserve

Human

Services

Economic Developmen

t

GIS in York County – Prior to Enterprise Implementation

‐Different Networks‐Different Software Versions‐ Redundant Datasets ‐ potentially different and no standards‐ Datasets e‐mailed or distributed on CDs quarterly or Annually‐ No Public/Municipal access to data other than manual data extracts – access to limited data on parcel web map

Conservation District

Different Network

SameNetwork

SameNetwork

SameNetwork

Different Network

Different Network

Different Network

6/30/2015

3

Goals

Goal: Design a system that enables each department take advantage of the coordinated data efforts and new standards while allowing them to work independently with their current systems and setup.

• Ultimate Goals

– Enterprise‐Wide Adoption of Official, Up‐To‐Date GIS Datasets

– Promote Enterprise‐Wide Data Sharing

– Consolidate Redundant Datasets (500+ down to 200)

– Adopt State‐Of‐The‐Industry Software and Data Standards

Timeline

• 2006 – Study completed by Penn State and Advanced Skills Center –Recommended a GIS Department or a GIS Director in Commissioner’s office –Voted no by Commissioners

• 2006 – York County GIS Advisory Council Created for Coordination

• 2010 – Grant Funded Enterprise GIS Implementation Plan completed by GeographIT

• 2012/2013 – Enterprise GIS Implementation Started– Hardware / Software Updates

– Educated staff

– Data Investigations

– Database Needs and Design

– Data Migration

– Data Replication to other departments

– Applications Created

• 2014 – Enterprise Database Deployed

6/30/2015

4

BackgroundEnterprise GIS Implementation Plan

• Creation of GIS Management Committee

• Development of Strategies

– Hardware, Software Re‐Purposing & Upgrades

– Identification of Critically Important Datasets

– Data Stewardship and Distribution

– Overcoming Technical Challenges (Network & Licensing)

• Migration Procedures

• Budgetary Estimates


• Inventory of Existing Data and IT Infrastructure

– Network, Hardware, Software, Licenses

• Inventory of GIS Business Practices at YC Agencies

• Inventory of Existing & Required Skills

6/30/2015

5


Administrative Center

Internet

CableModem

10 Mbit, sharedwith nursing home,soon will be 50 Mbit

10 Mbit, to beupgraded to1 Gbit duringrenovations

EMA / 911Agricultural Preservation

MS SQL Server Express for SDE replication, may not be needed depending on bandwidth

SDE Copy of all GIS data from Planning

SDE Copy of Parcel data from Assessment

Ag Preservation and Cadastral Shape Files

Two way update scripts

Licenses:

1 ArcView 3.x

1 ArcInfo or ArcEditor 9.x

Human Services

MS SQL Server Express for SDE replication, may not be needed depending on bandwidth



Licenses:

1 ArcView or ArcEditor 9.x

Soil Conservation

MS SQL Server Express for SDE replication



Conservation Projects Shape File & Access DB for Plan & Practices

Two way update scripts

Licenses:1 ArcView 3.x1 ArcEditor 9.x

Economic Devel. Corp.

MS SQL Server Express for SDE replication



Licenses:1 ArcEditor 9.x2 ArcView or ArcEditor 9.x

DB Transactional Replication or Mirroring

ArcGIS Server+ ArcSDE +

RDBMS + Storage=

Dedicated Data Server for WEB Sites Database level replication backs up Production Data Server (including SDE replicas)

Planning Commission

Assessment Office

PlanningCommission

Switch / Firewall

DMZ4 Gbit

Firewall / Reverse Proxy

WEB Server

DB Transactional Replication or Mirroring

ArcGIS Server+ ArcSDE +

RDBMS + Storage=

Dedicated Data Server for WEB Sites Database level replication backs up Production Data Server (including SDE replicas)

1 GBit

WkStn +GIS Client

WkStn +GIS Client

WkStn +GIS Client

WkStn +GIS Client

Total of21 GISWkstns

Licenses:

2 MS SqlServer

1 ArcGIS Server Enterprise

14 ArcView 9.x

3 ArcInfo 9.x

4 ArcView 3.x

1 ArcPad 7.x

1 Lizardtech Geoexpress

1 Hazus

WkStn +GIS Client

WkStn +GIS Client

WkStn +GIS Client

WkStn +GIS Client

Total of15 GISWkstnsDB Transactional Replication or Mirroring

RDBMS + Storage=

Backup Server for Dedicated Data ServerDatabase level replication should insure that SDE level Copy of Planning data is included

RDBMS + Storage=

Dedicated Data Server for Parcel Production Geo-databaseThis where Parcel data is maintained


Licenses:

2 MS SqlServer

3 ArcInfo 9.x

1 ArcView 3.x

3 ArcInfo 9.x &9 ArcView 3.xcan be replaced by web apps.

RDBMS + Storage=

Dedicated Data Server for Production Geo-database / sThis is where all GIS data is maintained


SDE Copy of EMA /911 data

RDBMS + Storage=

Dedicated Data Server for Production Geo-database / sThis is where EMA GIS data is maintained



DMZ

WEB Server

Switch / Firewall / Reverse Proxy

6 Mbit down1 Mbit up

Licenses:

2 MS SqlServer

1 ArcGIS Server Enterprise

3 ArcView 9.x

1 ArcEditor 9.x

3 ArcInfo 9.x

AdministrativeCenter

NetworkBackbone

4 Gbit 1 GbitOne way

update scriptsParcel Data

Maintained in ArcInfo Workstation Format


Investigationand

RequirementsDesign

Instantiation and Loading

Testing

DataCleanup

FoundIssues?

YES NO

ProductionEnvironment

Vector Data Migration Process Overview

6/30/2015

6

Enterprise GIS Implementation

• Collaborative Effort led by YCPC

• Based on Enterprise GIS Implementation Plan

• With Participation of:

– All Other Concerned YC Agencies

– GIS Consultant: geographIT

Implementation Strategy

• IT Upgrades and Data Migration Follow Separate Overlapping Tracks

• Data Migration:

– Start with Critically Important Data Layers:

• Land Base (Parcels)

• Transportation (Road Centerlines)

– Major Participation of Stakeholders in Data Model Redesign

– Establish Templates and Procedures for Use with All Other Data Layers

– Script Complex Data Migration Procedures

• Simplify Re‐Deployment as New Hardware Becomes Available

6/30/2015

7

Land Base Data

• Parcels Maintained in Original Format by Data Steward (Assessment)

– Tile Based Coverage Model

• Reformatted for Distribution on Recurrent Basis by Automated Scheduled Procedure

– Seamless Geodatabase

• Procedure includes incorporation of CAMA Data

• Data Steward to Switch to Seamless Geodatabase at Future Time

Transportation Data

• Road Centerlines Maintained by Emergency Services– Fields added in for Routing (speeds, one‐ways)

• Format Must Remain Compatible with CAD System– Intermediate step to manually add data until CAD is upgraded

– Script will handle Schema Differences until testing is done

6/30/2015

8

Data Distribution

• Challenges

– All YC Agencies Are Not On Same Network

– Data Exchange Must Go Over Internet

– Low GIS Licensing Level Further Restricts What Agencies Can Do With Geodatabase Replication

Data Distribution

• Solutions

– Coalesce All Data onto Central Server at YCPC and Redistribute from there

– Minimize Use of Geodatabase Replication

– Use Python Scripts and SFTP Server to Move Packaged Sets of Feature Classes

6/30/2015

9

Data Distribution

• Solution: Circular Data Exchange Between YCPC and YC Agencies

• More Robust than Replication in Some Ways

– Failed ‘Sync’ Operation Self‐Repairs During Subsequent Exchange Cycle Without Manual Intervention

– Non‐Replication Exchanges Not Affected By Schema Changes

– Non‐Replication Exchanges Can Be Adjusted to Add or Remove Feature Classes Without Complex ‘Resets’

Planning Commission GIS Database

911GIS Database

Econ All.GIS Database

Ag PresGIS Database Assessment

GIS Database

YCCDGIS Database

Hum. Serv.GIS Database

SFTP Server at 911

Internet Applications and County Staff Desktops

Depts. Pushing updates into Central Database

Web Services

Future Data

Updates

York County Enterprise GIS Database

EnterpriseDatabase

1. Data updated Weekly2. Data Integrity 3. County data standards developed4. Eliminate Data Duplication5. Data/Tools available to County

Departments and Municipal staff

Benefits

6/30/2015

10

Scripts, Scripts and more Scripts

• Land Base Scripts – Builds LandBase Feature Dataset(FD), Feature classes within FD, sets up domains and populates data. Also used to update Parcels and related features on a monthly basis.

• Replication Scripts – Runs on Central database to push out data changes. Run in Departments to grab new data changes and copy to their system.

• Sync Scripts – Sync Edit database to production database nightly

• Compress Scripts – Compresses edit database on a weekly basis.

• Start/Stop Services Scripts – Part of compress scripts but can also be run separately

Scheduled Tasks (scripts)

6/30/2015

11

Accomplishments

• New Enterprise Database

• Standards (Naming Conventions, Fields)

• Domains

• Replication

Next Steps

• Create web apps using services from new database

• Fully integrate data replication with 911 and Assessment

6/30/2015

12

Questions???

Contact Information:

Wade Gobrecht, GISPChief, Information SystemsYork County Planning [email protected]‐771‐9870

Andrew RossSenior GIS [email protected]‐399‐7007

challenges and success of migrating to an...

Documents