from oracle to the web - automating spatial data updates
DESCRIPTION
TRANSCRIPT
![Page 1: From Oracle to the Web - Automating Spatial Data Updates](https://reader034.vdocuments.mx/reader034/viewer/2022051515/5541d9d25503469a0d8b4596/html5/thumbnails/1.jpg)
From Oracle to the Web - Automating Spatial Data Updates David Smith, FRS Program Manager, EPAAmy Ramsdell, GIS Manager, Blue Raster
April 8
![Page 2: From Oracle to the Web - Automating Spatial Data Updates](https://reader034.vdocuments.mx/reader034/viewer/2022051515/5541d9d25503469a0d8b4596/html5/thumbnails/2.jpg)
Overview
The Need: Automate spatial data refresh The Solution: Technologies used ArcGIS Server (AGS) map service final output FME workspaces that automate refresh process
File geodatabase (fgdb) creation Metadata refresh
Python shutdown script in workspace FME Server web services used EPA future plans for FME Questions
![Page 3: From Oracle to the Web - Automating Spatial Data Updates](https://reader034.vdocuments.mx/reader034/viewer/2022051515/5541d9d25503469a0d8b4596/html5/thumbnails/3.jpg)
The Need:
Automate refresh of map service serving spatial data
Operational database system The Facility Registry Service (FRS) is a centrally managed
database that identifies facilities, sites or places subject to environmental regulations or of environmental interest.
Disconnected static file for map service Fgdb faster performance in AGS Lessen activity on operational database
Monthly manual refresh
![Page 4: From Oracle to the Web - Automating Spatial Data Updates](https://reader034.vdocuments.mx/reader034/viewer/2022051515/5541d9d25503469a0d8b4596/html5/thumbnails/4.jpg)
The Solution:
![Page 5: From Oracle to the Web - Automating Spatial Data Updates](https://reader034.vdocuments.mx/reader034/viewer/2022051515/5541d9d25503469a0d8b4596/html5/thumbnails/5.jpg)
ESRI ArcGIS Server Final Output
Web server hosting GIS web services
REST endpoint for map servicehttp://geodata.epa.gov/ArcGIS/rest/services/OEI/FRS_INTERESTS/MapServer
Underlying data: file gdb 28 layers (tables in fgdb)
![Page 6: From Oracle to the Web - Automating Spatial Data Updates](https://reader034.vdocuments.mx/reader034/viewer/2022051515/5541d9d25503469a0d8b4596/html5/thumbnails/6.jpg)
Data Refresh FME Workspace
Oracle Spatial reader
ESRI File Geodatabase writer
Whole bunch of filtering in-between to create separate tables in fgdb Could build the
filtering in Oracle before it reaches FME
![Page 7: From Oracle to the Web - Automating Spatial Data Updates](https://reader034.vdocuments.mx/reader034/viewer/2022051515/5541d9d25503469a0d8b4596/html5/thumbnails/7.jpg)
Oracle Spatial Type Reader
Required for spatial object: Table, materialized view, or view
Shape field is SDO_GEOMETRYCREATE TYPE sdo_geometry AS OBJECT ( SDO_GTYPE NUMBER, SDO_SRID NUMBER, SDO_POINT SDO_POINT_TYPE, SDO_ELEM_INFO SDO_ELEM_INFO_ARRAY, SDO_ORDINATES SDO_ORDINATE_ARRAY);
USER_SDO_GEOM_METADATA table entry Metadata for spatial tables owned by schema
![Page 8: From Oracle to the Web - Automating Spatial Data Updates](https://reader034.vdocuments.mx/reader034/viewer/2022051515/5541d9d25503469a0d8b4596/html5/thumbnails/8.jpg)
ESRI File Geodatabase API Writer
Non-ArcObjects access to fgdb
C++ API
Does not require an ArcGIS license on FME Server or Desktop
Works in ArcGIS Server 10x
![Page 9: From Oracle to the Web - Automating Spatial Data Updates](https://reader034.vdocuments.mx/reader034/viewer/2022051515/5541d9d25503469a0d8b4596/html5/thumbnails/9.jpg)
Python Shutdown Script in Workspace
In Workspace Parameters – Advanced option
![Page 10: From Oracle to the Web - Automating Spatial Data Updates](https://reader034.vdocuments.mx/reader034/viewer/2022051515/5541d9d25503469a0d8b4596/html5/thumbnails/10.jpg)
Shutdown Script in 3 Steps
1. Package the output zipfgdb(geodatabase, path)
2. Refresh the map service updateArcGISServer(server, services, _ZIP,
_ZIPDESTINATION)
3. Notify dependent subscription workspace NotifySuccess()
![Page 11: From Oracle to the Web - Automating Spatial Data Updates](https://reader034.vdocuments.mx/reader034/viewer/2022051515/5541d9d25503469a0d8b4596/html5/thumbnails/11.jpg)
Shutdown Script: 1. Package the Output
Zip the file geodatabase
![Page 12: From Oracle to the Web - Automating Spatial Data Updates](https://reader034.vdocuments.mx/reader034/viewer/2022051515/5541d9d25503469a0d8b4596/html5/thumbnails/12.jpg)
Shutdown Script: 2. Refresh Map Service
1. Stop map service
2. Delete current fgdb
3. Copy to shared network drive
4. Extract new fgdb
5. Start map service
1
2
4
3
5
![Page 13: From Oracle to the Web - Automating Spatial Data Updates](https://reader034.vdocuments.mx/reader034/viewer/2022051515/5541d9d25503469a0d8b4596/html5/thumbnails/13.jpg)
Refresh Map Service: Stop/Start Services
Stopping and starting AGS map service
![Page 14: From Oracle to the Web - Automating Spatial Data Updates](https://reader034.vdocuments.mx/reader034/viewer/2022051515/5541d9d25503469a0d8b4596/html5/thumbnails/14.jpg)
Refresh Map Service: AGS Token Access
Generate token for admin access
![Page 15: From Oracle to the Web - Automating Spatial Data Updates](https://reader034.vdocuments.mx/reader034/viewer/2022051515/5541d9d25503469a0d8b4596/html5/thumbnails/15.jpg)
Refresh Map Service: Replace Fgdb
Delete existing fgdb and extract zip
![Page 16: From Oracle to the Web - Automating Spatial Data Updates](https://reader034.vdocuments.mx/reader034/viewer/2022051515/5541d9d25503469a0d8b4596/html5/thumbnails/16.jpg)
Shutdown Script: 3. Notify Subscription
![Page 17: From Oracle to the Web - Automating Spatial Data Updates](https://reader034.vdocuments.mx/reader034/viewer/2022051515/5541d9d25503469a0d8b4596/html5/thumbnails/17.jpg)
FME Server
![Page 18: From Oracle to the Web - Automating Spatial Data Updates](https://reader034.vdocuments.mx/reader034/viewer/2022051515/5541d9d25503469a0d8b4596/html5/thumbnails/18.jpg)
FME Notification Service - Topics
Create a Success and a Fail notification topic to perform some action based on the data refresh workspace execution status
![Page 19: From Oracle to the Web - Automating Spatial Data Updates](https://reader034.vdocuments.mx/reader034/viewer/2022051515/5541d9d25503469a0d8b4596/html5/thumbnails/19.jpg)
FME Notification Service - Subscriptions 2 subscribers
Email service to notify when the workspace fails Metadata workspace runs if data refresh workspace completes
successfully
![Page 20: From Oracle to the Web - Automating Spatial Data Updates](https://reader034.vdocuments.mx/reader034/viewer/2022051515/5541d9d25503469a0d8b4596/html5/thumbnails/20.jpg)
Metadata about Map Service
FGDC – Compliant metadata required in XML format XML file updated for date of fgdb refresh
https://edg.epa.gov/
![Page 21: From Oracle to the Web - Automating Spatial Data Updates](https://reader034.vdocuments.mx/reader034/viewer/2022051515/5541d9d25503469a0d8b4596/html5/thumbnails/21.jpg)
Subscriber: Metadata Refresh Workspace
![Page 22: From Oracle to the Web - Automating Spatial Data Updates](https://reader034.vdocuments.mx/reader034/viewer/2022051515/5541d9d25503469a0d8b4596/html5/thumbnails/22.jpg)
EPA Future Plans for FME
Kick off workspace from Oracle instead of manually in FME Server’s Services interface – good for more frequent refresh cycles than monthly using FME’s Job Submitter service http://
servername.com/fmejobsubmitter/Samples/workspacename.fmw?FMEServerHost=http://servername.com&SuccessTopic=InternetFGDB&Token=4fab5b76c6yufed2c4731s3d0387bfb366569949&FailTopic=InternetFGDBFail&UserName=admin&Password=password&opt_showresult=false&opt_servicemode=sync
Possibly enhance FRS Query Tools by adding spatial selections and making more file formats available
Integrate into other EPA program map service refresh flows
Research FME’s ability to translate Resource Description Framework (RDF), a semantic web data format
![Page 23: From Oracle to the Web - Automating Spatial Data Updates](https://reader034.vdocuments.mx/reader034/viewer/2022051515/5541d9d25503469a0d8b4596/html5/thumbnails/23.jpg)
EPA Facility Registry Services (FRS)
![Page 24: From Oracle to the Web - Automating Spatial Data Updates](https://reader034.vdocuments.mx/reader034/viewer/2022051515/5541d9d25503469a0d8b4596/html5/thumbnails/24.jpg)
Thank You!
Questions? For more information:
David Smith, EPA Facility Registry Service [email protected]
@DruidSmith
Judith Doherty, FRS Project Manager, Indus Corporation [email protected]
Amy Ramsdell, Blue Raster [email protected]
www.blueraster.com/blog @AmyRams