how to efficiently transform non-spatial data using fme

How to Efficiently Transform Non-Spatial Data using FME

March 13, 2013

Who Are We?

Dale LutzCo-Founder andVice President of DevelopmentSafe Software

Aaron KoningFME Server Product ManagerSafe Software

Daniel RiddleGIS Specialist for SYNCADD Systems Inc.

Kristofor CarleSoftware Developer for SYNCADD Systems Inc.

Guest Speakers

Questions are Encouraged!

This morning’s Q&A Support:

Mark StoakesManager Professional Services Safe Software

Iris Gutowski Product Support Specialist Safe Software

Powering The Flow of Spatial Data

FME – Feature Manipulation Engine

Powering The Flow of Data

FME – Feature Manipulation Engine

FME Capabilities

Transform Data to Use and Share

Convert spatial data between hundreds of formats

Transform spatial data into the

precise data model you need

Integrate multiple different data types into a single data model

Share spatial data with people where, when and how they need it

Workbench: Graphical Data Flow Authoring

Getting started page:

http://fme.ly/GetStarted

Attend a weekly FME Desktop overview webinar:

http://fme.ly/WeeklyIntro

New to FME?

Poll: How much of your data is Non-Spatial?

Beats Writing Code Easier to maintain Faster to configure

Powerful Transformations and Filtering

Supports Most Common Formats

Consistent Handling of Spatial and Non-Spatial in one tool

Why FME for Non-Spatial?

Spatializing Transformers

Spatializing Transformers

FME Supported Formats

Poll: What Non-Spatial Formats are of most interest to you?

1 - Old School Non-Spatial

Egg Information (CAT)NEST_ID EGG_ID DATE_LAID DATE_HATCHED INCUBATION_DAYS 20 0 20040819 20040826 7 20 1 20040823 20040829 6 20 2 20040818 20040826 8…

Species (CSV)NestId,Species100,BLUE JAY101,AMERICAN GOLDFINCH…

1 - Old School Non-Spatial

Create a DBF file of: Number of Eggs Average Incubation

… for each species of bird

DEMO

2 – The Office

2 – The Office

2 – The Office

Reorganize the Vancouver Business Licenses Excel File Separate tabs for license state Create summary tab Flag errors into a new file

DEMO

3 – Hipster

3 – Hipster

3 – Hipster

3 – Hipster

Check if my electric car needs charging Read the Tesla Vehicle Webservice (which returns JSON) Explode the JSON Check if charging needed

Send email if yes Honk horn if NO

DEMO

4 – Corporate

4 – Corporate

Update Oracle with results from complex MS SQL Server query

DEMO

5 – Cloudy

5 – Cloudy

Publish data from SalesForce to Socrata Filter out bad data along the way Plus do schema restructuring for data

sharing

DEMO

Daniel RiddleGIS Specialist for SYNCADD Systems Inc.

Kristofor CarleSoftware Developer for SYNCADD Systems Inc.

Guest Speakers

Story 1: Data Upload Monitoring

• The Mission: Monitor data uploaded via a web interface to an Army Geospatial Data Warehouse for compliance and data model validation, reporting the results.

• The Solution: Use FME Server and custom transformers to run QA tests and email the results as Excel spreadsheets.

SYNCADD

• Custom transformers are created and source user parameters are published to leverage FME Server.

• Readers Used: Schema; ESRI Personal, File, & SDE Geodatabase

SYNCADD

Custom transformers complete various tests on metadata tags, schema feature classes, and schema attributes.

SYNCADD

Results are exported as Microsoft Excel spreadsheets and emailed to the user using FME Server.

• The Mission: Evaluate, document, and reproduce a non-spatial ETL process currently maintained in MS Access so that business experts (not just data analysts/DBAs) can understand it.

• The Solution: Use FME to reorganize the process into diagrams in custom transformers. Add visual QA/QC checks directly into the workflow.

Story 2: Non Spatial MS Access ETL

SYNCADD

• Before: SQL queries in MS Access

SYNCADD

• After: FME creates a diagram that is easy to understand.

SYNCADD

• Inspection points and FME Data Inspector make it easy to evaluate the process even on the fly in a meeting.

• Visually compare link counts on printed diagrams after each data import to look for possible problems.

SYNCADD

Don’t let the Spatial scare you off!

FME is a powerful Non-Spatial problem solving machine

FME is not just for the desktop FME Server is engineered for enterprise data

transformation Automate your non-spatial workflows

Summary

2013 FME World Tour!

40+ FME User Meetings happening world wide and one live stream

Register atwww.safe.com/worldtour

Upcoming webinars

How to Easily Read and Write CityGML Data Using FME – March 27

How to Load Spatial and Non-Spatial Data into Terdata using FME – April 3

PostGIS 2.0 – How to Improve Interoperability using FME – April 24

Recorded Webinars: http://fme.ly/webinars

View the offerings at: http://fme.ly/online

Poll: Would you like more information on our free training options?

We’ll Be Following Up

Thank You!

For more information, contact: Sales

info@safe.com Support

support@safe.com (604)501-9985 ext. 278

Aaron aaron.koning@safe.com

Dale dal@safe.com

Daniel daniel.riddle@syncadd.com

Kristofor kristofor.carle@syncadd.com