FME User Stories from Around the World

Download FME User Stories from Around the World

Post on 24-May-2015

1.264 views

Category:

Technology

11 download

DESCRIPTION

See more FME World Tour 2014 presentations at www.safe.com/recap2014

TRANSCRIPT

  • 1. CONNECT. TRANSFORM. AUTOMATE. Lizard Island: On Location FME Stories From Around the World

2. Iowa, USA Snow Plows, ArcGIS Online, an iPhone, and FME 3. Snow Plows, ArcGIS Online, an iPhone, and FME Eric Abrams, Iowa Department of Transportation 901 Snow Plows 32 34 inches of snow 9400 miles of road 15 million gallons of brine 120,000 tons of salt on 45 inches of snowfall in 2013 ROI - Every dollar spent on AVL returns $6.40 A 10% reduction of salt is $1.4 million dollars in savings 4. Workflow 5. Automatic Vehicle Location Invisible to driver Real-time flow of data position, status, material usage, conditions Data uploading to Amazon cloud FME moves data to Oracle Spatial for internal usage, then to AGOL for public viewing 6. Dashcams Dash-mounted iPhones send image stream when vehicle is in motion FME handles KML generation and upload to Windows Azure 7. AGOL Public Data Access ArcGIS Online keeps public informed with current plow status and conditions 8. Dashcam Feeds Public can also see what the driver is seeing for better awareness of road and weather conditions making winter driving safer. 9. Safer Winter Driving 10. Sharing Open Data on GitHub with FME La Rioja, Spain 11. Sharing Public Data Ide Rioja committed to sharing and collaborating on public data. Spatial Data Sharing taken to the next level Creative Commons License Enter GitHub 12. Why GitHub GitHub is a web-based Version Control System (VCS) which records changes to a file or set of files over time. Allows: commit files to a public repository revert files back to a previous state review changes made over time see who last modified something, and more... 13. Why GitHub 14. Sharing Public Data 15. How does FME Help? Of course FME translates data from Oracle Spatial to GeoJSON for GitHub But first! FME reads the layer list from GitHub using Python Scripted Parameter git pull And after! FME commits updated GeoJSON to GitHub in Shut Down Script git push Scheduled Job on FME Server 16. How does FME Help? 17. The Beauty of GeoJSON in GitHub GitHub supports automatic rendering of GeoJSON repositories using Leaflet.js Looking ahead geojson.io a Chrome extension for editing IDE Rioja plans open collaboration on spatial data with GitHub FME can include links to image data when writing GeoJSON (automatic download service) 18. Learn More at FME User Conference Extended Version of this topic will be presented at the FME International User Conference 19. CC BY-SA 3.0 Tony Nordin FME Server and the Gavle Data Portal Gavle, Sweden 20. FME Server and the Gavle Data Portal Peter Jderkvist, GIS Developer & FME Certified Professional, Community Development Gvle Provides services and centralized workspace organization, FME usage tracking Dynamic forms via client communication with FME Server REST API An evolving UI: Peter recently added upload an irregular polygon to clip 21. Various Maps to DWG Most popular workspace Map type (5), contours, metadata, AutoCAD version XML geometry + parameters triggers FeatureReaders SchemaMapper, clip & output Example output DWG basemap 22. Specialty DWG Requests Custom workspaces generate specialty DWG output for other users Water & sewer mains for local water company Power distribution grid for local provider 23. 3D Model to PDF, Sketchup or DWG Output: Sketchup 8, 3D-PDF and DWG Add streets and water, yes/no Drape roof tops with aerial photography, yes/no Drape elevation model with aerial photography, yes/no Add roof models if they exist, yes/no Some buildings dont have heights, a parameter decides how to treat those, e.g. extrude by 7 meters 24. 1. all parameters set to yes except for add roof models. 2. all parameters set to no. 3. streets water and roof models set to yes Example output sketchup files 1 2 3 25. Reprojection Services On-demand coordinate reprojection File reprojection with error checking and format conversion 26. FME Portal Job History Job details written to history database Over 2100 run since launch Web app shows usage 27. Linear Referencing and Pipe Video with FME Los Altos, CA, USA 28. Linear Referencing and Pipe Video with FME Amanda Graf and Raymond Kinser, FME Certified Professionals, California CAD Solutions Challenge: Map and share non-spatial inspection video footage of all sewer lines for the City of Los Altos. Approach: Use FME and linear referencing methods to QA and position video, creating an automated, repeatable process. 29. Data QA Issues No spatial coordinates or geometry Data inconsistencies across video data vendors and databases Differences between measured pipe lengths from the vendors and the City Inconsistent data entry of defect types 30. Data Cleanup & Homogenization Filter for unwanted and bad data Time stamp formatting Defect notation standardization Match to best known good City records QA for flow direction Catch issues for manual intervention 31. Geometry Creation Adjust video session data for best pipe length Adjust for directionality (video with/against flow) Create geometry using linear measures, chopper, and NeighborFinder 32. Video Data Sharing 33. Results Easy access to data for all Future processing of new observation video automated City saves money on future contracts 34. Tableau Dataset Creation Birmingham, UK 35. Tableau Dataset Creation Dami Sonoiki, FME Certified Professional, Dotted Eyes (Miso) Problem: Tableau does great data visualization, but lacks good mapping capabilities Solution: Use FME to break down polygons 36. Geometry Manipulation Separate individual boundaries with Deaggregator Generalize and reduce vertices Deal with donuts Produce OGC Well Known Text values for polys 37. String Manipulation Format WKT values to extract coordinate strings StringConcatenator appends _part_number supplied by Deaggregator with Code_Count to provide unique ID PolygonPart PolygonPart defines Detail for Tableau reconstruction 38. List Creation ListExploder and ListIndexer create x,y coords for each polygon Tableau-ready format 39. Address Point Frontage Movement Australia 40. Address Point Frontage Movement Rajesh Dhull, FME Certified Professional & Senior Data Engineer, Data Development Asia Pacific, Pitney Bowes Software Problem: Addresses are pinpointed by lot centroids, but services are provided at the street. Solution: Create a value- added, dynamic geocoding dataset with addresses located at the front of the property. Its (almost) always best that your taxi arrives at the front door rather than the living room. 41. Requirements Close to 14 million address points need to be moved to a new position property frontage. The process should be robust, reliable and repeatable every quarter. The process should be able to handle heavy datasets. The process should be able to fit in the existing processes smoothly and should not lead to extra times or delays in the product releases. Source address points in Oracle, referential data (boundaries, streets) in MapInfo Tab files 42. The Approach 1. Create state-wise views in oracle as handling 14 million records in 1 process is not desirable. 2. Create single FME workspace for frontage movement process for states with smaller datasets. 3. Split this process in smaller manageable processes for states with bigger datasets as FME performance varies greatly based on the size of the datasets. 43. FME Workflow Overview Filtered, Buffered Roads Lot Boundary Polygons Candidates for movement Pull address centroids from Oracle Update Oracle 44. Output 14 Million points processed each quarter, automatically. 45. Railway Platform Profiling Brno, Czech Republic Photo Credit: Roman Ba, CSmap 46. Railway Platform Profiling Rudolf Stastny, FME Certified Professional, CSmap, s.r.o. Challenge: Process hundreds of railway platform profile DXF files derived from laser scans to look for areas outside tolerances (preventing collisions) Solution: Automate it with FME 47. Platform Profiles 48. Convex Hull 49. Point Selection 50. DXF Result 51. Telco Spatial Data Portal United Arab Emirates Telco Spatial Data Portal 52. Telco Spatial Data Portal Business Requirement: A Telecom customer wanted a web portal for secure internal data sharing/downloading. Layer and coordinate system selections needed Sources included imagery, vectors, and proprietary data, updated daily by external contractors 53. System Architecture Secure access ArcGIS Server Geodata base FME Download Page FME Server Firewall Users 54. Secure Interface 55. FME Server Processing Single workspace with Custom Transformers Geodatabase Reader Bounding Box create Clip Write to choice of format and projection 56. Budapest, Hungary Laser Scanning Roads and FME 57. Laser Scanning Roads and FME Gyula Sz. Fekete, Head of GIS Development and Data Production, BKK Kzt Zrt. (a company of the Municipality of Budapest) 1. Management of Mobile Laser Scanning (MLS) missions and post- processing 2. Data conversion from CAD-based data capture system to Oracle ArcSDE GDB 3. Point cloud data analysis 58. MLS Mission Management Aim visualize MLS (Mobile Laser Scanning) trajectories locations of MLS projects attribute information of each scanning project project metadata (scanner, driver, acquisition time, etc.) positions of all exposed images attribute information of each images comes from image header information. provide a GDB where post-processing steps can be visualized and modified on a WebGIS GUI. 59. Source Data MLS Trajectory Data Riegl MLS Project Log File Image Headers Post Processing Workflow Tasks 60. Output GDB with - Trajectory information Image information Post-processing workflow tasks WebGIS GUI with editable workflow tasks 61. CAD to 3D Geodatabase Conversion DGN + MDB Tables linked to DGN geometry 62. CAD to 3D Geodatabase Conversion 63. Point Cloud Analysis Find road surface errors based on MLS scanned 3D point clouds Generate vector data to be used for further spatial analysis Read: 3D point clouds and parameters 64. Analyze & Write: Road Surface Errors 65. Analyze & Write: Rugs 66. CC BY-SA Juhanson Belgium Comprehensive Waterways Data QA 67. Comprehensive Waterways Data QA Rob Vangeneugden, FME Certified Professional, GIM nv Challenge: Simplify and automate a complex data validation process for waterways authorities Solution: Create a Django user interface and use FME Server to validate, manage results, and perform database updates 68. The Project Spans multiple waterways authorities 3 primary workspaces 9 embedded custom transformers 27 linked custom transformers +/- 3000 transformers 69. Custom Transformers Uses both linked and embedded Each data type has a specific custom transformer that identifies Update/Insert/Delete by comparing geometry and attributes to operational tables: Terminal Bridge Bollard Fairway Berth Node Lock Lock Chamber 70. Using Parameters Published Passed by Django application Format-specific (3-GML, 9-shape) and user credentials, verify authority Private PostGIS connections Schema parameters Output location 71. Database Updates Example update database process (lock chamber) 72. Benefits Central storage Divided management Detailed validation feedback Summary table Geographic files (downloads) Full update history Easy to expand (data formats, validation rules) Fast data update / Up-to-date database 73. Thank You! Questions? For more information: www.safe.com blog.safe.com