validation and quality control with fme and fme server
DESCRIPTION
Why are we talking about this? - Bad data costs time, money and even lives. - FME and FME Server are powerful tools for checking data and providing feedback - Many partners, integrators and users use FME today for Validation and Quality ControlTRANSCRIPT
![Page 1: Validation and Quality Control with FME and FME Server](https://reader035.vdocuments.mx/reader035/viewer/2022081721/54804514b4795955578b464d/html5/thumbnails/1.jpg)
CONNECT. TRANSFORM. AUTOMATE.
Validation and Quality Control with FME and FME Server
![Page 2: Validation and Quality Control with FME and FME Server](https://reader035.vdocuments.mx/reader035/viewer/2022081721/54804514b4795955578b464d/html5/thumbnails/2.jpg)
CONNECT. TRANSFORM. AUTOMATE.
Validation and Quality Control with FME and FME Server
![Page 3: Validation and Quality Control with FME and FME Server](https://reader035.vdocuments.mx/reader035/viewer/2022081721/54804514b4795955578b464d/html5/thumbnails/3.jpg)
CONNECT. TRANSFORM. AUTOMATE.
Validation and Quality Control with FME and FME Server
![Page 4: Validation and Quality Control with FME and FME Server](https://reader035.vdocuments.mx/reader035/viewer/2022081721/54804514b4795955578b464d/html5/thumbnails/4.jpg)
CONNECT. TRANSFORM. AUTOMATE.
Outline
Why are we talking about Validation and QC ?
What can we check?
Schema
Attribute Values
Geometry
Etc.
Validation Workflows on FME Server
Partner Examples
![Page 5: Validation and Quality Control with FME and FME Server](https://reader035.vdocuments.mx/reader035/viewer/2022081721/54804514b4795955578b464d/html5/thumbnails/5.jpg)
CONNECT. TRANSFORM. AUTOMATE.
Why are we talking about this?
Bad data costs time, money and even lives.
![Page 6: Validation and Quality Control with FME and FME Server](https://reader035.vdocuments.mx/reader035/viewer/2022081721/54804514b4795955578b464d/html5/thumbnails/6.jpg)
CONNECT. TRANSFORM. AUTOMATE.
Why are we talking about Validation and Quality Control?
FME and FME Server are powerful tools for checking data and providing feedback
Many partners, integrators and users use FME today for Validation and Quality Control
![Page 7: Validation and Quality Control with FME and FME Server](https://reader035.vdocuments.mx/reader035/viewer/2022081721/54804514b4795955578b464d/html5/thumbnails/7.jpg)
CONNECT. TRANSFORM. AUTOMATE.
What can we check?
Schema
Feature Type Name
Attribute Present
Attribute Names
Attribute Data Types
Coordinate System
![Page 8: Validation and Quality Control with FME and FME Server](https://reader035.vdocuments.mx/reader035/viewer/2022081721/54804514b4795955578b464d/html5/thumbnails/8.jpg)
CONNECT. TRANSFORM. AUTOMATE.
What can we check?
Checking Schema – Key Tools
Schema Reader
FeatureTypeFilter
StatisticsCalculator
AttributeClassifier transformer
XMLValidator Transformer
JSONValidator
![Page 9: Validation and Quality Control with FME and FME Server](https://reader035.vdocuments.mx/reader035/viewer/2022081721/54804514b4795955578b464d/html5/thumbnails/9.jpg)
CONNECT. TRANSFORM. AUTOMATE.
What can we check?
Attribute Values
Values exist
Data Type (numeric, alpha etc)
Value within in range
No duplicates
![Page 10: Validation and Quality Control with FME and FME Server](https://reader035.vdocuments.mx/reader035/viewer/2022081721/54804514b4795955578b464d/html5/thumbnails/10.jpg)
CONNECT. TRANSFORM. AUTOMATE.
What can we check?
Checking Attribute Values – Key Tools
Tester and TestFilter
AttributeClassifier
AttributeRangeFilter & AttributeFilter
NullAttributeMapper
Matcher
DuplicateRemover
StringSearcher
StatisticsCalculator
![Page 11: Validation and Quality Control with FME and FME Server](https://reader035.vdocuments.mx/reader035/viewer/2022081721/54804514b4795955578b464d/html5/thumbnails/11.jpg)
CONNECT. TRANSFORM. AUTOMATE.
What can we check?
Geometry 2D and 3D
Intersections
Orientation
Duplicate Vertices
Missing Surface Normals
Invalid Solid Voids
etc
![Page 12: Validation and Quality Control with FME and FME Server](https://reader035.vdocuments.mx/reader035/viewer/2022081721/54804514b4795955578b464d/html5/thumbnails/12.jpg)
CONNECT. TRANSFORM. AUTOMATE.
What can we check?
Geometry Checking – Key Tools
GeometryValidator (12 tests and counting)
Detection and repair
old transformers bundles in eg Self-Intersector
GeometryFilter
Matcher
SpatialFilter (Sweco example)
SliverRemover & SpikeRemover
AreaCalculator & LengthCalculator
OrientationExtractor
![Page 13: Validation and Quality Control with FME and FME Server](https://reader035.vdocuments.mx/reader035/viewer/2022081721/54804514b4795955578b464d/html5/thumbnails/13.jpg)
CONNECT. TRANSFORM. AUTOMATE.
Validation Workflows on FME Server
Submit data via email, upload, directory watcher, scheduled task
Provide report or results via email, form, data download, PDF
![Page 14: Validation and Quality Control with FME and FME Server](https://reader035.vdocuments.mx/reader035/viewer/2022081721/54804514b4795955578b464d/html5/thumbnails/14.jpg)
CONNECT. TRANSFORM. AUTOMATE.
Workspace Example
![Page 15: Validation and Quality Control with FME and FME Server](https://reader035.vdocuments.mx/reader035/viewer/2022081721/54804514b4795955578b464d/html5/thumbnails/15.jpg)
CONNECT. TRANSFORM. AUTOMATE.
Validation Workflows on FME Server
![Page 16: Validation and Quality Control with FME and FME Server](https://reader035.vdocuments.mx/reader035/viewer/2022081721/54804514b4795955578b464d/html5/thumbnails/16.jpg)
CONNECT. TRANSFORM. AUTOMATE.
Partner Validation Examples#1 Axmann GeoChecker
1. Users upload data
2. Workspaces Check Schema, Geometry, Attributes
3. Email sent with PDF Report and Bad Data in AutoCAD
![Page 17: Validation and Quality Control with FME and FME Server](https://reader035.vdocuments.mx/reader035/viewer/2022081721/54804514b4795955578b464d/html5/thumbnails/17.jpg)
CONNECT. TRANSFORM. AUTOMATE.
Partner Validation Examples#1 Axmann GeoChecker
Detailed Reports & AutoCAD File of Errors
![Page 18: Validation and Quality Control with FME and FME Server](https://reader035.vdocuments.mx/reader035/viewer/2022081721/54804514b4795955578b464d/html5/thumbnails/18.jpg)
CONNECT. TRANSFORM. AUTOMATE.
Partner Validation Examples#2 Veremes - Qualigéo
Generic Approach to avoid “Chamber of Horrors”
![Page 19: Validation and Quality Control with FME and FME Server](https://reader035.vdocuments.mx/reader035/viewer/2022081721/54804514b4795955578b464d/html5/thumbnails/19.jpg)
CONNECT. TRANSFORM. AUTOMATE.
It's easy
Constraint FME tool
Geometry GeometryValidator
Spatial Relationship SpatialFilter/SpatialRelator
Integrity Constraint Feature Merger
Topology
Attribute Domain String Searcher
Statistic Statistic Calculator
Schema Schema reader
Unique Key Duplicate Remover
![Page 20: Validation and Quality Control with FME and FME Server](https://reader035.vdocuments.mx/reader035/viewer/2022081721/54804514b4795955578b464d/html5/thumbnails/20.jpg)
CONNECT. TRANSFORM. AUTOMATE.
It's complicated
![Page 21: Validation and Quality Control with FME and FME Server](https://reader035.vdocuments.mx/reader035/viewer/2022081721/54804514b4795955578b464d/html5/thumbnails/21.jpg)
CONNECT. TRANSFORM. AUTOMATE.
Hyper generic workspace
• 1 unique workspace to validate all formats, all data models, all constraints...
Using an external description of the data model and constraints
Solution
![Page 22: Validation and Quality Control with FME and FME Server](https://reader035.vdocuments.mx/reader035/viewer/2022081721/54804514b4795955578b464d/html5/thumbnails/22.jpg)
CONNECT. TRANSFORM. AUTOMATE.
Partner Validation Examples#3 Metria SwedenEnvironmental Monitoring Data – ISO
QC-service on FME server
Metadata
XML
INSPIREWFS/SOS
QC-report
Metadataportal
DatasetGML
![Page 23: Validation and Quality Control with FME and FME Server](https://reader035.vdocuments.mx/reader035/viewer/2022081721/54804514b4795955578b464d/html5/thumbnails/23.jpg)
CONNECT. TRANSFORM. AUTOMATE.
Partner Validation Examples#3 Metria SwedenEnvironmental Monitoring Data – ISO
![Page 24: Validation and Quality Control with FME and FME Server](https://reader035.vdocuments.mx/reader035/viewer/2022081721/54804514b4795955578b464d/html5/thumbnails/24.jpg)
CONNECT. TRANSFORM. AUTOMATE.
Partner Example #4
1Spatial - UKPostcodeValidator
![Page 25: Validation and Quality Control with FME and FME Server](https://reader035.vdocuments.mx/reader035/viewer/2022081721/54804514b4795955578b464d/html5/thumbnails/25.jpg)
CONNECT. TRANSFORM. AUTOMATE.
Thank You!
Questions?
For more information:
www.safe.com