www.usask.ca data quality thats par for the course quality assurance methodologies and the data...
TRANSCRIPT
www.usask.ca
Data Quality that’s par for the courseQuality Assurance Methodologies and the Data Quality Golf Card
www.usask.ca
Introduction Our “UDW” Product Our ETL Process Creation of a Quality Assurance Environment
www.usask.ca
The “Up” Methodology Data Quality as a Percentage Data Analytics with the concept of improving Scores and numbers that make sense to
executives Works well in a completely defined problem
space
www.usask.ca
The “Down” Methodology Data Quality score that relates to the number of
errors Data Analytics with the concept of lowering the
score Relates better for Data Sets without completely
defined errors
www.usask.ca
Screens Screening for Data Filtering out the “Dirt” Leaving the “Gold” Our Methodology and Language
www.usask.ca
Orphaned Data Orphaned Data is an artifact of building a Data
Store or Data Warehouse Managing Orphaned Data Testing for Orphaned Data issues
www.usask.ca
What we’re doing The Data Quality Golf Card Using Severity Score, once aggregated called
“Data Quality Index” Meeting with Units, Leaders, and Front-Line staff
to continue to add new tests and define a workflow process for fixing them
www.usask.ca
Types of Tests Tests for our office, and tests for our clients• Data Integrity (our office)• Workflow (our clients)• Missing Values (both)• Entity Resolution (both)
www.usask.ca
Getting Buy-in Using the Score Showing “Unknowns” on reports Describing the impact on institutional reporting
as it relates to the errors being seen
www.usask.ca
The Data Quality Golf Card All tests are organized by the office responsible
for resolving the issue Currently achieved using SQL Queries output
into an Excel pivot table Each score has associated with it a number of
test results, resulting in an index Drilling into the index gives the office what’s
needed to solve the errors
www.usask.ca
Golf Card Demo
www.usask.ca
The Future of the Golf Card Implemented in SAS EBI More Workflow Options Data Quality Dashboard
www.usask.ca
Recommended Reading The Kimball Group Reader • ISBN: 978-0-470-56310-6• Chapter 11.12, Data Quality Screens
MDM in Practice• ISBN: 978-0-470-91055-9
Customer Data Integration• ISBN: 978-0-471-91697-0