1 copyright © 2013, oracle and/or its affiliates. all rights reserved. · 6 copyright © 2013,...


Oracle Enterprise Data Quality Overview and Roadmap

Martin Boyd – Senior Director, Product Strategy

Mike Matthews – Director, Product Management


The following is intended to outline our general product direction.

It is intended for information purposes only, and may not be

incorporated into any contract. It is not a commitment to deliver

any material, code, or functionality, and should not be relied upon

in making purchasing decisions. The development, release, and

timing of any features or functionality described for Oracle’s

products remains at the sole discretion of Oracle.


Program Agenda

Why Care About Data Quality and Governance?

Oracle Enterprise Data Quality

Roadmap and Demonstration


“Ultimately, poor data quality is like dirt on the windshield. You may be able to drive for a long time with slowly degrading vision, but at some point, you either have to stop and clear the windshield or risk everything.”

Ken Orr, The Cutter Consortium

../../MDM/Platon_-_Information_Management_Fairy_Tale.flv


Companies

Individuals

Data Changes in the Real-World

Source: D&B, US Census Bureau, US Department of Health and Human Services, Administrative Office

of the US Courts, Bureau of Labor Statistics, Gartner, A.T Kearney, GMA Invoice Accuracy Study

• 5,769 individuals in the US will

change jobs

• 2,748 individuals will change

address

• 515 individuals will get married

• 263 individuals will get divorced

• 186 individuals will declare a

personal bankruptcy

Master data changes at a rate of 2% per month

Products

• On average 20% duplicates in

product data

• 90% product introductions fail

• Retailers lose $40B or 3.5% of total

sales each year due to item master

inaccuracy

• 60% of all invoices will have an

error

• Companies with global data Sync

will realize 30% lower IT costs

In one hour… In one hour… In one year…

6

240 businesses will change

addresses

150 business telephone numbers will

change or be disconnected

112 directorship (CEO, CFO, etc.)

changes will occur

20 corporations will fail

12 new businesses will open their

doors

4 companies will change their name


Business Impact of Data Quality

With Bad Data With Good Data

• Reduced ROI

• Increased project risk, time and cost

• Expensive downstream consequences –

wrong shipment, wrong invoices,

incorrect parts…

• Increased ROI on existing systems

• Increased agility

• Increased efficiency

• Increased customer satisfaction

• Increased scalability

“Only 30% of BI/DW

implementations fully succeed.

The top two reasons for failure?

Budget constraints and data

quality.”

“Data integration and data quality are

fundamental prerequisites for the

successful implementation of enterprise

applications, such as CRM, SCM, and

ERP.” ”

“#1 reason CRM projects fail:

Data Quality”


Typical Customer/Party Data Issues

Variation or Error

Example Variation or

Error Example

Sequence errors • Mark Douglas or Douglas Mark Transcription

mistakes • Hannah, Hamah

Involuntary corrections

• Browne – Brown Missing or extra

tokens • George W Smith, George Smith, Smith

Concatenated names

• Mary Anne, Maryanne Foreign sourced

data

• Khader AL Ghamdi, Khadir A.

AlGamdey

Nicknames and aliases

• Chris – Christine, Christopher, Tina Unpredictable

use of initials • John Alan Smith, J A Smith

Noise • Full stops, dashes, slashes, titles,

apostrophes Transposed

characters • Johnson, Jhonson

Abbreviations

• Wlm/William, Mfg/Manufacturing Localization • Stanislav Milosovich – Stan Milo

Truncations • Credit Suisse First Bost Inaccurate dates • 12/10/1915, 21/10/1951, 10121951,

00001951

Prefix/suffix errors

• MacDonald/McDonald/Donald Transliteration

differences • Gang, Kang, Kwang

Spelling & typing

errors • P0rter, Beht Phonetic errors • Graeme – Graham


Typical Product/Item Data Issues

10hp motor 115V Yoke mount

mtr, ac(115) 10 horsepower 115volts

MOT-10,115V, 48YZ,YOKE

This 10hp yoke mounted motor is rated for

115V with a 5 year warranty

10 Caballos, Motor, 115 Voltios

TEAO HP = 10.0 1725RPM 115V 48YZ YOKE MTR

Motor, TEAO, 1725 RPM, 48YZ, 15 Voltios,

Montaje de Yugo, hp = 10

Item Motor

Classification 26101600

Power 10 horsepower

Voltage 115

Mounting Yoke


Putting your Data to Work Common Data Quality Use Cases

System Consolidation/Migration

• Enforce new system standards on

legacy data

Compliance

• Drive consistent data and processes

to meet regulatory requirements

(watchlist screening, anti-money

laundering, tax compliance, etc.)

Application Enablement

• Clean-up and govern application

data (CRM, HR, PLM, Retail

search, etc.)

Business Intelligence Enablement

• Enforce BI standards on disparate

data

MDM Enablement

• Verify, standardize, match and

and merge data from disparate

sources


• How do you know?

• What is the business impact?

• What should you do about it?

Data Quality – Is Your Data “Fit for Purpose”?


Health Check – Is Your Data “Fit for Purpose”?

Understand current data ‘fitness for purpose’

Estimate DQ impacts & ROI

Identify critical issues & quick wins Understand

Improve

Protect

Govern Your

Data

Your Experts

Current

issues,

gaps,

errors

Business &

data

standards


Improve Data, Improve App Performance

Improve ROI and performance of existing applications

Engage users and executives

Bring data to a known, baseline quality – ready to roll-out new

applications and initiatives

Understand

Improve

Protect

Govern

Metrics,

KPIs

Fit for

purpose

data

Parse/

extract

Stand-

ardize

Match/

merge

Verify

Enrich

‘Gold’

data

Apply data standards


‘DQ Firewall’ – Continuous Protection for Information Assets

Continuous, consistent enforcement of standards

High quality data drives ROI

No more DQ projects!

Understand

Improve

Protect

Govern

Hub

Apply data standards/validate

External

sources/

feeds

Data Integration/ETL Non-DQ/MDM-

aware Apps

DQ/MDM-

aware Apps Web

service

call


DQ Governance – Continuous Process Improvement

Monitor ongoing effectiveness

Track and resolve issues

Improve overall effectiveness

Understand

Improve

Protect

Govern

Target

system DQ

metrics

‘Gold’

data

Apply data standards

Source

system DQ

metrics

DQ

process

metrics


Program Agenda





Modernization MDM SOA Big Data

Oracle Data Integration Complete Offering for Enterprise Data Integration

Complete and best-of-breed

approach for enterprise data

integration

Maximum performance with

lower TCO, ease of use and

reliability

Certified for leading

technologies to deliver fast

time to value

Oracle Data Integrator

Oracle GoldenGate


Oracle Data Service Integrator

OLTP

Applications

Legacy

Unstructured

Synchronization Custom BI


Enterprise Data Quality

• Process metrics

• Quality metrics

• Case Management

• Remediation

• Party (individuals,

households) match

• Entity match

• Semantic (category)

match

• Statistical match

• Match review

• Merge/survivorship

• Global parse

• Category parse

• Extract

• Transform

• Address verification &

geocoding

• Substitute

• Enrich

• Classify

• Statistics

• Patterns

• Phrases

• Duplicates

• Completeness

• Max/min values

Profile

Standardize

Match

Govern

Quickly understand data content

Drive conformance to standards

Identify & merge duplicates

Monitor effectiveness & resolve problems

Co

mm

on

Acce

ss/U

I

Enterprise DQ Platform

Enterprise DQ Cloud Services

• Packaged cloud services for cloud applications Enterprise DQ

Matching Cloud

Service

Enterprise DQ

Address Verification

Cloud Service


Enterprise Data Quality

Broadest DQ offering • Best of breed capabilities for both Party Data and Product

Data

• Profiling, standardization, matching, case management,

governance

Most usable DQ offering • Completely integrated offering – designed to work together

• Designed for business and technical users

• Transparent operation and results – no black boxes

Pervasive operation for enterprise quality governance • Within legacy systems and MDM Hubs

• As part of migration/system load

• On data entry/capture

• As part of data movement/transfer

Profile

Standardize

Match

Govern





Co

mm

on

Acce

ss/U

I


Enterprise DQ Cloud Services Enterprise DQ

Matching Cloud

Service

Enterprise DQ


Cloud Service


EDQ Web Services Enforce common DQ standards across the enterprise

Common

Services

Applications App 1 App 2 App 3

Library of

enterprise

standard DQ

services

Any EDQ process may

be called as a real-time

web service

Call any process from

any application to

1. Enforce common

standards

2. Minimize

architectural

changes


Case Management for Governance

Usage • Cases/alerts are assigned a work queues and a priority

• Data specialists sign in and review/resolve issues

• Management reports allow monitoring of work queues and productivity

• Helpful for

o One-time cleanse/migration

o Ongoing governance program

Features • Hierarchical Case/alert functionality

• Configurable Workflows

• Automatic prioritization of cases/alerts

• Timers

• Email Notification Support

• Comprehensive audit trail

• Immediate ad-hoc reporting

Review and resolve exceptions from the DQ process


Data Prep for System Migration/Implementation

EDQ Process

Apps and hubs

Governance and Case Management to ‘Perfect’ Data

• DQ Insight (Dashboard)

• Reporting

• Trend Analysis

• Case Management

• Workflow

• Remediation

Legacy Data ‘Fit for Purpose’ Data


Program Agenda





EDQ Investment Areas

Integrated DQ &

Governance

Integrated best-in-class Customer and

Product DQ

Expand Governance to include

operational confidence reporting

Integration Across Oracle

Deeper Siebel Integration

Out-of-the-box DQ for Fusion Apps

Integration with ODI

Endeca, ATG, EBS…

Cloud/SaaS

SaaS deployment for Fusion Apps

Full clustering and elastic provisioning

support

Cloud DQ Services

Global Rules & UI

Global Identity Resolution

DQ Rules and Reference Data for

major locales

Additional UI Localizations

Advanced Techniques

Statistical parsing & classification

Statistical outlier detection

Entity identification & extraction for Big

Data


EDQ in the Cloud

Cloud Data Services powered by EDQ

• Providing data enhancement services in the Oracle Cloud

• Uses EDQ as the matching engine and to ensure reference

data quality

EDQ in Fusion Apps

• EDQ to be deployed and used by Fusion Apps

• Leveraging Oracle DB and FMW cloud support

EDQ in Managed Cloud

• Growing number of customers already choosing to run full service EDQ in the Oracle Managed Cloud

EDQ powering Partner Cloud Offerings

• Kaygen partnering with Oracle to deliver Data Governance in managed cloud with EDQ

• Several others following suit


EDQ for Fusion Applications

Fusion Applications

Integration

– EDQ deployed in Fusion

Apps as the attached DQ

engine

– Advanced Search, Duplicate

Prevention, Master Data

Matching

– Address Verification and

Cleaning for all countries Profile

Standardize

Match

Govern





Co

mm

on

Acce

ss/U

I


Enterprise DQ Cloud Services Enterprise DQ

Matching Cloud

Service

Enterprise DQ


Cloud Service


EDQ 11 - Major New Features

Case Management Expansion

– Instant reports on high volume data

– Aggregated reports (e.g. activity by period, priority, etc.)

– Improved case search and filter

– Expanded workflow options

Reference Published Processors

– Enables development of ‘locked’ IP to extend EDQ

– Full reuse and upgrade of processors across processes/projects

UI Localization to 9 Languages

– Chinese, Japanese, Korean, Brazilian Portuguese, French, Italian, German,

Spanish, English


EDQ 11 - Improving Productivity

New Job Manager

– User-defined job layouts and canvas notes

– ‘Blocking’ triggers allow jobs to be called within jobs with

execution control

– Additional externalization options

New Process Canvas

– Improved canvas usability and multi-language support

Browser-based Web Service Tester

– Faster testing of EDQ Web Services


EDQ 11 – Other Changes

Oracle Universal Installer

– Automated installation process for all platforms

Fusion Middleware Integration

– Enables use of WebLogic OPSS for security and authentication

– Uses FMW Audit Control to capture key configuration changes

Automated Results Purge capability

Support for Subversion 1.7

Array support in Data Interfaces

Multi-attribute data type converters


EDQP 11 - Major Features New Integrations

– Connector for Endeca Guided Navigation

– Integrated with Agile PLM 9.3.2

Statistical Matching Module

(StatSim)

– Quick Rules Free Configuration

– Match or classify verbose semi-structured

data

– Integrated with Governance Studio

Remediation Capabilities

– Provides List of Values for Data

Enrichment

– Integrated with AutoLearn Workflow


EDQP Drives Endeca Navigation

Integrated Data Quality:

– Populate – Identify, extract and standardize product

dimensions & properties

– Integrate – Automatically create required dimensions

within Endeca (avoid manual dimension setup)

Endeca

Engine

EDQP

Client

Browser Data

Source Data

Source

Data

Source

Improved data improves user experience

Standardize data structure

Standardize data values

Integrated directly into Endeca pipeline

Endeca

Load

Data Preparation

EDQP ‘pushes’ required metadata

into Endeca to create required

navigation dimensions

PIM or any other

data source


EDQ 12c

Data Quality Governance II

Integrated Semantic Data Engine (EDQ-P)

Full WebLogic Server Clustering support

– Shared config for multiple EDQ servers

– Session balancing and failover

– Active-Active Case Management

Oracle Access Manager integration

Hadoop Connectivity

Fully automatable Reference Data Generation


Data Governance with EDQ

Single DQ environment

DQ Engine

Data sources

Real-time checks

Apps and hubs

Enabling People and Process with Technology

• DQ Insight (Dashboard)

• Reporting

• Trend Analysis

• Case Management

• Workflow

• Remediation

Current capabilities to

be enhanced and

combined into a new

cloud-enabled DQ

Governance UI


EDQ Application Integration

• Fusion Applications – Deep integration in progress; planned for Fusion R9 release

• Siebel CRM and UCM– Deep integration in place using services architecture; more stable,

performant, functional and scalable than 3rd Party or OEM integrations

• EBS – Template connectors available for common integrations (customer/party, etc.)

• Salesforce.com – Template connectors available for batch cleansing

• Oracle Product Hub; Fusion Product Hub – Deep integration for batch and real-time load

• ENDECA (Oracle Commerce) – Data cleansing and metadata sync to streamline managing complex

product data schemas for eCommerce

• Agile PLM – Template connectors for batch and real-time validation, BOM validation and BOM sync

Enabling Applications with Data Quality Services

• Application owners are painfully aware of the impact & costs of poor data

• EDQ is investing heavily in providing out-the-box Application DQ solutions


Join the Data Integration and MDM Community

Twitter Facebook Blog LinkedIn YouTube

blogs.oracle.com/dataintegration

blogs.oracle.com/mdm

1 copyright © 2013, oracle and/or its affiliates. all rights reserved. · 6 copyright © 2013,...

Documents