10 worst practices in master data management

Worst Practices in Master Data Management

Mike FergusonManaging DirectorIntelligent Business StrategiesInformation Builders WebinarApril 2016

2

About Mike Ferguson

Mike Ferguson is Managing Director of Intelligent Business Strategies Limited. As an independent analyst and consultant he specializes in business intelligence, analytics, data management and big data. With over 34 years of IT experience, Mike has consulted for dozens of companies, spoken at events all over the world and written numerous articles. Formerly he was a principal and co-founder of Codd and Date Europe Limited – the inventors of the Relational Model, a Chief Architect at Teradata on the Teradata DBMS and European Managing Director of DataBase Associates.

www.intelligentbusiness.bizmferguson@intelligentbusiness.biz

Twitter: @mikeferguson1Tel/Fax (+44)1625 520700

3

Topics

Ten worst practices in MDM

Succeeding with MDM – a few guidelines for success

4

Worst Practice #1 - Implementing MDM With No Business Case

MDM System

CR

U

D

Prod

Asset

Cust

ImpactProject fails because of lack of sponsorshipCost of operating remains higher than it should be

Duplicate business process consolidation and rationalisation not possible

Return on investment

5

Building A Business Case For MDM Example – Customer Master Data Anomalies In ERP System

ERP

What happens if you have to invoice a customer?

What happens when you receive a payment from a customer?

Do you have duplicate customers in your ERP system(s)?

Duplicate customers?

Change customer details

If you change the details of a customer address do you change all duplicates?

Does your ERP system send customer data to other systems? If so does it send all duplicates? What happens if duplicates are not in sync?

6

Master Data Maintenance - The Problem of Multiple Data Entry Systems and Master Data Synchronisation

Mortgage System

Customerdata subset

BranchBankingSystem

Customerdata subset

LoansSystem

Customerdata subset

ERPSystem

Customerdata subset

Credit Card

SystemCustomer

data subset

Call CentreSystem

Customerdata subset

The “synchronisation

nightmare”

This has to be done for changes to EVERY master data entity

The problem gets worse as you add more applications

7

Master Data Synchronisation – The Spaghetti ArchitectureComplexity & Lack of Integration Works Against Business

Where is the complete set of master information?

How do I get the master data I need when I need it?

With so many definitions for master data what does it mean?

Can I trust it? Is it complete and correct? How do I get it in the form I

need? How do I know where it

goes and if it is correct? How do I control it?

Spaghetti Interfaces between systems

How much does it cost to operate this way??!

8

Inconsistent Master Data Can Disrupt Operations and Drive Up Costs Due To Manual Intervention Being Needed

order credit check

fulfill ship invoice paymentpackage

Order to cash process

prod custasset

Master data

X

How many people do you employ to fix and reconcile data because it is not synchronised?

What master data entities are used in your core processes

In what systems in your core processes does it reside?

Where in your core processes is master data created?

Where in your core processes is it consumed?

9

XYZ Corp.

Many Companies Have Business Units, Processes & Systems Organised Around Products and Services

Customers/Prospects

Product/service line 1

order credit check

fulfill ship invoice paymentpackage

Product/service line 2

Product/ service line 3Cha

nnel

s/O

utle

ts

order credit check

fulfill ship invoice paymentpackage

order credit check

fulfill ship invoice paymentpackage

Order(product line 1)

Order(product line 2)

Order(product line 3)

Enterprise

What is the im

pact on the customer is customer and product

master data is not managed across these processes?

10

Business and Data Complexity Can Spiral Out Of Control if Processes & Systems Are Duplicated Across Geographies

Product line 1

Product line 2

Product line 3

Product line 1

Product line 2

Product line 3Product line 1

Product line 2

Product line 3

Product line 1

Product line 2

Product line 3

Product line 1

Product line 2

Product line 3

Suppliers

Products/Services

AccountsAssets

Employees

Customers Partners

Materials

11

MDM Business Case Recommendations

Quantify the business impact of anomalies in your business caused by lack of synchronised master data on• Core operational business processes• Decision making • Compliance

Quantify the cost of fixing those anomalies by implementing MDM

Put together a set of candidate business cases ranked in order of return on investment

Use business SMEs to help you

MDM System

CR

U

D

Prod

Asset

Cust

12

Worst Practice #2 - Using A Data Warehouse As Your MDM System

EDWmart

DW & marts

order fulfill ship invoice paymentpackage

New customer?

Create customer process

Impact• Turns the DW into an OLTP system• What if several DWs exist?• DW becomes embroiled in synchronisation of all

OLTP systems• OLTP and analytics become entangled• If transactions occur 24 x 365 then the DW must

become 24 x 365• DW tables are typically de-normalised• Changes to DW impact OLTP systems• ……

13

MDM System

C

R

U

D

Prod

Cust

Asset

MDM Versus A Data Warehouse – They Should Be Separate Systems

Versus

Integrated Master DataNormalised Master DataHistorical Master DataSingle Customer ViewSingle Product ViewMaster Data de-coupled from all systemsMaster data optimised for CRUDMDM System is a data source feeding BI System AND Operational Systems

Integrated Master DataDe-Normalised Master DataHistorical Master DataSingle Customer ViewSingle Product ViewMaster Data in the BI System OnlyMaster data optimised for analysisEnterprise DW feeds data marts

DW

mart

mart

mart

Dat

a in

tegr

atio

n

Operational systems BI System

Mas

ter d

ata

inte

grat

ion

Operational systems

14

(SBV definitions)

C

R

U

D

prod cust

asset

Impact of Master Data Management on DW/BI Systems

Mas

ter d

ata

inte

grat

ion

Operational systems

MDM System

Enterprise Data Warehouse has shared common dimension data

transaction data

DW

Historic data

DF

D D

Dtime

product

Customer

FD

location

Dat

a in

tegr

atio

n

BI Tools(Reporting and Analysis)

Data Virtualization

MDM is a data source to data warehousing systems to improve consistency of dimensional data

15

Worst Practice #3 - Building An MDM System Without Understanding How Your Master Data Is Maintained

Where is master data maintained?• What processes?• What applications?

Call CentreSystem

Customerdata subset

ProductData subset

BranchBankingSystem

Customerdata subset

ERPSystem

Customerdata subset

Credit Card

SystemCustomer

data subsetProduct

data subsetProduct

data subset

changes changeschanges changesDataEntry

Systems

Mortgage System

Customerdata subset

LoansSystem

Customerdata subset

Productdata subset

Productdata subset

changes changes

Are your processes duplicated?

You WILL compromise the integrity of master data if you don’t know where it is

maintained

16

Call CentreSystem

Customerdata

subset

Productdata

subset

If The MDM System Becomes A System Of Record (SOR) Then DES Changes To Master Data Flow To MDM System

BranchBankingSystem

Customerdata

subset

ERPSystem

Customerdata

subset

Credit Card

SystemCustomer

data subset

CustomerSOR

Productdata

subset

Productdata

subset

ProductSOR

changes changeschanges changes

changes changes

Multiple Data Entry Systems still

maintain the master data

Mortgage System

Customerdata

LoansSystem

Customerdata

Productdata

subset

Productdata

subset

SOR = System Of RecordDES= Data Entry System

DataEntry

Systems

DataEntry

Systems

17

Call CentreSystem

Customerdata

subset

Productdata

subset

If The MDM System Becomes A Data Entry System Then Do You Know The Impact of Change? It Is Very Significant

Mortgage System

Customerdata

subset

BranchBankingSystem

Customerdata

subset

LoansSystem

Customerdata

subset

ERP System

Customerdata

subset

Credit Card

SystemCustomer

data subset

CustomerDES& SOR

Productdata

subset

Productdata

subset

Productdata

subset

Productdata

subset

ProductDES & SOR

changes changeschanges changes

changes changes

Customer master data changes

Product master data changes

SOR = System Of RecordDES= Data Entry System

Do you understand the impact of introducing centralised data

entry on an MDM system?

18

Recommendation– Identify Master Data “Producer” and “Consumer” Applications In ALL Your Processes

C

R

U

D

customer

New Customer Process

CreateNew Customer

What processes andapplications create master data?

InboundWhat processes use master data created elsewhere in the business?

Outbound

Customer serviceFinanceDistributionoperational & BI systems

Sales & Marketing

E.g. Manufacturing

19

Worst Practice #4 - Multiple MDM Systems For The Same Master Data Entity

MDM System

C

R

U

D

Customer

MDM System

C

R

U

D

Customer

MDM System

C

R

U

D

Customer

ImpactCost! - OverspendMaster data integrity potentially compromised?

Duplicate master data with different IDs?

Maintenance?

Major opportunity for error if these are silos

Which one is the master?

20

Distributed Master Data With Multiple Overlapping Subsets Needs Very Careful Management

C

R

U

D

prod custasset

C

R

UD

C

R

UD

C

R

UD

Global ID Col 1

Col 2 Col 3 Col 4

Shared attributes

Global ID Col 1

Col 2 Col 3 Col 4 Col 11 Col 12

Global ID Col 8 Col 9 Col 10Global ID Col 1

Col 2 Col 5 Col 6 Col 7

Local hub Local hub

Local hubmust remain

read only

must remain read only

Identical to central master

Identical to central master

Identical to central master

exclusive to local environment

exclusive to local environment

exclusive to local environment

can be created/updated

sync

sync

sync

line of business 1

line of business 2

Enterprise wide

line of business 3

sync

MUST remain read only as maintenance of shared attributes is done centrally

21

A More Efficient Way Might Be To Combine MDM And Data Virtualisation To Get Multiple Virtual Views

MDM System

C

R

U

D

Customer

ViewView View

Data Virtualisation

ViewViewView

22

Worst Practice #5 - Building an MDM System And Thinking You Are Done

MDM System

C

RU

D

Prod

Asset

Cust

budget

Change management

No budget !

23

Where Do You Start The Change Process?

You need to assemble people on your team that know• Your existing processes associated with master data• Your existing Data Entry Systems where master data is

maintained• How master data flows between people and existing systems

A master data change management program has to simplify your ‘spaghetti architecture’ to create benefits

Simplify

24

Understanding The Impact of Introducing An Enterprise Master Data Management System

Applications

Data

Processes & workflows

User Interfaces

People Documents

ChangeManagement

Changes need to be made to all of

these

25

Worst Practice #6 – Not Including Master Data Changes In Inbound MDM System Data Integration

BranchBankingSystem

Customerdata

subset

ERPSystem

CustomerData

Credit Card

System

changes changesDataEntry

Systems

Customerdata

subset

MDM System

C

RU

D

Prod

Asset

Cust

Call CentreSystem

Customerdata

subset

BI/DWSystem

changes

✓You WILL

compromise the integrity of master data if you do this

✓

26

Worst Practice #7 – Not Using MDM For Outbound Synchronisation

BranchBankingSystem

Customerdata

subset

ERPSystem

CustomerData

Credit Card

System

changes changesDataEntry

Systems

Customerdata

subset

MDM System

C

RU

D

Prod

Asset

Cust

Call CentreSystem

Customerdata

subset

BI/DWSystem

changes

✓

✗✗

Should come from MDM

You WILL compromise the

integrity of master data if you do this

27

Worst Practice #8 NOT Recognising The Value Of MDM In A DQ Firewall To Validate Transaction Data

order credit check

fulfill ship invoice paymentpackage

Order-to-Cash Process

An ideal situation would be smooth operation, increased automation, no delays, no defects and no unplanned operational cost

Orders

Business operational transaction processing – the ideal situation

28

Data Issues In Transaction Processing Can Impact Profitability By Causing Unplanned Operational Costs

order credit check

fulfill ship invoice paymentpackage

Data errors

Orders

Order-to-Cash Process

errorserrors

££

data qualityproblems e.g.

missing or wrong data on order entry

£

Unplanned operational cost = (£ + £££ + ££) * Number of Orders

£££

manual interventionand process

delays

All these defects add up to unplanned operational cost of processing an Order

Whatever you do has to reduce unplanned operational cost

Domino impact

What about other types of transactions

have data related problems?

29

The Impact of Data Anomalies In Transaction Processing As The Business Scales Can Be Considerable

order credit check

fulfill ship invoice paymentpackage

Data errors

Orders

Order-to-Cash Process

errorserrors

££££

data qualityproblems e.g.

missing or wrong data on order entry

£££

Unplanned operational cost increases as the business scales if anomalies are not fixed and data is not governed

• E.g. growth in back office headcount to deal with anomalies

£££££££

manual interventionand process

delays

Domino impact

30

The Value Of MDM In A Data Quality Firewall In Validating In-Bound Transaction Data

Validate& enrich

C

R

Uprod client

asset

D

e.g. Order

Validate, enrich and resolve identity

Master data

DQ services

DQ Firewall

31

Call CentreSystem

Customerdata

subset

Worst Practice #9 – Allowing Conflicting Constraints In ‘Consumer’ Systems On Outbound Master Data Synchronisation

Risk Management

BI SystemMortgage System

BranchBankingSystem

LoansSystem

ERPSystem

Credit Card

SystemCustomer

data subset

Customerdata

subset

Customerdata

subset

Customerdata

subset

Customerdata

CustomerDES & SOR

Transactions

Customer master data

changes

Transactions

Transactions TransactionsTransactions Transactions

DELETE CascadeUPDATE Cascade

Customerdata

subset

DELETE RestrictUPDATE Restrict

DELETE Set NullUPDATE Set Null

Nulls

NOT Nulls

CHECK column IN (‘A’, ‘B’, ‘C’)

CHECK column IN (‘A’, ‘D’, ‘E’, ‘F’)

SOR = System Of RecordDES= Data Entry System

You WILL compromise the

integrity of master data if you allow this

32

Outbound Master Data Synchronisation Recommendations

Master data synchronisation processes are needed• The master data hub is the source• The disparate consumer system is the target

Synchronisation CANNOT be based on a “fire and forget” approach

Schema and constraint correction may be needed in disparate systems to uphold master data integrity across the enterprise

33

Worst Practice #10 – Not Using Common Processes And Services To Maintain Master Data

MDM System

Prod

Asset

Cust

MDM System

C

RU

D

Prod

Asset

Custversus

Common services and processes • Facilitate re-use across multiple

applications• Reduce application development and

maintenance costs • Drive consistency across applications

Master Data Processes e.g. New Customer, New Product

Succeeding With Master Data Management- A Few Guidelines For Success

35

Know Your Processes !

You won’t find good business cases, and you can’t implement MDM or master data change

management unless you know how existing processes and applications work with master data

36

Identify Processes And Their Activities That Use and Maintain Master Data

Process can span multiple organisational

departments

Order Entry, Fulfilment and Tracking Process

Maintains Customer data +Access product data

Access product data

Access Customer data +Access product pricing data

Access customer data

37

Establish A Data Governance Operating Model – Master Data Entity and Transaction Based Approach

Data Gov control board

Business data

steward

Business data

steward

Business data

steward

Data Gov control board

Business data

steward

Business data

steward

Business data

steward

Data Gov control board

Business data

steward

Business data

steward

Business data

steward

Enterprise Data Gov control board

All control boards have a dispute resolution process

Control board approval processes for data naming, integrity rules….

ProductClient OrdersDispute

Resolution process

Dispute Resolution

process

Dispute Resolution

process

Dispute Resolution

process

Customer OrdersProduct

Operating model is independent of

location and line of business

Virtual community Virtual communityVirtual community

sponsor

38

Data Stewards Should Be Accountable For The Consistency And Quality Of Master Data Across Processes

ordercredit check fulfil ship invoice paymentpackage

Process Example - Manufacturing Order to cash

schedule

Order entry

system

Finance credit control system

Production planning & scheduling

system

CAMsystem

Inventory system

Distribution system

Billing Gen Ledger

Orders data Customer data Product data

Data steward(Customer data)

Data steward(Customer data)

Data steward(Customer data)

39

To Implement MDM A Methodology Needs To Be Applied To EACH Master Data Entity To Bring It Under Control

SBV

Model

Discover

Map

ProfileClean

Integrate

Provision

Monitor

IDE

NTI

FY

ASSESS

SER

VE

INTEGRATE

DEFINE

CONTROL

CUSTOMER Data

Governance

SBV

Model

Discover

Map

ProfileClean

Integrate

Provision

Monitor

IDE

NTI

FY

ASSESSSE

RVE

INTEGRATE

DEFINE

CONTROL

PRODUCT Data

GovernanceSBV

Model

Discover

Map

ProfileClean

Integrate

Provision

Monitor

IDE

NTI

FY

ASSESS

SER

VE

INTEGRATE

DEFINE

CONTROL

SUPPLIERData

Governance

Customer Data Governance Product Data Governance

Supplier Data Governance

Also ASSETEMPLOYEEACCOUNTMATERIAL…..

40

A Shared Business Vocabulary Is The Anchor Point For Any MDM Project

IDE

NTI

FY

ASSESS

SER

VE

INTEGRATE

DEFINE

CONTROLSBV

Model

Discover

Map

ProfileClean

Integrate

Provision

Monitor

Define all Common

Master Data,attributes

41

Discover And Map Disparate Master Data in Different Systems To Standard, SBV Defined Master Data Entities

Standard SBV Model for Customer Master Data

Sales Force Automation Schema

Disparate Customer data

SFA System Customer Mapping

Branch System Customer Mapping

Billing System Customer Mapping

Branch System SchemaDisparate Customer

data

Billing System SchemaDisparate Customer

data

Data Discovery

42

To Get A Single View Of Master Data You Need To Have Global IDs, A Master SBV, AND Know All Data Mappings

ID = Party_ID Party_FirstNamePary_SurnameParty_StreetNo

Party_StreetNameParty_City

…….

C_NameC_Address

C_City

Client_NameClient_Addr

CardHolder_ForenameCardholder_SurnameCardholder_Address

Acc_NameAcc_Addr1Acc_Addr2

Acc_BirthdateCustomerMaster Data

ID = Loan Number

ID = Mortgage Number

ID = Account Number

ID = CCard Number

Loan system

Mortgage system

Savings system

Card system

43

Use People, Processes, Policies And Technology To Implement MDM

SBV

Model

Discover

Map

ProfileClean

Integrate

Provision

Monitor

IDE

NTI

FYASSESS

SER

VE

INTEGRATE

DEFINE

CONTROL

CUSTOMER Data

Governance

Data & Metadata

Relationship Discovery

Tools

Data Quality

Profiling & Monitoring

Tools

Data Modelling &

Data Integrity Tool

DataCleansing &

MatchingTools

Data Integration

Tools

metadata

Enterprise Data Management Platform – All tools share a common repository

BusinessGlossary

Tool

Data Governance Console

C

R

Uprod cust

asset

D

44

Implement Shared Master Data Services To Provide A Common Approach To Master Data Maintenance

Master data access and maintenance services• A common set of shared services to access

and maintain master data i.e. for use as a data entry system (DES)

• Allows an MDM system to be integrated with applications, processes and portals to ‘consume’ master data

• May have support for other APIs in addition to web service interface e.g. Java and .Net

Integration with Enterprise Service Bus / Message Broker software to manage synchronisation of changes to master data to all operational and analytical systems that use complete sets or subsets of this data

C

R

U

D

prod custasset

45

www.intelligentbusiness.bizmferguson@intelligentbusiness.biz

Twitter: @mikeferguson1Tel/Fax (+44)1625 520700

Thank You!

Worst Practices in MDM

Jake FreivaldVice PresidentInformation Builders

46

Founded: 1975 in NYCEmployees: 1,600Direct customers: 9,000+OEMs: 10,000+End users: Tens of millions

Goal: Deliver the industry’s best-engineered software and top customer service to ensure customer success.

Corporate SummaryInformation Builders

50+ locations

Software Data integration (iWay) Data integrity (iWay) BI and analytics (WebFOCUS)

Customer Support Global 24x7, local and online

Consulting services Expertise, mentoring, and rapid

application developmentEducation

Our facilities, your facilities, or onlineCustomer communications

Newsletters, IB Magazine, and onlineUser community

Summit conference, local groups, advisory councils, FocalPoint social network , Facebook, Twitter, etc.

Information BuildersAward-Winning Customer Service

48

Customer

User Community &Social Network

CustomerSupport

Consulting Education

Documentation Premium

Wisdom of CrowdsTM

Mastering Master Data Management

49

Understanding Processes and Their ValueInformation Management

50

MultipurposeReal-timeLifeWatch

MultisystemDQ firewallTRAC

MultidomainSingle, 360° viewLA CAFE

Project Components Integration Quality and standardization Mastering Generation of consumption

artifacts Historical data management Testing Issue remediation

The “Real Work” Behind MDM

Project Characteristics Transformational at the

business level Technically complex Dependent on solid

architecture

Knowledge requirements Staffing requirements Integration expertise Mastering expertise Business process transformation experience Testing and quality assurance

The “Real Work” Behind MDM

MDM App(End State)

• MDM hub• Data warehouse• Partner interface

• Quality process

• Operational systems• BI/analytics app

Agile Approach to MDM and Data IntegrationTypical Data Integration, Quality, & Mastering Approach

• MDM hub• Data warehouse• Partner interface

• Quality process

• Operational systems• BI/analytics app

MDM App(End State)

Omni-Gen

Drive from the business (#1) • Deliver mapping from many systems of record (#3) • Flexible system enables use of only one MDM repository (#4) • Eliminate one-and-done mentality with business-user remediation (#5) • Facilitate rapid change, with master data changes in inbound MDM data integration (#6) and changes to outbound MD synchronization considered (#9) • Include a DQ firewall as part of MDM (#8) • Use common processes and reuse rules to maintain consistency (#10) .

Traditional MDM Development ProcessWithout Omni-Gen

55

Data

Mgm

t Te

amAn

alyti

cs

Team

Busin

ess T

eam

Define Golden Record

Test

Dat

a Re

adin

ess

Go-li

ve

Define Analytics

Define Cleansing Rules

Define Master Rules

Attach History

Create API & Feeds

CreateGovernance Portal

Define Master Data Store

Write Mastering Rules

Write Cleansing Rules

Analytics Initial Implementation

Define Mapping Rules

Rapid, Automated, and Parallel Development ProcessWith Omni-Gen

56

Data

Mgm

t Te

am Write Cleansing Rules

Define Mapping Rules

Write Mastering Rules

Auto

mati

c

Attach History

Create API & Feeds

Generate Interface Doc Specification (IDS)

Anal

ytics

Te

am Analytics Initial Implementation

Busin

ess T

eam

Define Golden Record

Test

Dat

a Re

adin

ess a

nd G

o-liv

e

Shorter, automated, and

rule-driven lifecycle

means more iterative

development and

compressed cycle times

Define Governance Rules

Define Cleansing Rules

Define Mastering Rules

Define Analytics

Omni-Gen

Profiling Integration Cleansing Mastering

Integration Edition

Data Quality Edition

Master Data Management Edition

Product ViewOmni Information Management Platform

Omni-Gen

Profiling Integration Cleansing Mastering

Integration Edition

Data Quality Edition

Master Data Management Edition

Omni-Patient Omni-Payer Other and Custom Omni Applications

Product ViewOmni Information Management Platform

The DW is a client of the MDM system (#2) • Synchronization managed through outbound facilities of the MDM system (#7)

Onboarding Data for Improved Collaboration and OutcomesOmni-Payer Benefits

PrivatePractice

HEDIS / CMS

Hospitals

Informaticsand Analytics

ProviderRelations

CommunityCare

Thank you!