wp mdm-tech-overview

Copyright © Kalido 2012 1

Managing Master Data with Kalido® Master Data Management™A Technical Overview


Executive Summary

Master data management (MDM) is a discipline that continues to gain momentum in organizations globally. This is due to the need for accurate data used in both operational and analytical operations of the business. Use cases where MDM is critical include data harmonization for analytics, data harmonization for operations, hierarchy management for business intelligence (BI), operational data governance, systems migration and consolidation, and enterprise-wide MDM.

To optimize and gain insight into corporate performance, companies make major investments in operational systems to manage their business processes, and in analytical systems to analyze business transactions in aggregate. The context for these processes and transactions—master data—includes information about products, customers, suppliers, regions, employees, charts of accounts, key performance indicators, brands and more. Organizations could literally have hundreds of different types, or domains, of master data used to run and manage the business.

Master data is often duplicated and managed in multiple systems. Poor master data used in operational business processes can lead to unfavorable business operating results and performance. Poor master data in Business Intelligence results in inaccurate views of corporate performance leading to millions of dollars in lost sales and cost savings opportunities annually. These problems are most acute in enterprises which regularly undergo mergers and acquisitions; which introduce, retire, buy and sell product lines; that open and close locations; and that operate under different and changing regulatory environments across corporate divisions or geographic regions. This can make it nearly impossible to gain consolidated views of enterprise performance.

The issue of managing master data is getting greater visibility due to pressure to increase corporate transparency, data governance, accountability, and performance on an enterprise-wide basis.

This white paper describes implementing a master data management solution using Kalido® Master Data Management™, an MDM application which is available standalone and as part of the Kalido Information Engine. Kalido MDM is a workflow-driven, web-based application that enables business people to collaboratively define, consolidate, authorize, publish, govern and maintain master data. Once you have developed a repository of clean and accurate master data, you can use that data to make business processes more consistent, and analytics more accurate, and make it available to business users across your organization so they have access to the clean information they need to do their jobs.


Master Data Management Overview

Master data (or reference data) represents a company’s business vocabulary—the business entities, terminology, definitions and classifications used to describe business information. Master data provides the context for recording transaction data—facts about events in the business.

Consider the following transaction:

Sales rep Laurie Murphy sold 100 8 foot cherry dining tables to Rochester Furniture on May 29 for $45,000 at a profit of $15,000.

Data collected from this transaction description has many different uses:

Here’s how this translates into a requirements breakdown by a business analyst:

The context for this transaction—the master data—includes who made the sale, what was sold, when it was sold and to whom. For accuracy in the business process of collecting this order, it is important to get the information about the product correct for inventory management reasons and accurate address information for shipping reasons, etc. And, if you lack an accurate list of sales reps, products, time periods and customers it is impossible to accurately analyze sales performance and profitability by customer, by product and by sales territory.

This is a hierarchical representation as seen by an architect:


Further, there are a number of other considerations related to the master data: what system – or systems – is this data stored in? What business rule determines “profit”? How is this data used in other downstream systems such as data warehouses, data marts and business intelligence tools?

Master data inconsistencies can cause other problems:

• Operational silos where divisions within an enterprise have different nomenclature, definitions and visibility into the business This results in misunderstandings, operational inefficiencies, weaker customer service, channel conflict and other margin- and agility-lowering effects

• Inconsistent customer, supplier and product IDs across operational systems make it impossible to get a “single view” of customer and supplier relations or product performance on a global basis

• 50% of enterprises surveyed maintain master data separately in 11 or more source systems

• 50% of enterprises surveyed consider master data management to be a high or top priority

• Over 80% of enterprises surveyed plan on centralizing master data management

As a result, your organization could lose millions annually due to lost cross-sell and upsell opportunities to a single customer, or repeatedly manufacturing and selling unprofitable products. The real benefit of master data management is better data to improve business outcomes.


Master Data Management Challenges

Managing enterprise master data is challenging. Consider the following scenario:

A global producer of consumer food products wants to analyze profitability trends over the last two years by product, channel, customer, brand and region.

Challenge #1: Master data required for this analysis (products, customers) is duplicated in multiple operational systems (CRM, ERP, etc.) that run independently in each region. It must be consolidated and inconsistencies must be eliminated.

Challenge #2: Some of the master data is outside the scope of operational systems—lists of brands and channels are supplied by business users, or necessary information comes from third party external systems not under your control. Extending operational systems to support this data enterprise-wide can be prohibitively expensive or simply not possible due to the inherent restrictions in how some operational systems are constructed or in how the external data is delivered.

Challenge #3: Definitions for some master data exist in programming logic (such as spreadsheet functions and data transformation routines) and are therefore inaccessible to users and systems—this needs to be available and controlled by business people.

Challenge #4: The accuracy of this company’s profit analysis depends on associations between customers and channels, channels and regions, products and brands, etc. being correct. Master data is inter-related and hierarchical; referential integrity must be validated.

Challenge #5: Business users must be integrated into the master data management process—they are best equipped to reconcile data inconsistencies, contribute missing data, ensure the validity of the data, authorize its use and maintain its validity throughout its lifecycle.

Challenge #6: Analyzing historic trends is difficult—customers merge, products are discontinued, etc., but the operational systems from which master data is sourced reflect the present. Historic master data views must be maintained to support trend analysis and audits.

Challenge #7: Different “views” of master data must co-exist to support different business functions. A solution must capture and map between the different master data views.

Customer Success Story A consumer packaged goods company is managing hundreds of thousands of customer, supplier, financial, product and other master data items in Kalido MDM. Business data experts at the head office author and define central views of the data (including business definitions), and coworkers in other countries map their domestic views of the data into Kalido MDM to facilitate BI reporting and information exchange across divisions. The information is accessible to employees via a corporate intranet to help everyone gain greater visibility into the business.


Challenge #8: Once master data is prepared, it must be transferable, in whole or in part, to support the organization’s necessary business processes. These processes can include facilitating business analyses via a data warehouse and in transaction systems for greater operational consistency across all facets of the business: supply chain, financial, human resources, customer relationship management and others.

Challenge #9: Master data is as important to people as to systems—it should be accessible throughout the enterprise to increase corporate transparency and help everyone understand far-reaching customer relationship implications and market presence of brands, etc. for efficient operational execution and reporting.

Customer Success Story An international brewing company increased business transparency and operational efficiency across its brand companies by publishing 3,000 business policy and metric definitions, along with customer, product and market master data to employees via Kalido MDM.

Based on common scenarios and challenges like those above, Global 2000 companies are discovering that an ideal master data management solution must deliver the following capabilities:

• Manage all types/domains of master data

• Ensure relationships between master data domains can be created and managed

• Provide environment for central authoring and managing custom master data domains and entities

• Manage master data through its life stages and all validation processes

• Match and harmonize incoming master data from multiple sources

• Model and maintain simple and complex hierarchies for reporting and analysis

• Provide a simple interface for business users to use clean master data

• Automate and drive the data stewardship process when people need to make decisions

• Handle changes in structure with changing requirements, keeping an historical record

• Provide a secure environment with an audit trail for the full lifecycle of data

Kalido MDM meets these requirements, enabling companies to create a central master data repository where business and technical users collaboratively control, enrich and authorize master data to ensure consistency and semantic validity.


Kalido® MDM Key Capabilities

Every MDM Domain Support A large enterprise typically has hundreds of master data categories and Kalido MDM can manage all of them—from products, customers, brands, markets and territories to suppliers, financial data, key performance indicators and more. Therefore, a key objective which customers have come to expect when evaluating MDM tools is to manage all types/domains of master data. Many competing MDM solutions excel at managing a single domain, such as customer or product, but struggle to handle other domains. This is often due to the hard-coded nature of those tools’ data model. Kalido’s generic data model means there is no inherent bias to overcome when supporting the many different master data types needed to drive better information in business processes.

Benefits: • A flexible model allows a business to grow without code or software boundaries preventing change, while maintaining a full audit trail of events and history • A single tool can be used for all master data types • Consistency as domains are added • Easier for IT to support and maintain, instead of different tools used for each different domain

What is Kalido MDM?Kalido MDM is an agile software application for harmonizing, storing, managing and governing master data over time. It increases the consistency and accuracy of business processes and corporate performance reporting by enabling business people to collaboratively manage, author and control master data in a workflow-driven environment. It produces a master data repository from which “golden copy” master data can be distributed to enterprise applications and business people throughout the organization.

Kalido MDM offers a number of key capabilities which include:


Sophisticated Modeling As master data domains are added to an MDM repository, the number and complexity of relationships between the data grow exponentially. A key objective is to ensure relationships between master data domains can be created and managed. Kalido offers a visual master data modeling capability that supports complex modeling concepts such as supertyping and subtyping which makes it possible to support inheritance of attributes of a supertype among its subtypes. These MDM models can be created directly in the Kalido Business Information Modeler, or can be imported into the Modeler from existing models and taxonomies. Kalido’s business information model allows IT and business users to define and manage the business rules and validation rules to ensure that master data is fit for use in the business processes it supports. Additionally, the model is easily accessible, and understandable, by both IT and non-technical business users.

Benefits: • Easily handles the many complex data relationships and entities • Common understanding between IT and line of business end users

Master Data Authoring Unfortunately, not all data that is required for some business processes can be authored in an operational system. Additionally, it is often desirable to provide a central place to author or propose new master data – new customer records, new products, etc. – and distribute it to operational systems, especially in a heterogeneous systems landscape with different fields for each piece of master data used across systems. The objective of data authoring is to provide an environment for managing custom master data domains and entities. Kalido offers a model-driven, automatically generated user interface for authoring data. Instead of custom screen coding and development, Kalido MDM populates screens based on the context of the task, and validation of authored data is driven by the model, so you can be sure that the new master data elements match the requirements.

Benefits: • Enhanced data integrity when creating new master data records with data matching to ensure quality • Increased flexibility and fully exposed API • Support for user-specific requirements


Life Cycle Management A piece of master data goes through many steps over time. The concept of life cycle management refers to managing master data through its life stages and all validation processes. Kalido MDM provides a single environment to author, load, match, validate, publish, change and archive master data. To accomplish this, and still keep the business processes running, Kalido separates master data into “work in progress” (WIP) and “published” contexts. If a piece of master data needs to change, that change is handled as WIP and only makes it to published status once the change has been approved. Kalido MDM supports incremental update based on events for change data capture downstream data consumption. Kalido also keeps a full audit trail for all changes, and steps through the process, and manages all the human touch-point interaction through the various steps.

Benefits: • Single environment facilitates process, reduces tool integration needs • Business processes are insulated from disruption during change management • Separation of roles and responsibilities, ensuring accountability

Data Matching It would be ideal if master data was the same across all your systems that run your business processes, but unfortunately it is not. Master data comes from multiple sources, some of which may not be internal to your organization. Therefore a critical requirement is to match and harmonize incoming master data from multiple data sources. Kalido meets this need by allowing customers to define, test, save and execute complex data matching and mapping rules without writing any code. This capability allows you to match different data values across multiple sources to the central master data repository. The matching engine in Kalido MDM accurately finds matches between attributes, even for incomplete or partial similarity. Kalido offers an intuitive user interface for creating and previewing match results so stewards can confirm in real time if the match rules are appropriate. Combined with Kalido MDM’s workflow approval with human and system decision thresholds, true matches are automated and manual review is only required for potential matches. Additionally, Kalido can match any type of data, not just name & address (which is often the case in other matching engines) and offers error-tolerant lookup.

Benefits: • Easily manage increasing master data volumes and complexity • Vastly reduced manual effort saves costs


Hierarchy Management Managing hierarchies is a classic MDM use case. Typically hierarchies are managed in a data warehouse as operational systems don’t accommodate hierarchies, but business users can also require many different views. The objective of this capability is to model and maintain the range of simple to very complex hierarchies for reporting and analysis, which will vary across an organization. Kalido MDM supports these requirements, allowing local to global hierarchy mapping to enable departmental and regional views and allowing the business to manage standard hierarchies as well as very complex multi-level or “ragged” hierarchies. Hierarchies are easily and visually designed using the Kalido Business Information Modeler or imported from existing taxonomies which can then be altered and enriched to meet current and changing requirements.

Benefits: • Easily accommodates the most complex aggregations and hierarchies • Allows lines-of-business and subsidiaries to manage and analyze the business to local needs

MDM Consumer Interface One of the important facets of master data management is engaging with your subject matter experts who can help make and maintain clean master data – those who have the context for how master data is used, and who in turn need access to it to do their jobs. The objective of the MDM consumer interface is to provide a simple interface for business users to use clean master data, and also play a role in its stewardship. The Kalido MDM consumer interface provides a web-based capability to browse enterprise master data using the business information model, intuitive hierarchy browsing, a web search result display for finding customer records, product records, et al, and an ability to raise a change request when they see something that needs to be updated.

Benefits: • Gives business users direct access to the most accurate information they need to do their jobs • Better engagement with MDM consumers in the stewardship process

Workflow Workflow in MDM is perhaps the most important capability in ensuring a smooth data stewardship process. Some MDM tools don’t support workflow, relying instead on the customer to integrate the MDM tool with an existing workflow engine. In addition to requiring on-site integration by the customer, this adds unnecessary expense and adds a new point of failure to an MDM implementation. Kalido MDM offers embedded workflow to automate and drive and guide the data stewardship process when people need to make decisions. Its flexible, robust built-in workflow routes master data validation actions to specific users or groups for human resolution and approval. It is event-driven and allows you to build flexible and comprehensive state transition steps. While external workflow integration is not required, sometimes it is desirable to include external processing before moving to the next step in a workflow and integration into operation systems for enrichment. Kalido supports user exits via an API to external code modules to support these complex workflows. Taken together, the capabilities of Kalido MDM easily manage increasingly complex governance and stewardship processes for delivering accurate and consistent master data to your business processes.

Benefits: • Automated data stewardship process saves time and effort • Focuses data stewards on only resolving exceptions


Worklists Worklists in Kalido MDM are used in combination with workflow when you want to process a set of master data subjects as a group. The subjects are validated, approved, authorized and published as a unit. This is used extensively to process multiple subjects which gather information from various sources and must be complete before publication. The worklist feature allows multiple subjects to move through an enrichment approval process before becoming master data, instead of having to be handled and processed individually.

Benefits: • Working on multiple subjects as a group reduces processing time • Fewer workflow tasks required for handling like processes

Change Management As your business and business processes evolve, so will your master data. A key requirement is therefore to handle changes in master data structure with changing requirements, keeping an historical record for cases where you need historical views. Because Kalido MDM is model-driven, it can easily change design definition as your business evolves. It supports time-variant capture and management of master data values by category, and supports full logging of changes to not only the model, but also to actual master data values.

Benefits: • Master data is always up to date with how the business is run and managed • Reduced effort and cost to maintain timely data • Easily audit historical states for more informed changes

Security & Audit Security and audit are of course key requirements of any MDM implementation. For many use cases, a secure environment with an audit trail is a critical capability, particularly for any regulatory or statutory compliance use. Kalido supports role-based security, and the role can be assigned at any level in the data. Users can be managed in LDAP and Active Directory. For data security, Kalido supports object and data-level security using ACLs. For data values, Kalido keeps track of who changed the model and the data, and when it was changed, as well as history tracking of prior master data values. With this capability, customers have a “corporate memory” of master data as it was at any point in time.

Benefits: • Bullet-proof audit trail for compliance tracking • Recreate prior model and master data views for easy historical comparison analysis.


Model-Driven API Toolkit One of the challenges – and goals – of an MDM program to not only get the data clean, but also put in place processes to enable the data to stay clean. This means bringing access to MDM functions to the business applications and business application users. Kalido offers a model-driven toolkit that allows delivery of direct integration with existing business applications, including an ability to seamlessly call Kalido MDM functions from them, and to generate custom interfaces to facilitate interaction with published master data. With “MDM applets” like this, clients can deploy a mobile application so field employees can easily look up master data when performing various business tasks, which may include proposing changes to master data as well as new master data elements such as new customer records. When new records are proposed, all the validation rules and governance processes are automatically enforced and tracked. Kalido MDM can now more easily hook into operational systems to deliver a sustained, stay-clean environment. Additionally, this capability expands the participation in the master data governance process by allowing for audience-specific interfaces that can appear not only in browsers but also mobile devices such as iPads.

Benefits: • Expands master data governance participation across the enterprise • Direct integration into business applications

Customer Success Story A financial services company governs changes to financial master data such as chart of account codes to ensure accurate statutory reporting and to increase visibility into data lineage. Changes to financial master data are made by users within Kalido MDM (rather than within their various financial systems), and via workflow, the information is passed to others for review and is used to drive automatic updates in financial applications. Central governance gets fast, accurate views of financial performance with less manpower spent on consolidated reporting and without the cost and delay of standardizing accounting nomenclature across the company.


Kalido MDM ArchitectureThe Kalido MDM architecture consists of three layers:

• the Data layer, where the data sources reside

• the Application layer, where the Kalido MDM application is installed and where tasks are run

• the Presentation layer, which includes the Web Server, J2EE container and the browser

Kalido MDM ApplicationKalido MDM is comprised of three modules:

• MDM Consumer—a Web application designed for all users who need reliable, up-to-date business information to execute their tasks and projects. It provides read-only access to all published master data. Consumers can easily raise change requests against the data and monitor the progress of the changes.

• Kalido MDM—the main Web application where models are defined and deployed and all in-progress master data is stored. Two Contexts facilitate the rapid flow of large volumes of data from source systems through publication to the wide consumer audience. The Contexts represent data that has been published and is visible in the MDM Consumer (Master Context) and data that requires more work (Working Context).

• Data Matching Module—a fully integrated module that allows data managers to define, test, save and execute complex data matching and mapping rules. It complements MDM’s workflow‐driven data cleansing system with powerful lookup technology that allows data stewards to overcome content errors and data inconsistencies that impair data quality and slow down typical deployments


Kalido MDM UsersKalido MDM has three types of user:

• Administrators—Often IT workers, they define master data models, security, workflows, worklists, templates and the user interface.

• Providers—Business users who collaboratively control, edit, authorize and publish master data within the context of models, workflows and worklists created by an Administrator.

• Consumers—Business users who browse, search, and raise issues or change requests on published master data to gain greater visibility into and understanding of the business.

Master Data ModelingMaster data models define master data structure, definitions and relationships. Models can be highly conceptual, and communicate rich business meaning (which is why master data can be readily browsed by Consumers)

• Data can be rapidly loaded into the model from disparate IT systems, even if the incoming data does not rigidly conform to the model. As the data becomes conformant, it can be incrementally published to the Master Context where it is immediately visible to all consumers.

• Data that does conform to the model is loaded and published directly into the Master Context

• Models can be changed without reloading or reformatting published or in-progress master data already stored in the master data repository. Users are notified of the number of subjects that will become invalid if the modified model is deployed. Upon deployment, invalid subjects are automatically brought back into the Working Context to be enhanced until they are ready for re-publication.

Models can be created within Kalido MDM or they can be generated based on models defined within an existing Kalido Information Engine structure, which is in turn generated from a business model defined with Kalido Business Information Modeler. The data model can be changed at any time. You can also define advanced data models directly via XML and the Kalido MDM Services Application Programming Interface (API), which allows you to customize and extend the Kalido MDM solution using .Net, JAVA, and Web Services.


Integrating Kalido MDM with a data warehouseThe Kalido Information Engine includes a capability to build and operate enterprise data warehouses. Driven by business models developed in Kalido Business Information Modeler, it automates tasks such as data warehouse schema design and generation, data loading, query definition, data mart generation and more. Thus data warehouses meet new business needs iteratively in days or weeks, not months.

Kalido MDM can import model definitions and data created in a Kalido data warehouse, and export data back to it—typically after the master data has been authorized and published.

Kalido MDM helps assure consistent, accurate and trustworthy data is available in the Kalido Information Engine for use in business intelligence applications.

For non-Kalido warehouses, Kalido MDM can be accessed in many ways to fuel the analytical process through its API which is exposed as web services, and directly from relational tables. Depending on the business requirement for reporting and ad hoc analysis, master data in Kalido MDM can be mined and supplied to meet any analytical questions. Kalido MDM can act as a central place to manage and serve analytical requests for such things as multiple different views on hierarchical roll ups or time-based MDM data reporting.

Managing Master Data in Operational SystemsIn addition to supporting master data for downstream analytical uses, Kalido MDM can be used to fuel upstream operational systems. With its robust sophisticated modeling, it can handle multiple different business needs and systems.

Once master data is accurate and consistent, Kalido MDM can export master data in CSV, fixed format, or XML for batch update or entry into operational systems. Kalido offers a Web services-based API that can be called in real-time from or to operational systems for checking records against the “golden copy” held in the Kalido MDM repository or to trickle feed a system with new authored records of master data, also mentioned earlier in this paper.

A number of forms based tools such as MS InfoPath, Adobe LifeCycle, and BPM tools are frequently used as part of this operational master data management entry. To support this activity, Kalido MDM exposes real time matching capabilities to aid in elevating duplicate entry into the operational system using their internal Master Data as the golden standard for incoming records.


Summary

Kalido MDM is a key data management component of the Kalido Information Engine, and it uniquely enables business people to collaboratively define, consolidate, manage, authorize and publish master data in a workflow-driven, web-based environment – without needing to modify operational IT systems.

The benefits of managing master data with Kalido MDM and distributing it to enterprise data warehouses and employee information portals include:

• Highly accurate and consistent business performance reporting

• Greater visibility into the business for more efficient operations

The benefits are achieved through the unique capabilities of Kalido MDM, which include:

• Manages every type of master data – a large enterprise typically has hundreds of master data categories and Kalido MDM can manage all of them – from products and customers to brands, markets, territories and more

• Allows business users easy access to published master data so they can raise issues or change requests as necessary

• Facilitates data governance – puts business people in control of master data quality in a collaborative, workflow-driven environment

• Offers agile and flexible master data modeling via the Kalido Business Information Modeler – featuring powerful cataloging, segmenting, merging and mapping facilities

• Loads non-conformant master data – instead of rejecting invalid master data, all master data is loaded even if it doesn’t conform to the master data model. Valid data is published directly to the Master Context while other data is sent to the Working Context for further manipulation. Workflows ensure that the data, or the model, is revised accordingly

• Maintains master data history – historic views can be recreated to support audits and historical analysis as master data adapts to business changes over time

• Handles time-variance – multiple date values are allowed over time

• Allows publication of a set of data as a version – preventing further editing by users

• Exchanges master data with other systems such as data warehouses, ERP and other systems

DS-MDMTECH04232012

wp mdm-tech-overview

Documents

accurate master data

accurate data

poor master data

master data inconsistencies

reference data

data warehouses

data marts

data storedin