implementing a successful enterprise data quality initiative

www.EWSolutions.com© 2008 Enterprise Warehousing Solutions, Inc. (EWSolutions) – 1

EWSolutions

Strategic Partner & Systems IntegratorIntelligent Business Intelligencesm

Implementing a Successful Implementing a Successful Enterprise Data Quality Enterprise Data Quality

InitiativeInitiativeBy David Marco

PresidentEWSolutions



EWSolutions’ BackgroundEWSolutions is a Chicago-headquartered strategic partner and full life-cycle systems integrator providing both award winning strategic consulting and full-service implementation services. This combination affords our clients a full range of services for any size enterprise information management, managed meta data environment, and/or data warehouse/business intelligence initiative. Our notable client projects have been featured in the Chicago Tribune, Federal Computer Weekly, Crain’s Chicago Business, and won the 2004 Intelligent Enterprise’s RealWare award, 2007 Excellence in Information Integrity Award nomination and DM Review’s 2005 World Class Solutions award.

For more information on our Strategic Consulting Services, Implementation Services, or World-Class Training, call toll free at 866.EWS.1100, 866.397.1100, main number 630.920.0005 or email us at [email protected]

Best Business Intelligence ApplicationInformation Integration

Client: Department of Defense

World Class Solutions Award

Data Management

2007 Excellence in Information Integrity Award Nomination

MIT Information Quality Industry Symposium, July 16-17, 2008

292



EWSolutions’ Partial Client List

For more information on our Strategic Consulting Services, Implementation Services, or World-Class Training, call toll free at 866.EWS.1100, 866.397.1100, main number 630.920.0005 or email us at [email protected]

Arizona Supreme CourtBank of MontrealBankUnitedBasic American FoodsBecton, Dickinson and CompanyBlue Cross Blue Shield companiesBranch Banking & Trust (BB&T)British Petroleum (BP)California DMVCollege BoardCorning Cable SystemsCountrywide FinancialDefense Logistics Agency (DLA)Delta DentalDepartment of Defense (DoD)Driehaus Capital ManagementEli Lilly and CompanyFederal Aviation AdministrationFederal Bureau of Investigation (FBI)Fidelity Information Services

Ford Motor CompanyGlaxoSmithKlineHarris BankThe HartfordHarvard Pilgrim HealthCareHealth Care Services CorporationHewitt AssociatesHP (Hewlett-Packard)Information Resources Inc.International PaperJanus Mutual FundsJohnson ControlsKey BankLiquidNetLoyola Medical CenterManulife FinancialMayo ClinicMicrosoftNational City BankNationwide

Neighborhood Health PlanNORCPhysicians Mutual InsurancePillsburyQuintiles Sallie MaeSchneider NationalSecretary of Defense/LogisticsSouth Orange County Community CollegeSunTrust BankTarget CorporationThe Regence GroupThomson Multimedia (RCA)United Health GroupUnited States Air ForceUnited States NavyUnited States Transportation CommandUSAAWells FargoWisconsin Department of TransportationZurich Cantonal Bank



Professional Profile/Contact InformationMr. Marco is an internationally recognized expert in the field of enterprise information management, data warehousing, Capability Maturity Model (CMM), business intelligence, and is the world’s foremost authority on meta data management. Mr. Marco has authored several books including the widely acclaimed “Universal Meta Data Models” (Wiley, 2004) and the classic “Building and Managing the Meta Data Repository: A Full Life-Cycle Guide” (Wiley, 2000). These groundbreaking books have been broadly endorsed by many of the largest software companies in the industry and by several major magazines.

Selected to the prestigious 2004 Crain’s Chicago Business “Top 40 Under 40”Crain’s Chicago Business anointed him the “Melvil Dewey of Metadata”2008 DAMA Data Management Hall of Fame (Professional Achievement Award)Chairman of the Enterprise Information Management Institute (EIMInstitute.ORG)2007 DePaul University named him one of their “Top 14 Alumni Under 40”Presented hundreds of keynotes/seminars across four continentsPublished hundreds of articles on information technologyAuthor of several best selling information technology booksTaught at the University of Chicago and DePaul UniversityJudged dozens of various industry awards in meta data management and data

warehousing

Email: [email protected]


293



Agenda

The State of Data QualityData Quality ChallengesData Quality FoundationsData Quality SolutionsReal-World Data Quality Case Studies



The State of Data The State of Data QualityQuality


294



The State of Data Quality

Most organizations data quality can be best described as abysmalLarge bank’s Executive VP stated: “The numbers (data in their data warehouse) are the best that we have, and as long as they are calculated the same way every time, they give us a directionally correct view of the business.”

((

))

(())

(())



The State of Data Quality

When data is presented to a business analyst or executive typically the person does not know:

if the data accurate?what percentage of the data is not accurate or in doubtthe origin of the datathe meaning of the datawhere the data came from

Our information technology (IT) system’s architectures are not conducive to data quality


295



What Does One Process Look What Does One Process Look Like for a Large Company?Like for a Large Company?



Fortune 50 – One Process

STO

Sales Request

Web/Em

ail/Fax

Web/Em

ail/FaxW

eb/Email/Fax

Web/Email/FaxWeb/Email/Fax

Sales Request

Web/Em

ail/FaxW

eb/Email/Fax

Web/Em

ail/Fax

Web/Em

ail/Fax

Web/EDIWeb/EDI

Tele/Web

Tele/Web

Kiosk Order

Kiosk Order

Kiosk Order

Tele/Web

EDI Order

Kiosk Order

Web/ED

I

Web/ED

I

Sales Request

Sales Request

Sales Request

Sales Request

IPS Only

IPS Only

CTO

products

Web/Em

ail/Fax

Tele/Web

ePrime - HP

Global Biz Link(Extranets) -

NA - CPQ

CatalogProcurement -CGBX - NA -

CPQ

VendoreMarketPlace -

CPQ

Fast-Web& Ptnr

Prime -CPQ

B2B IntegSVR / CEI'sWeb Svr -

CPQBuysite

& MarketSite.com

Biz-Store- HP

Kiosk -CPQ

Kiosk -HP

SAP 3.1 - EPH -OM - CPQ

SAP - ConsumerDirect SV - OM -

HP

Commercial

SMB

Enterprises

AGSCQ 3PO

CQ DirectFactory

CQ AlliancePartners

Quanta- HP

SCI

HP 3PO

QMS-

AQS -NA -

CPSA/MSS- CPQ

FOCUS- CPQ

CDAT -

RackAssistant

Kylor (AA) -CPQ

Virtual SalesRep - HP

EDIGateway -NA - CPQ

WWOMS US - HP

APOGEE -Storage - HP

Fremont

HoustonM1/M2

Roseville

SCI -Alabama

Houston M5

Rack Builder- CPQ

Consumers / Micro

CorporateRelationships

CommercialResellers

Agents

RichmondSupplies

Completion& DC

Woodland IJCompletion& Supplies

DC

Memphis IJCompletion& Supplies

DC

SandstonLJ Comp. &

SuppliesDC

Ontario LJCompletion& Supplies

DC

Tijuana LJCompletion& Supplies

DC

CPQ PrinterSuppliers

CPQ ProjectorSupplier

OHLMemphi

s DC

Heart - HP

Sales Force

Tech Data

Ingram Micro

Vista - Fulfillment - CPQ

Vista - OM - CPQ

SAP - Replenishment D7 - OM - HP

SAP - Replenishment D7 - Fulfillment - HP

SAP - Fusion - NA- Fulfillment - HP

SAP - Fusion - NA - OM -HP

SAP 3.1 -EPH -

Fulfillment -CPQ

Retailer

HP ShoppingFinancialInstitution

MenloLogistics

Tatung

IrishExpress

Ryder VentureMSL

Genco

PowerPackaging

Compal- HP

SAP - HPS- NA -

Upfront -

Astro -NA -HP

Fireman- NA -

SCA - NA- HP

ITRC - SUM - NA- HP

SAP - HPS - NA -OM - HP

SAP - HPS - NA -OM Finance - HP

SAP - HPS - NA -Virtual Sourcing -

HP

FieldAdmin

PM Tools- HP

Parts RepairCustomers

and Partners

Tiger -NA - HP

TIM -HP

SW Depot -Shopping - HP

Legacy OM- Finance -

HP

Legacy Parts -OM - HP

Legacy Parts -SC - HP

SAP - HPFO- NA - HP

APOGEE/SHIPUS - Services -

HP

HPFO Legacy

SAP -IPC D7 -

ChannelLinksDescartes

DAC

Compucom

WWOMSCanada - HP

WW CISYS -HPSAP -

XRS -

SAP - XRS -Matl - HP

SAP -XRS - IPC

- HP

RyderMM3000 - HP

BusinessPC,

Servers,Consum

er CallCenter -

LJ CallCenter -

US

GEM

GEMS -NA -CPQ

CAFE -CPQ

CEI'sOmaha

Call Center,or Littleton

eCAT - CPQ

Watson -HP

SBW- HP

QuickQuote -

e-Store -Canada -

HP

SAP -CaLado -OM - HP

BusinessAdvantage -

CPQ

SAP - CaLado -Fulfillment - HP

FactoryOutlet -

SWDepot -

OM - HP

SW Depot -Fulfillment -

HP

SAP -ConsumerDirect SV -Fulfillment -

HP

Compucom -OM

Conrad - HPSystemT - CPQ

Earl -HP

SAP -ConsumerDirect SV -

Finance - HP

CPQFinancialServices

CyberS

HitachiSymnex

Daisy Tech

Affiliates

ACS -CPQ

LSO -HP

OLS -

Chameleon -CPQ

Microsoft

Call Centers -Rockville,Roseville,Cupertino,

Ontario

Vista -Finance -

CPQ

ProductAdvisor

Comm.Resellers

DACFulfillment

eTracking -

EDR -CPQ

Q-runFIC

Game DB -CPQ

ActiveAnswers

Cybrant- HP

CPL -Prod. &

Price

VISA -CPQ

GPSYCPL -

HP

GoldenEggs -CPQ

ICON -HP

OSS - HP

WWSNRS -CPQ

CustomerAdvantage

Direct Plus -CPQ

CustomerAdvantage

eProcurement -CPQ

CustomerAdvantageExtranets -

CPQ

NCAS- NA -CPQ

NCAS -QL -CPQ

CCSU

Eclipse -HP

PPS -CPQ

LesWeb -CPQCOPE/BET

ClearContract -

CPQ

Prophecy -

SBS -Pears - NA- CPQ

SIS -CPQ

SRS -CPQ

FOCUS(252) -CPQ

ASM -CPQ

CPN -CPQ

CSN -NA -CPQ

eFLS -CPQRMS -

CPQ

ORS -CPQ

Primus- NA -CPQ

CompaqDirect

SupportPricingTool -

CCDB - NA -HP

Aurora Mgr -NA - HP

Plato -NA - HPOM Server

- NA - HP

AUE -NA - HP

SWAT - HP

ePartsePO - NA

- HP

SAM -NA - HP

CMG - NA- HP

MAT -NA - HP

SORDS- NA -

HP

SACS -NA - HP

E-Eureka -NA - HP

Allegro -NA - HP

IOM -NA - HP

PROMIS- NA -

HP

Norm -NA -HP

SPD -NA -HP

SiebeleChannel

BPS - CPQ

PWEB- US -

Profiler- HP

Explorer4- CPQ

SFDM(M1/M2) -

CPQ

RemedyUS - HP

ResellerWeb - HP

Siebel CallCenter 2000- US - CPQ

QuotersWB - CPQ

MAXCIM- HP

RosettaNetPortal - HP

ESN -US -

Fast -CPQ

Armada- NA -

PartnerDirect -

CPQ

Backplane/Shopfloor -NA - HP

Clarify - WFM -NA - HP

Clarify -Pulse -

CPQ

Core -CPQ

Entitlement -NA - HP

EDI/XMLGateway - NA/AP

- HP

GOLD- CPQ

Power -CPQ

RCM - HP

SAP 3.1 - EPH- Finance -

CPQ

FAI - NA - HP

EIA - NA - HP

WWFTP -HP

SSIM- HP

e-Pack -

ResourceMarketplace

- NA - HP

Plus -NA -HP

SmartCube- HP

GDS -NA - HP

SAP -Education -

NA - HPCDO - NA -HP

Knight- CPQ

WWPAK- HP

eStores- CPQ

XelusPlan -NA -HP

SAP 4.6 - SWSupply Chain

- HP

WWComcat -

SMARTdb -

CPQ

eParts- HP

HPShoppingAt Home

- CPQ

Customers

Shopping/Quoting

Order Management

Supply Chain

Finance


296



Maybe this Image Clears When Maybe this Image Clears When you Look at One Application?you Look at One Application?



Islands of Data

Operational Applications

Islands of Data

End User Reporting Systems

Claims

Pharmacy

Financial Analysis

Health & MedicalServices (CQM, HSA)

Actuarial & Underwriting Analysis

Sales & Mktg /Network Development

Utilization

Membership/Enrollment &Billing

Standard

Adhoc

Standard

Adhoc

Standard

Adhoc

Standard

Adhoc

Standard

Adhoc

Standard

Adhoc

Standard

Adhoc

Standard

Adhoc

North E/B

TOPPSES-9000

IPS- Inter Practice SystemVAX(Burlington)

PSIMEDHPE-980

MGD ClaimsAS/400

Query

Pharmacare (outside vendor)AS/400

Pharmaview (outside vendor)

NED E/BACPS- Automatic ClaimsProcessing SystemPASS - Patient AppointmentScheduling SystemENCOUNTERFFS - Fee For ServiceREF - Referrals

HP-992 NEDClinical Computer

Pharmacy DBProvider Master

AMISYSMARS

•Hospital Summary•Quality Assurance Request

HP (9 series)

•Network & Medical Request•Actuarial End User Request•ClaimsProv DBDrug DB

AMRS-Advanced Medical Record SystemRIS- Radiology InformationSystemLAB/LIS- LaboratoryInformation System

VAX ClusterRX42 HPE-980

MHUM - Mental HealthUtilization Management

AST 486 serverHealthchex

Pentium PCBulletin Board System (BBS)

NED DatawarehouseHP937

HSA dept. Lan ServerNED Multiview GL and AP

PHC Pharmacy Server

PHC Actuarial Analysis Server

Analytic DatabaseQuantumDec Alpha

MAMSYS/PAPSYS

PHC JV Finance ServerPHC Network Development Server

PHC Medical Services Server

PHC Multi-view GL.

(14) Foxpro Applications

ASAP - Actuarial System Analysis Program

MS SQL Server

DB2 and SAS datasetDecision Analyzer

GL

SAS , SAS screens,Viewpoint

GMIS-Claim Check

Actuarial SAS dataset

ES-9000

MedicalGroups

HCD MGD NED PHCLegend Multiple colors indicate the system is

used by multiple divisions.


297



How Did We Get Here

Most companies did not plan their infrastructure…rather it just grew over timePreviously companies focused on lines-of-business, as opposed to an enterprise view



Why is Data Quality Growing as an Issue?

Enterprise initiatives (e.g. data warehousing, customer relationship management, supply chain, etc.) are most significantly impacted by data quality issuesCorporations are moving from a decentralized structure to a centralized structureGovernment regulations require data quality (SOX, 21 CFR Part 11, BASIL II, HIPPA, DoDAF, ISO 11179, etc.)


298



Data Quality SolutionsData Quality Solutions



Data Quality Solutions

Data Quality Falls Under the Larger Enterprise Information Management Umbrella

Meta Data Management is the Technical Enabler of Data Quality

Data Stewardship defines the Business Processes of Data Quality


299



Data Quality Data Quality &&

Enterprise Information Enterprise Information ManagementManagement



Data Quality & EIM

Many companies rush to implement data quality “silo” solutions without realizing that they need to understand the “larger picture”Data quality falls under the larger Enterprise Information Management (EIM) umbrella


300



What is EIM?

Enterprise Information Management (EIM):Enterprise Information Management (EIM): The systematic processes and governance procedures for applications, processes, data, and technology at a holistic enterprise perspectiveThe purpose of enterprise information management is to bring enterprise order, purpose, structure, efficiency, and performance to applications, processes, data, meta data and technologyEIM is not a single technology or component, but a coordinated framework of disciplines for managing data, meta data and information assets throughout the organizationData Does Not Manage Itself!!Data Does Not Manage Itself!!



Data Quality & EIM

There are three foundational elements that span each area of EIM

Meta Data ManagementData GovernanceData Management

No matter what focus area of EIM you are targeting you will need to address each of these elements


301



Data Quality & EIM

Data Management

Enterprise Information Management

Dat

a A

rchi

tect

ure

Pro

cess

Man

agem

ent

Dat

a Q

ualit

yD

ata

Qua

lity

IT P

ortfo

lio M

anag

emen

t

Mas

ter D

ata

Man

agem

ent

Info

rmat

ion

Del

iver

y

Info

rmat

ion

Sec

urity

Data Management is the foundation for all of the other EIM focus areas.



Meta Data Meta Data ManagementManagement


302



Meta Data vs. Data

Meta Data:Meta Data: Meta data contains the knowledge that a 1) field is called “Customer_Name”, is 40 characters in length, and exists in systems A, B, and C; 2) that our company has 3 systems which contain customer master data. These systems are…Data:Data: Data would be a specific instance of “Customer_Name” equaling “John Doe”Information:Information: Data that is meaningful to a business user. They understand it and they know what to do with it



EIM Fundamentals

(content) (context)


303



Integrating DQ into IT2007 Monthly Global Sales Report2007 Monthly Global Sales Report April 7, 2008April 7, 2008

MonthMonth Sales $ (in thousands)Sales $ (in thousands)U.SU.S

Sales $ (in thousandsSales $ (in thousandsInternational International

Product Category Product Category Sales $ (in thousandsSales $ (in thousandsTotalTotal

December TV 22,101VCR

12,190

Digital 4,002

1,209Miscellaneous

Cellular Phone11,190

10,200

7,193

1,301

870

4,30032,301

19,383

5,303

2,079

15,490

November TV 42,000

VCR

28,193

Digital 8,901

2,730Miscellaneous

Cellular Phone

21,190

22,200

12,193

2,901

1,530

9,878

64,200

40,386

11,802

4,260

31,068

October TV 70,100VCR

41,700Digital 20,000

4,850Miscellaneous


32,950

17,5504,1002,850

14,878103,050

59,25024,1007,700

46,778

“Sales $ U.S.” is comprised of aggregated sales revenues from the United States, Canada, and Mexico, but does not subtract sales dollars from returned orders

Meta Data



Integrating DQ into IT2007 Monthly Global Sales Report2007 Monthly Global Sales Report April 7, 2008April 7, 2008

MonthMonth Sales $ (in thousands)Sales $ (in thousands)U.SU.S

Sales $ (in thousandsSales $ (in thousandsInternational International

Product Category Product Category Sales $ (in thousandsSales $ (in thousandsTotalTotal

December TV 22,101VCR

12,190

Digital 4,002

1,209Miscellaneous


10,200

7,193

1,301

870

4,30032,301

19,383

5,303

2,079

15,490

November TV 42,000

VCR

28,193

Digital 8,901

2,730Miscellaneous

Cellular Phone

21,190

22,200

12,193

2,901

1,530

9,878

64,200

40,386

11,802

4,260

31,068

October TV 70,100VCR

41,700Digital 20,000

4,850Miscellaneous


32,950

17,5504,1002,850

14,878103,050

59,25024,1007,700

46,778

Data Quality Tracking Statistics8.4% of the dollar values were not loaded1.7% of the records were not loaded

Meta Data


304



Does Understanding Does Understanding Your Data Lead To Your Data Lead To Better Decisions?Better Decisions?



NASA Example – Mars Orbiter

December 11, 1998 the Mars Climate Orbiter was launchedEngineers calculated rocket firing using feet-per-second, the orbiter was programmed in meters-per-second (metric system) of thrustThe difference was 4.4 feet per second“Each time there was a burn (rocket firing) the error built up,” said Art Stephenson, Director of the Marshall Spaceflight Center and Head of the NASA Investigation Team“We entered the Mars atmosphere at a much lower altitude (than planned),” said Ed Weiler, NASA's chief scientist. “It (the spacecraft) either burned up in the Martian atmosphere or sped out (into space). We're not sure which happened.”The cost of this mission was $250 - $300 million

Source: Associated Press, Paul Recer, http://www.anomalous-images.com/news/news537.html


305



Integrating DQ into ITMeta Data for the Business January 20, 2008Campaign Analysis Query FirstRateMotors

CampaignDates

CampaignDescription

CampaignName

Auto Type

The “Buy Now, Pay Later” campaign was a nation-wide campaign. This campaign offered customers the option not to begin payments for a new car purchase until January 1, 2004. This offer is only applicable for those customers with better than standard credit ratings.

Buy Now, Pay Later ECON – Smaller, cheaper, economy car line

AutoSales

7,045

SUV – Sport Utility Vehicles

9,500 MID – Mid-size, sedan automobiles

22,010

01/01/2003 -03/31/2003

SPTS – Sports/performance cars

14,700

The “Buy One, Get A Free Scooter” campaign was a nation-wide campaign. This campaign offered customers a free scooter for the purchase of a new car. Dealer can only discount cars, up to $1,000 above customer shown invoice.

Buy One, Get A Free Scooter

ECON – Smaller, cheaper, economy car line5,205



17,888

04/01/2003 -06/30/2003


10,900

The “$1,000 Minimum Trade-In” campaign was a nation-wide campaign. This campaign offered customers a $1,000 minimum trade-in, regardless of the condition of their trade-in vehicle, only for those customers purchasing a new car.

$1,000 Minimum Trade-In




19,750

07/01/2003 -09/30/2003


12,400

The “0 down and 0% Interest” campaign was a nation-wide campaign. This campaign offered customers standard car discounts, with 0 down and 0% interest for only those customers with standard or better credit ratings.

0 down and 0% Interest




20,820

10/01/2003 -12/31/2003


13,220

53,255Grand Total

41,243Grand Total

46,582Grand Total

49,665Grand Total

Data Quality Statistics2.3% of the records were not loaded in the data warehouse batch runs.Skew percentage equals 2.3%



Data Quality and Meta Data Quality and Meta Data ManagementData Management


306



Data Quality and Meta Data Management

Good data quality professionals understand the that meta data management is their key key technical enablertechnical enablerWhere do business rules for data quality go?

a spreadsheet?a document?maybe we can just commit them to memory?in a Managed Meta Data Environment (MME)

What is a data quality rule? Its meta data!Let’s look at some MME implementations that have strong focuses on data quality



Data StewardshipData Stewardship


307



Data Stewardship

Data Stewardship: The process of having data stewards work with the data and meta data of an organization to ensure its quality, accuracy, formats, domain values, and that it is properly defined and understood across the enterpriseData Steward: A person(s) responsible for working with the data and meta data. There are different types of data stewards The data steward acts as the conduit between IT and the business. The data steward (often not just one person, but a collection of people) aligns the IT systems (both decision support and operational) with the business’requirements. The data steward has the challenge of guaranteeing that one of the corporation's most critical assets--its data--is used to its fullest capacity



Data Stewardship

Data is one of the most important assets in a corporationData has value when it is delivered timely, properly formatted, concise, accurate and understoodunderstoodBusiness ownership of the data and active participation are criticalThe role of the data steward has grown considerably over the years


308



Data Governance Organization

Every organization forms their data governance organization a little differentlySome have a more or less complex organizationWhat is critical is that the organization:

is actively using the MMEhas clear lines of communicationhas a defined and well understood decision making processwell defined feedback loop



Subject AreaGroups

Data Governance Organization

Subject AreaUser

Group #1

• Business Steward(s)• Technical Steward(s)

• Interested Parties

Rec

omm

enda

tions

DataStewardship

Group

ChiefSteward

EIM Focus Area #1

Steward Team

Data GovernanceExecutive Committee

Policies, Procedures,Standards, etc.

Members:•Executive Sponsor•Chief Steward•Business Mgrs•CIO•Key business staff•Key IT staff

Managed Meta Data Environment

Enterprise Oversight

Subject AreaUser

Group #2



Subject AreaUser

Group #3



EIM Focus Area #2

Steward Team

EIM Focus Area #3

Steward Team


309



RealReal--World MME World MME ImplementationsImplementations



AllstateAllstateCorporationCorporation

The following slides are derived from the book “Universal Meta Data Models”, David Marco & Michael Jennings, Wiley 2004


310



Allstate Corporation

The nation’s largest publicly held personal lines insurerProvides insurance products to more than 16 million households and has approximately 12,300 exclusive agents and financial specialists in the United States and CanadaMulti-channel organization: customers can access products and services through Allstate agents, or in select states at allstate.com and 1-800-Allstate®Allstate Financial Group includes the businesses that provide life and supplemental insurance, retirement, banking, and investment products through distribution channels that include Allstate agents, independent agents, and banks, and securities firmsThe MME was part of a larger Data Asset Management effortThe MME was part of a larger Data Asset Management effort




Challenges/OpportunityEarly 1990s Allstate, like most large corporations, found itself challenged with managing disparate systems to satisfy its IT needsNeeded applications to talk across platformsSystems that have different coding schemes for common codes and mismatches in field types and sizes cannot interchange data easilyDramatically reduce the number of point-to-point interfacesNeeded a precise understanding and knowledge of the data that the analysts of the data warehouse would utilize


311




Challenges/OpportunityNeed to Manage Code Data

Very “code-driven” environment because every state has its own regulatory statutes to which Allstate must adhereHighly time-consuming and difficult to integrate systems because projects would take longer to deliver and be too costlySound meta data management techniques was expected to reduce or eliminate IT rework, speed up projects, and lower their overall costs

Enable Data Warehousing ApplicationsData warehousing made data quality more important than everPreviously subject matter experts in an application area had to know what the data represented, what it actually meant, and how to use itNow this data was going to be presented to actual end users, andAllstate cannot afford to have user “interpretations” of what the data maymean be used to make “live” business decisions.A strong, centralized data management environment would be the basis for consistent data driving high-quality decisions by end users




MME OverviewMME OverviewCustom built MMEInitially focused on the management of the codes that permeated their systemsA portion of the MME (Codes Management System (CMS)) allows for the identification of enumerated domains (those for which a set list of values can be listed) and define the various coding schemes that were found in the different applications and the associated business valuesCMS allowed Allstate’s Codes Analysts group to do their job more effectivelyCodes analysts, along with a group of data administrators, then became the nucleus of the Enterprise Data Management groupKept an enterprise perspective by documenting each unique domainthey encountered and storing it in the MMEAs they worked with subsequent projects, they were able to see where the same data had been encoded differently between applications


312




Data Warehouse/Marts Metadata

Allstate Metadata Management

ERwin

DBMSCatalog

CMS

Mapping

Sanctioning

ENT XMLMessages

BusinessRules REPOSITORY

ADMINISTRATION

METADATA VIEWERS

CMS, DA, Business Rules, EDW, NOVA Data

BRIO

Applications

ApplicationsWEB

SER

VIC

ES

STAT

IC D

ATA

SER

VIC

E LA

YER

UniversalCode Table

MESSAGEBROKERXML

ENTERPRISEMESSAGES

Logical Data ModelsPhysical Structures

Enumerated DomainsSanction DocumentsEnterprise Messages

Business RulesPhysical References

& Mappings

METADATAREPOSITORY

ApplicationCode Tables

Allstate MME EnvironmentAllstate MME Environment




Data StewardshipData StewardshipJanuary 1997 formed a Data Stewardship Council (DSC)DSC is a cross-business-unit team focused on the business aspects of managing data as a valued enterprise assetDSC is a part-time, virtual team of Allstate employees who have strong business knowledge, vision, and the ability to look horizontally across the enterpriseData stewards are focused on addressing the business issues behind key data resource management objectives: managing data redundancy, implementing data shareability and standardization, and managing and improving data integrityStewards follow several basic principles for managing data resources of any type; these include the following:

Requirements for the resource must be anticipated and fulfilled proactivelyAllstate cannot afford an infinite amount of the data resource; therefore, the amount must be optimizedData resource should be shared and leveraged in as many ways as possible, in order to maximize its value while diminishing its overall costsData resource must be carefully managed to ensure that its use in the business is prudent, efficient, effective, and secure


313



Department of DefenseDepartment of Defense

Best Business Intelligence ApplicationInformation Integration



How Large is the DoD Supply Chain?

$480 Billion Dollars Budget$80 Billion Supply Chain3 Million People8 Million PartsGlobal – 150 CountriesDynamic Supply Chain

EWSolutionsBest Business Intelligence Application

Information Integration


314



What’s the Challenge

Can’t Account for Several Billion Dollars a YearHave Data Quality IssuesFix or Refine a Process and Break Several Other ProcessesNever Planned the Enterprise The Enterprise “Just Grew”



DoD Enterprise Architecture is Based on:Support rapid, agile deployment, employment, sustainment and reset/reconstitution Total Force,

within acceptable risk, across the full spectrum of operations in a cost-effective manner (ROI)


315



Enterprise Integrated Data Environment (EIDE)

Provide an enhanced environment that enables the DoD Logistics Enterprise to execute practices, processes, applications and decision support tools to achieve logistics interoperability and allow for information exchange within and between internal and external DoD business partners.

- Non-system dependent transactions- Consolidation and reuse of Interfaces- Data integration/sharing- Leverage Modernization Efforts- Data Standards not Standard Data



DoD Enterprise Logistics

DMMSS(G005M)

ATLASSII+

CEMS1(D042A)

REMIS

CAEMS

MUMMS-SS06

LOGMOD

OWRMR(D072)

RSSL(D040)CMOS

JFAST

GDSSTOPS

WPS

IBS

AMS(MTMC)

CCSSGFM(CFM-HOST)

TC-AIMSII

SDS

MRDB MRDB

SAMMS EMACS

WARRS

IMWRP(D035A)CMOS

CALM

LOGMODWSMIS-ETADS(O132A)

CEMS1(D042A)

MUMMS-SS10

ATLASSII+

TMS

MUMMS-SS06

CAEMS

MRP(MC)

DISMSDISMS

DSS DAISY

FAS

HMIS

TOPS

GTN JFAST

WPS

IBS

MISILCISIL

TC-ACCIS

SDSSARSS-1 MTMS

MRDBSPBS-R

AFMIS

COMPASS

ERSSARSS-2AC/2BAMS(MTMC)

TC-AIMSII

SAAS-MOD

AALPS

ETADS

DSS FAS

DAISY

HMIS

TOPS

WPS IBS

JFAST

GTN

MISILCISIL

SARSS-GATEWAY

COMPASS

TC-AIMSIIAALPS TC-ACCIS

AMS(MTMC)SARSS-1

MIDAS

AMS(MTMC)

CCSS

SAAS-MODCOMPASS SPBS-R

SARSS-GATEWAYTC-ACCIS

AFMIS

MIDAS

SARSS-2AC/2BSARSS-2AD SARSS-1

SDS

SUPMIS

ROLMS TANDEM

RAM

UICP-RESYS

LOGMOD

REMISMIICS(D043)

WARRSIMWRP(D035A)

JAMSSRSSL(D040)

CMOS

SBSS/ILS-SAFPS(D220)

SCS(D035)WSMIS-ETADS(O132A)

CALMETADS

OWRMR(D072)

CEMS1(D042A)

ITEMAPPS

MRP(MC)MUMMS-SS07

MUMMS-SS10

SASSYMAARSII

ATLASSII+

CAEMS

DAISY

DSS

BOSS

EMACS

FAS

DISMS

SAMMS

TOPS

WPS IBS

JFAST

GTN

AFMIS

TC-ACCIS

AMS(MTMC)CCSS TC-AIMSII

COMPASSMIDAS

SPBS-R

AALPS

SARSS-GATEWAYSARSS-1 SARSS-2AD

SARSS-2AC/2B

SAAS-MOD

SDS

TANDEM

RAM UICP-RESYS

SUPMIS

ROLMS

CALM

WSMIS-ETADS(O132A)LOGMOD

CMOSREMIS

AFPS(D220)MIICS(D043)

ETADS

SBSS/ILS-SSCS(D035)

IMWRP(D035A)CEMS1(D042A)

JAMSSWARRS

OWRMR(D072)

RSSL(D040)

MUMMS-SS07

ATLASSII+MAARSII

ITEMAPPS

MRP(MC)MUMMS-SS10

SASSY

CAEMS

BOSS

DISMS

DSS

FAS

SAMMS

EMACS

DAISY

JFAST

TOPS WPS

GTN

IBS

ACCLAIMS COMPASS

ATAVGCSS-A LIDB

TC-AIMSIISAAS-MODELIST

IC3

CMOS

BROKERCEMS1(D042A)

LOGMODMANPER-B

MAGTFII

CAEMS

LMIS

ATLASSI

MCDSS

JFRGIIMDSSII

MUMMS-SS06

WRS

GCCSICIS

TOPS

AMPAPODModel

IBS

WPSGDSS

JFASTBDSS

OWRMR(D072)

RSSL(D040)

SAMMS

SARSS-2AC/2B

SARSS-2ADSARSS-1

SPBS-RSDS

SARSS-GATEWAYSAAS-MOD

MIDASAMS(MTMC)

CCSSATS

UITATAV

DESCBS-X

AFMIS

ROLMS

SUPMIS

FACTSUICP-RESYS

HMMS

CAIMSMMS

RAM

TANDEM

MIICS(D043)

CEMS1(D042A)

AFPS(D220)REMIS

JTAVSAMMS

BOSSDFAMS

DAISY

DESEXDSS

DISMS

FAS EMACS

SDSCCSS

REMISDMMSS(G005M)

ATLASSI

ATLASSII+

MDSSII

EMACS

SDSCCSS

REMIS

OWRMR(D072) RSSL(D040)

DMMSS(G005M)

ATLASSII+

MDSSII

ATLASSI

EMACSSAMMS

DAMMS-RBLKIII

GFM(CFM-HOST)

AMS(MTMC) ELIST

TC-ACCIS

GOPAX

FACTSNAOMIS

CMOS

GATES LOGMOD

GTN21

TC-AIMS

CAEMS MDSSII

MAGTFII

JFRGII

WPS

JFAST GTN

GDSS

IBS

SDSIFS

CCSS

DMMSS(G005M)

EMACS

DBMS

DAMMS-RBLKIIIELIST

TC-AIMSIICOMPASS

GFM(CFM-HOST)GOPAX

TC-ACCISAMS(MTMC)

AALPS

FACTSNAOMIS

GATES

ETADS

CALM

WSMIS-ETADS(O132A)

LOGMOD

CMOS

GTN21

JFRGII

MAGTFII CAEMS

TC-AIMS

MDSSII

JFAST

WPS

GTN

GDSS

TOPS

IBS

TRAC2ES

SARSS-2AD

CCSSCBS-X

DESUIT

SDSATS

SARSS-GATEWAYSARSS-2AC/2B

SARSS-1ATAV

SPBS-RAFMIS

MIDASAMS(MTMC)

SAAS-MOD

UICP-RESYS

RAM

ROLMSTANDEM

HMMS

CAIMSSUPMIS

MMS

FACTS

REMIS

MIICS(D043)

AFPS(D220)OWRMR(D072)

JAMSSSIS(D035T)

RSSL(D040)

SBSS/ILS-SCEMS1(D042A)

SCS(D035)SSRAC(D169)

WARRS

IMWRP(D035A)

MUMMS-SS07

MRP(MC)MUMMS-SS10

SASSY

MUMMS-SS06

ATLASSII+

MDSSIIATLASSI

ITEMAPPS

MAARSII

DISMS

DESEXDSS

DFAMS

DAISY

BOSS

JTAVFAS

SAMMS

EMACS

SARSS-2AC/2B

SARSS-GATEWAY

GFM(CFM-HOST)

SARSS-1 ACLDBFEDLOG

ERSMIDAS SDS

SB-700-20

SPBS-RSAAS-MOD

AFMISCCSS EIC

LIF MRDBMILSTEP

SARSS-2AD

MMS

ROLMS

SUPMIS

UICP-RESYS

RAM

TANDEM

MIICS(D043)

AFERS(D143C)REMIS

CSCPS(D155)CE-CM(D042E)

CEMS1(D042A)WARRS

FSSSYSTEM

SSRAC(D169)

DMMSS(G005M)


SBSS/ILS-SSCS(D035)

IMWRP(D035A)MIIDB(D043A)

JAMSS

RSSL(D040)

ATLASSII+

ITEMAPPS

MRP(MC)MUMMS-SS05

TDMS

SASSY

SAMMS

EMACS

MEDALSFAS

DAISYBOSS

BSM

DSSDISMS

DESEXDFAMS

TOPS

RASFIARS

CISILMISIL

MOCAS

MTMS

SDSERS SPBS-R

MRDBSAAS-MOD CCSS

AFMISSARSS-1SARSS-2AC/2B

JCALS

ATIS(32BIT)

ATIS(16BIT)

MSPECS

MCA

SETS

JEDMICS

CE-CM(D042E)

CEMS1(D042A)

REMIS

DMMSS(G005M)

IMWRP(D035A)

WARRS

OLVIMS

MUMMS-SS10

ATLASSII+SASSY

MUMMS-SS06

TMSTDMS

MRP(MC)

CAEMS

TMPODS

DSSEMACS

MEDALS

HMISFAS

DISMS

DAISY

MISILCISIL

MIIDB(D043A)

JAMSSRSSL(D040)

FSSSYSTEMOWRMR(D072)

MIICS(D043)SBSS/ILS-S

CEMS1(D042A)

WARRS

AFPS(D220)

CE-CM(D042E)IMWRP(D035A)

SCS(D035)REMIS

CSCPS(D155)DMMSS(G005M)

SSRAC(D169)

AFERS(D143C)

ATLASSII+

MAARSII

SASSYMRP(MC)

ATLASSI

ITEMAPPS

TDMS

MUMMS-SS05MUMMS-SS10

MUMMS-SS07

MUMMS-SS06

TMPODS

FLIS

CALM

ETADSIMWRP(D035A)

CEMS1(D042A)

WARRSCMOS

LOGMOD

WSMIS-ETADS(O132A)

MRP(MC)

ATLASSII+

MUMMS-SS06

TMS

MUMMS-SS10

CAEMS

SDS

SPBS-RAFMIS CCSS

SAAS-MOD

SARSS-2AC/2B

SARSS-2AD

ROLMS

RAM TANDEM

UICP-RESYS

SUPMIS

AFPS(D220)

CALMOWRMR(D072)

RSSL(D040)JAMSS

ETADSREMIS

SBSS/ILS-S

WSMIS-ETADS(O132A)CEMS1(D042A)

MIICS(D043)WARRS

IMWRP(D035A)SCS(D035)

LOGMOD

CMOS

MAARSII

CAEMSSASSY

MUMMS-SS10

MUMMS-SS07MRP(MC)

ITEMAPPS

ATLASSII+

FAS

DAISY

DSS

SAMMS

EMACS

BOSS

DISMS

IBS

GTN WPS

TOPS

JFAST

AALPS

TC-AIMSII

OWRMR(D072)JAMSS

SIS(D035T)

RSSL(D040)SSRAC(D169)

SCS(D035)WARRS

IMWRP(D035A)

SBSS/ILS-S

MUMMS-SS10

MDSSII

SASSYATLASSII+

MRP(MC)

MUMMS-SS07MAARSII

ATLASSI

ITEMAPPS

MAARSII

MUMMS-SS10ATLASSI

MUMMS-SS07

MUMMS-SS06

DFAMS

EMACS

FLISDSS

MEDALSDISMS

SAMMS

DESEXDAISY

BSMBOSS

TMPODS

FAS

TOPS

CISIL

RASFIARS MOCAS

MISIL

CCSS

AFMISGFM(CFM-HOST) EIC

SDSSPBS-R MILSTEP

LIFMRDB SARSS-2AC/2B

ERSFEDLOG SARSS-GATEWAY

SB-700-20SARSS-1 SAAS-MOD

SARSS-2ADMIDAS

ACLDB

SUPMIS

MMS

ROLMS

TANDEM

RAM

UICP-RESYS

WARRS

SCS(D035)REMIS

JAMSSRSSL(D040)

IMWRP(D035A)SBSS/ILS-S

CEMS1(D042A)

AFERS(D143C)

CSCPS(D155)

SSRAC(D169)DMMSS(G005M)


MIIDB(D043A)MIICS(D043)

FSSSYSTEM

CE-CM(D042E)

ATLASSII+

SASSY

TDMSMUMMS-SS10

MUMMS-SS07

MUMMS-SS06

MUMMS-SS05

MRP(MC)ITEMAPPS

ATLASSI

MAARSII

DISMS

FLIS

ELISTATAV

SAAS-MOD

ACCLAIMSGCSS-A

COMPASS

IC3

MANPER-BBROKER

MCDSSWRS

JFRGII

MAGTFIIMDSSII

ATLASSILMIS

GCCSICIS

AMPAPODModel

SDSAFMIS SAAS-MOD

SARSS-1

SARSS-2AC/2B

ERS SPBS-R

MTMSJCALSCCSS

MSPECS

JEDMICS

ATIS(16BIT)

ATIS(32BIT)

MCA

SETS

WARRS

IMWRP(D035A)

CE-CM(D042E)

OLVIMS

FASDSS

SAMMSEMACS

MEDALS

BSMDAISY

DFAMSBOSS

TMPODS

DESEX

TOPS

MOCAS

MISILCISIL

RASFIARS

SAAS-MOD

AFMIS LIF

GFM(CFM-HOST)

SARSS-2AC/2B MIDAS

MILSTEPMRDB SPBS-R

FEDLOG

ERSSDS ACLDB

SB-700-20CCSS SARSS-2AD

SARSS-GATEWAY

SARSS-1

EIC

RAM

SUPMIS

TANDEM

UICP-RESYS

MMS

ROLMS

DMMSS(G005M)

REMIS

CEMS1(D042A)

ATLASSII+MRP(MC)

MUMMS-SS10

SASSYTMS

MUMMS-SS06

TDMS

CAEMS

HMIS

EMACSMEDALS

TMPODS

DAISYFAS

DISMS

DSS

MISILCISIL

SPBS-RSDS AFMIS

SARSS-2AC/2B

CCSS

MTMS JCALS

SARSS-1SAAS-MODERS

SETS

JEDMICS

ATIS(16BIT)

ATIS(32BIT)

MCA

MSPECS

WARRS

IMWRP(D035A)

DMMSS(G005M)

OLVIMS

REMIS

CE-CM(D042E)

CEMS1(D042A)

MUMMS-SS06

ATLASSII+TMS

MUMMS-SS10

MRP(MC)TDMS

SASSY

CAEMS

DSS

DAISYEMACS

TMPODS

DISMSMEDALS

FAS

HMIS

CISILMISIL

COMPASS

MTMSSPBS-R SDS

TC-ACCISAMS(MTMC)

SARSS-2AC/2B

SARSS-1

SAAS-MODAALPS TC-AIMSII

MRDB

ERS

AFMIS

CMOS

LOGMODCALM

IMWRP(D035A)

WSMIS-ETADS(O132A)WARRS

ETADS

CEMS1(D042A)

TMS

ATLASSII+

CAEMS

MRP(MC)

MUMMS-SS06

MUMMS-SS10

DAISY

HMIS FAS

DISMS

DSS

GTN

WPS IBS

TOPS

JFAST

CISILMISIL

SAAS-MOD

AFMISERS AMS(MTMC)

TC-ACCISMTMS

AALPS

SARSS-1

COMPASSSARSS-2AC/2BTC-AIMSII

SDS

MRDB

SPBS-R

ARMY

NAVY

AIR FORCE(USAF)

MARINE CORPS(USMC)

DLA

TRANSCOM

OTHER

PLAN SOURCE MAKE DELIVER RETURN ENABLE

Plan

Sup

ply

Chain

Plan

Sou

rce

Plan

Mak

e

Plan

Deli

ver

Plan

Ret

urn

Sour

ce

Stock

ed P

rodu

ctSou

rce

Make-

to-O

rder

Prod

uct

Sourc

e En

gine

er-to

-Ord

er

Prod

uct

Make-

to-S

tock

Make-

to-O

rder

Engi

neer

-to-O

rder

Delive

r Sto

cked

Prod

uct

Delive

r Mak

e-to

-Ord

er

Prod

uct

Delive

rEn

gine

er-to

-Ord

er

Prod

uct

Retur

n Def

ectiv

e

Prod

uct

Retur

n MRO

Produ

ct

Retur

n Ex

cess

Prod

uct

Enab

le Pl

an

Enabl

e Sou

rce

Enab

le M

ake

Enab

le D

elive

rEna

ble R

eturn

DoD ENTERPRISE LOGISTICSOperational Activity to System Function Traceability Matrix (SV-5)

NOTE: The data displayed in thisdiagram is collected from the LSM'sFuture Logistics Enterprise OV-5.

= System with Operational to System Process Mapping

= System with Operationalto System Activity Mapping


316



Canonical Model and Legacy Information within the EIDE

NSN UID Navy Part # Supplier Part #

1615011975441 25.77272.34 1615011975441AH 114R2080-9

1615013005539 45.97499.43 1615013005539AH 540-011-101-129

Business Process &

UID

Canonical Model



Distributed Meta Data Architecture (DMDA)

End User -Reporting Tool

(e.g. Data Shopper, Cognos,Crystal Reports)

COI Data Champion

NavyDLA/DSS

Global Meta Data

Public Meta DataPrivate Meta Data

Other Repositories

Meta Data

Air force

Meta Data Sources

Global Data Champion

Logistics Domain Global Repository

DOD Meta Data

Registry

Msg/Trans Broker

Rules Engine

Army Marines

Weapons Platforms

Suppliers

Other Domains(Acquisition)

USTranscom


317



Applying ComplianceApplying ComplianceArchitecture and ProgramsArchitecture and Programs

The ToolkitThe Toolkit””



Architecture and Program ComplianceComponent Architecture Evaluation

INITIATION

Preparation

Define Problem/Challenge

Business Case(High Level)

AlternativesAnalysis

Requirements(High Level)

Project Charter

Approval

Approval

ACQUISITION

Preparation

Requirements(mid-level)

Develop andIssue RFP/RFQ

Vendor Evaluations

License/Agreement

Award

Approval

Preparation

Develop andIssue RFP/SOO

VendorEvaluations

Service Agrmt& Task Order(s)

Award

Approval

IMPLEMENTATION

Preparation

Blueprint *

Design/Build **

Transition/Cutover

Approval

InitialGo-Live Support

Approval

Project Close

OngoingSupport

Process/FunctEnhancementsCompetency

CenterReviews and

Training

ProductionRollout

Upgrades

SUSTAINMENT

CONTINUOUSIMPROVEMENT

Systems IntegrationServicesSoftware

PII EvaluationAdvocacy Criteria I

PI Evaluation

SCRs: Architecture Evaluations and

Advocacy Reviews as required during Sustainment / Improvement Stages

= Architecture

= Domain Owner Advocacy

Advocacy Criteria III b

Milestone A

Milestone B1

Milestone C

Milestone B2

SCR Evaluation

Criteria

Go Live

Advocacy Criteria II

Advocacy Criteria IIIa

Advocacy Criteria IVb

Advocacy Criteria IVa

* Blueprint = Detailed Requirements and High-Level Design**Design/Build = Detailed Design, Configure/Develop, and Test

= Milestone Decision

= Domain Advocacy Criteria

Advocacy Criteria V

PIIIApproval


318



Summary

Plan Your Enterprise Around the “To Be Business Processes” not your “As Is” (80-20 Rule)Data Strategy is the Key to Data Interoperability Across the EnterpriseNo Meta Data Management, No Data StrategyCompliance is an “On-Going Process”Your Business Processes Should Not Be Unique

EWSolutionsBest Business Intelligence Application

Information Integration



DonDon’’t Limit Yourselft Limit Yourself


319



Questions


320