data vault whats the combo-1

5/21/2018 Data Vault Whats the Combo-1

1/31

Data Vault

RMOUG Training Days

2006Colorado Convention Center

Denver, Colorado

February 15-16


2/31

Data Vault;Whats The Combination?

Jeff Meyer

Enterprise Data IntegrationOracle DBA

Department of Technology Services

Denver Public Schools


3/31

Data Vault

Who are we? DBAs

Managers

Analysts

Enterprise Data Warehouse Projects Currently in process

Planned

Data Marts


4/31

Data Vault

Brief History and Revisit Some

Definitions

Three Basic Building Blocks of the

Data Vault

Advanced Features Questions


5/31

Data Vault


Definitions


Data Vault



6/31

Data VaultBrief History and Revisit Some Definitions

1970Dr. E.F. Codd of IBM

1979First Working Relational

Database by Relational Software

Incorporated

Oracle v2 1991William H. Inmon published

Building the Data Warehouse


7/31


Legacy System any system that has been put into production.

(para-phrased W.H. Inmon)

Operational Data Store a subject-oriented, integrated, volatile, current

or near current collection of operational data.W.H. Inmon


8/31


Data Warehouse a subject-oriented, integrated, time-variant, non-volatile

collection of data designed for support of business decisions

W.H. Inmon

Data Vault a detail-oriented, historical tracking and uniquely linked set

of normalized tables that support one or more functionalareas of business.

Dan Linstedt


9/31


Data Mart a subset of a data warehouse, for use by a single

department or function.

www.e-formation.co.nz/glossary.asp

Corporate Information Factory

the framework that exists that surrounds the datawarehouse; typically contains an ODS, a data warehouse, data

marts, DSS applications, exploration warehouses, and so forth.

W.H. Inmon
http://www.e-formation.co.nz/glossary.asphttp://www.e-formation.co.nz/glossary.asphttp://www.e-formation.co.nz/glossary.asphttp://www.e-formation.co.nz/glossary.asp


10/31


* Source: Bill Inmon and Claudia Imhoff


11/31

Data VaultWhy?

Why do we need it? We finally have a Data Model that will work for small,

medium, or large business Anyone building a Data Warehouse can use these techniques.

Weve got issues in constructing the data warehouse

from 3rdnormal form, or star schema form. There are inherent road blocks to each method that we must

solve technically through our Data Model.


12/31

Data Vault


Definitions


Data Vault



13/31

Data VaultThree Basic Building Blocks

Hubstand alone table; list of unique business

keys; used for business identification Satellitedescriptive data; historical data; used

for descriptive information for the HUB or LINK

Linkassociative table; list of uniquerelationships between keys; used for relationshipsbetween HUBs and LINKs


14/31

Data VaultThree Basic Building BlocksPreview

Hub Employees

Hub Schools

ELAName

EEOCDates

Hub Students

EEOCName

ShotsAddrs

Assign Enrollments


15/31

Data VaultThree Basic Building BlocksHUB

Primary Key

Load DTS

Record Source

Primary Key

Load DTS

Record Source

Sample Data Set CUSTOMER

CONTRACTS2-2-200093KFLLA10

CONTRACTS2-2-2000929ABC29859

CONTRACTS2-2-2000PAFJG28958

FINANCE2-2-2000PPRU_32597

SALES8-3-2001HUJI_BFIOQ6

SALES6-4-2001LLOA_82J5J5

CONTRACTS3-7-2000KKO92854_dd4

CONTRACTS1-25-2000DKEF3

CONTRACTS10-2-2000ABC925_24FN2

RCRD SRCLOAD DTSCUSTOMER #ID

MANUFACT10-12-2000ABC1234561

CONTRACTS2-2-200093KFLLA10

CONTRACTS2-2-2000929ABC29859

CONTRACTS2-2-2000PAFJG28958

FINANCE2-2-2000PPRU_32597

SALES8-3-2001HUJI_BFIOQ6

SALES6-4-2001LLOA_82J5J5

CONTRACTS3-7-2000KKO92854_dd4

CONTRACTS1-25-2000DKEF3



MANUFACT10-12-2000ABC1234561

A Hub is a list of unique business keys.


16/31

Data VaultThree Basic Building BlocksSATELLITE

Primary Key

Load DTS

Detail

Business Data

Aggregation Data

{Update User}

{Update DTS}

Record Source

Primary Key

Load DTS

Detail

Business Data

Aggregation Data

{Update User}

{Update DTS}

Record Source

CONTRACTS10-2-2000ABC925_24FN2MANUFACT10-12-2000ABC1234561


CONTRACTS10-2-2000ABC925_24FN2MANUFACT10-12-2000ABC1234561


CONTRACTSWorldwide Suppliers Inc10-14-20002

CONTRACTSWorldPart10-2-20002

CONTRACTSABC DEF Incorporated12-2-20001

MANUFACTABC Worldwide Suppliers, Inc10-31-20001

MANUFACTABC Suppliers, Inc10-14-20001

MANUFACTABC Suppliers10-12-20001

RCRD SRCNAMELOAD DTSCSID

CONTRACTSWorldwide Suppliers Inc10-14-20002

CONTRACTSWorldPart10-2-20002

CONTRACTSABC DEF Incorporated12-2-20001

MANUFACTABC Worldwide Suppliers, Inc10-31-20001

MANUFACTABC Suppliers, Inc10-14-20001

MANUFACTABC Suppliers10-12-20001

RCRD SRCNAMELOAD DTSCSID

CUSTOMER NAME SATELLITE

A Satellite is a time-dimensional table housing detailed

information about the hubs business keys.


17/31


Hub Employees

ELAName

EEOCDates

Employees HUB and some of its Satellites


18/31

Data VaultThree Basic Building BlocksLINK

Primary Key

Load DTS

Record Source


MANUFACT10-12-2000ABC1234561



MANUFACT10-12-2000ABC1234561


FINANCE10-14-2000CONT259101


RCRD SRCLOAD DTSCONTACT #ID



RCRD SRCLOAD DTSCONTACT #ID

FINANCE10-14-20001012

FINANCE10-14-20001001

RCRD SRCLOAD DTSCONTACT IDCSID

FINANCE10-14-20001012

FINANCE10-14-20001001

RCRD SRCLOAD DTSCONTACT IDCSID

A Link is an associative or intersection table, representing the

connection between information between business elements.

Link Table


19/31


Hub Employees

ELAName

EEOCDates

Hub Schools

Geo CdAddr

FloorBldg

Assign

Sat

Hub and SatellitesHub and Satellites

Link and Satellites


20/31

Data Vault


Definitions


Data Vault



21/31

Data VaultAdvanced Features

Point-In-Time A structure which sustains integrity of joins across time to all

the SATELLITES that are connected to the HUB or LINK.

Bridge A single row table that contains the latest Load Date Time

Stamp (DTS). Similar to Point-In-Time except it spans a

subject-area or a schema.

User Grouping Link The information provides the user with a customized view

from a reporting standpoint and does not affect the

underlying information.


22/31


Point-In-Time (PIT)

MANUFACT10-12-2000ABC1234561


MANUFACT10-12-2000ABC1234561


ABC DEF Incorporated12-2-20001

ABC Worldwide Suppliers, Inc10-31-20001

NAMELOAD DTSCSID

ABC DEF Incorporated12-2-20001

ABC Worldwide Suppliers, Inc10-31-20001

NAMELOAD DTSCSID

123 World Drive12-5-20001

123 World Dr10-14-20001

ADDRESSLOAD DTSCSID

123 World Drive12-5-20001

123 World Dr10-14-20001

ADDRESSLOAD DTSCSID

12-5-200012-2-200012-5-20001

10-14-200012-2-200012-2-20001

10-31-2000

10-14-2000

NAME_LOAD_DTS

10-14-2000

10-14-2000

ADDRESS_LOAD_DTS

10-31-20001

10-14-20001

LOAD DTSCSID

12-5-200012-2-200012-5-20001

10-14-200012-2-200012-2-20001

10-31-2000

10-14-2000

NAME_LOAD_DTS

10-14-2000

10-14-2000

ADDRESS_LOAD_DTS

10-31-20001

10-14-20001

LOAD DTSCSID

A structure which sustains integrity of joins across time to

all the satellites that are connected to the hub.

Customer Name Satellite Customer Address Satellite

Hub Key

Load Date

{Sat Load DTS}

{Sat Load DTS}

{Rec Source}

Hub Key

Load Date

{Sat Load DTS}

{Sat Load DTS}

{Rec Source}


23/31


Bridge

A single row table that contains the latest

Load DTS with multiple columns. A Bridgeis not a helper table.

Similar to a PIT Table except it spans or

applies to a subject-area or schema. A PITTable is HUB (LINK) and SATELLITE

specific.


24/31


User Grouping Link

Primary Key

Load DTS

Record Source

Primary Key

Load DTS

Record Source

EXCEL10-2-2000Small Customers2

EXCEL10-12-2000Big Customers1

RCRD SRCLOAD DTSGrouping LabelID

EXCEL10-2-2000Small Customers2

EXCEL10-12-2000Big Customers1

RCRD SRCLOAD DTSGrouping LabelID

FINANCE10-14-2000ABC-1101

FINANCE10-14-2000ABC295882100

RCRD SRCLOAD DTSCustomer #ID

FINANCE10-14-2000ABC-1101

FINANCE10-14-2000ABC295882100

RCRD SRCLOAD DTSCustomer #ID

EXCEL10-14-20001011

EXCEL10-14-20001001

RCRD SRCLOAD DTSCustomer #Grp#

EXCEL10-14-20001011

EXCEL10-14-20001001

RCRD SRCLOAD DTSCustomer #Grp#

The User Grouping Link, allows users to state how they wantroll-ups to occur in situations where source data doesnt exist.

BASE TABLE:


25/31

Data VaultHow is DPS using DVHub_Students

Student_IDSIS_Code

Load_DTS

Rec_SRC

Hub_Students

Student_IDSIS_Code

Load_DTS

Rec_SRC

Hub_Schools

School_IDSchool_Number

Load_DTS

Rec_SRC

Hub_Schools

School_IDSchool_Number

Load_DTS

Rec_SRC

Hub_Employees

Employee_ID

HR_Emp_ID

DPSID

Load_DTS

Rec_SRC

Hub_Employees

Employee_ID

HR_Emp_ID

DPSID

Load_DTS

Rec_SRC

Lnk_School_Enrollments

Sch_Enr_ID

School_ID

Student_ID

Grade_Name

Load_DTS

Rec_SRC

Lnk_School_Enrollments

Sch_Enr_ID

School_ID

Student_ID

Grade_Name

Load_DTS

Rec_SRC

Lnk_Teacher_Schools

Teacher_School_IDSchool_ID

Employee_ID

Load_DTS

Rec_SRC

Lnk_Teacher_Schools

Teacher_School_IDSchool_ID

Employee_ID

Load_DTS

Rec_SRC

The directionof the arrows

equate to

crows feet.


26/31

Data VaultWhy is DPS using DV

Storage considerations.

Vertical partitioning of data (rate ofchange).

All the FACTS all the TIME.

Scalability and Extensibility.


27/31

Data VaultWhat was not covered.

How to apply Data Vault Modeling.

Best practices.

Lessons Learned. Dan Linstedts use of DECODE in

determining changed data capture.

Whos data is it? SLAs?

The new regulations / compliance that willaffect all of us.


28/31

Data VaultQuestions?


29/31

Data Vault - References

DATA VAULT OVERVIEW: THE NEXT EVOLUTION IN DATA MODELING

Dan Linstedt - Core Integration Partners, Inc.http://www.tdan.com/i021hy01.htm

DATA VAULT OVERVIEW THE NEXT EVOLUTION IN DATA MODELING SERIES 2Dan Linstedt - Core Integration Partners, Inc.

http://www.tdan.com/i023hy02.htm

DATA VAULT - SERIES 3 END-DATES AND BASIC JOINSDan Linstedt - Core Integration Partners

http://www.tdan.com/i024hy02.htm

DATA VAULT - SERIES 4 LINK TABLESDan Linstedt - Core Integration Partners

http://www.tdan.com/i027ht04.htm

DATA VAULTTM OVERVIEW THE NEXT EVOLUTION IN DATA MODELING SERIES 5LOADING TABLESDan Linstedt - Core Integration Partners

http://www.tdan.com/i027ht04.htm

Data Vault ModelingClass Materials and Notes; copyright 2002-2003

Dan LinstedtCore Integration Partnershttp://www.coreintegration.com

Home of the Data Vault; www.danlinsedt.com

Audit the Dataor Else. Un-audited Data Access Puts Business at High Risk ; Bloor, Robinand Baroudi, Carol; Lumigent, Inc.; copyright 2004
http://www.tdan.com/i021hy01.htmhttp://www.tdan.com/i021hy01.htmhttp://www.tdan.com/i023hy02.htmhttp://www.tdan.com/i023hy02.htmhttp://www.tdan.com/i024hy02.htmhttp://www.tdan.com/i024hy02.htmhttp://www.tdan.com/i027ht04.htmhttp://www.tdan.com/i027ht04.htmhttp://www.tdan.com/i027ht04.htmhttp://www.tdan.com/i027ht04.htmhttp://www.coreintegration.com/http://www.coreintegration.com/http://www.danlinsedt.com/http://www.danlinsedt.com/http://www.coreintegration.com/http://www.coreintegration.com/http://www.tdan.com/i027ht04.htmhttp://www.tdan.com/i027ht04.htmhttp://www.tdan.com/i027ht04.htmhttp://www.tdan.com/i027ht04.htmhttp://www.tdan.com/i024hy02.htmhttp://www.tdan.com/i024hy02.htmhttp://www.tdan.com/i023hy02.htmhttp://www.tdan.com/i023hy02.htmhttp://www.tdan.com/i021hy01.htmhttp://www.tdan.com/i021hy01.htm


30/31

Data VaultContact Information

JEFFREY MEYER

[email protected]


31/31

Data Vault

RMOUG Training Days

2006Colorado Convention Center

Denver, Colorado

February 15-16

data vault whats the combo-1

Documents

data model

data vaultwhats

data warehousefrom

volatilecollection of

data vaultwhy

inmon data vault

inmon5212018 data vault

data vaultbrief history