data vault whats the combo

8/3/2019 Data Vault Whats the Combo

1/31

Data Vault

RMOUG Training Days

2006Colorado Convention Center

Denver, Colorado

February 15-16


2/31

Data Vault;Whats The Combination?

Jeff MeyerEnterprise Data Integration Oracle DBA

Department of Technology Services

Denver Public Schools


3/31


4/31

Data Vault

Brief History and Revisit Some

Definitions Three Basic Building Blocks of the

Data Vault

Advanced Features Questions


5/31

Data Vault



Data Vault



6/31

Data Vault

BriefHistory and Revisit Some Definitions

1970 Dr. E.F. Codd of IBM

1979 First

Working RelationalDatabase by Relational Software

Incorporated

Oracle v2 1991 William H. Inmon published

Building the Data Warehouse


7/31

Data Vault


Legacy System any system that has been put into production.

(para-phrased W.H. Inmon)

Operational Data Store a subject-oriented, integrated, volatile, current

or near current collection of operational data.W.H. Inmon


8/31

Data Vault


Data Warehouse a subject-oriented, integrated, time-variant, non-volatile

collection of data designed for support of business decisionsW.H. Inmon

Data Vault a detail-oriented, historical tracking and uniquely linked set

of normalized tables that support one or more functionalareas of business.

Dan Linstedt


9/31

Data Vault


Data Mart

a subset of a data warehouse, for use by a singledepartment or function.

www.e-formation.co.nz/glossary.asp

Corporate Information Factory the framework that exists that surrounds the data

warehouse; typically contains an ODS, a data warehouse, data

marts, DSS applications, exploration warehouses, and so forth.

W.H. Inmon


10/31

Data Vault


* Source: Bill Inmon and Claudia Imhoff


11/31

Data Vault Why?

Why do we need it? We finally have a Data Model that will work for small,

medium, or large business Anyone building a Data Warehouse can use these techniques.

Weve got issues in constructing the data warehouse

from 3rd normal form, or star schema form. There are inherent road blocks to each method that we must

solve technically through our Data Model.


12/31

Data Vault



Data Vault



13/31

Data Vault

Th

ree

Basic Building Blocks

Hub stand alone table; list of unique business

keys; used for business identification

Satellite descriptive data; historical data; usedfor descriptive information for the HUB orLINK

Link associative table; list of uniquerelationships between keys; used for relationshipsbetween HUBs and LINKs


14/31

Data Vault Three Basic Building Blocks

Preview

Hub Employees

Hub Schools

ELAName

EEOCDates

Hub Students

EEOCName

ShotsAddrs

Assign Enrollments


15/31

Data Vault Three Basic Building BlocksHUB

Primary Key

Load DTS

Record Source

Primary Key

Load DTS

Record Source

Sample Data Set CUSTOMER

CONTRACTS2-2-200093KFLLA10

CONTRACTS2-2-2000929ABC29859

CONTRACTS2-2-2000PAFJG28958

FINANCE2-2-2000PPRU_32597

SALES8-3-2001HUJI_BFIOQ6

SALES6-4-2001LLOA_82J5J5

CONTRACTS3-7-2000KKO92854_dd4

CONTRACTS1-25-2000DKEF3

CONTRACTS10-2-2000ABC925_24FN2

RCRD SRCLOAD DTSCUSTOMER #ID

MANUFACT10-12-2000ABC1234561

CONTRACTS2-2-200093KFLLA10

CONTRACTS2-2-2000929ABC29859

CONTRACTS2-2-2000PAFJG28958

FINANCE2-2-2000PPRU_32597

SALES8-3-2001HUJI_BFIOQ6

SALES6-4-2001LLOA_82J5J5

CONTRACTS3-7-2000KKO92854_dd4

CONTRACTS1-25-2000DKEF3



MANUFACT10-12-2000ABC1234561

A Hub is a list of unique business keys.


16/31


SATELLITE

Primary Key

Load DTS

Detail

Business Data

Aggregation Data

{Update User}

{Update DTS}

Record Source

Primary Key

Load DTS

Detail

Business Data

Aggregation Data

{Update User}

{Update DTS}

Record Source


MANUFACT10-12-2000ABC1234561



MANUFACT10-12-2000ABC1234561


CONTRACTSWorldwide Suppliers Inc10-14-20002

CONTRACTSWorldPart10-2-20002

CONTRACTSABC DEF Incorporated12-2-20001

MANUFACTABC Worldwide Suppliers, Inc10-31-20001

MANUFACTABC Suppliers, Inc10-14-20001

MANUFACTABC Suppliers10-12-20001

RCRD SRCNAMELOAD DTSCSID

CONTRACTSWorldwide Suppliers Inc10-14-20002

CONTRACTSWorldPart10-2-20002

CONTRACTSABC DEF Incorporated12-2-20001

MANUFACTABC Worldwide Suppliers, Inc10-31-20001

MANUFACTABC Suppliers, Inc10-14-20001

MANUFACTABC Suppliers10-12-20001

RCRD SRCNAMELOAD DTSCSID

CUSTOMER NAME SATELLITE

A Satellite is a time-dimensional table housing detailed

information about the hubs business keys.


17/31


Hub Employees

ELAName

EEOCDates

Employees HUB and some of its Satellites


18/31


LINK

Primary Key

Load DTS

Record Source


MANUFACT10-12-2000ABC1234561



MANUFACT10-12-2000ABC1234561


FINANCE10-14-2000CONT259101


RCRD SRCLOAD DTSCONTACT #ID



RCRD SRCLOAD DTSCONTACT #ID

FINANCE10-14-20001012

FINANCE10-14-20001001

RCRD SRCLOAD DTSCONTACT IDCSID

FINANCE10-14-20001012

FINANCE10-14-20001001

RCRD SRCLOAD DTSCONTACT IDCSID

A Link is an associative or intersection table, representing the

connection between information between business elements.

Link Table


19/31


Hub Employees

ELAName

EEOCDates

Hub Schools

Geo CdAddr

FloorBldg

Assign

Sat

Hub and SatellitesHub and Satellites

Link and Satellites


20/31

Data Vault


Definitions

Three Basic Building Blocks of the

Data Vault



21/31

Data Vault Advanced Features

Point-In-Time A structure which sustains integrity of joins across time to all

the SATELLITES that are connected to the HUB orLINK.

Bridge A single row table that contains the latest Load Date Time

Stamp (DTS). Similar to Point-In-Time except it spans a

subject-area or a schema.

User GroupingL

ink The information provides the user with a customized view

from a reporting standpoint and does not affect the

underlying information.


22/31


Point-In-Time (PIT)

MANUFACT10-12-2000ABC1234561


MANUFACT10-12-2000ABC1234561


ABC DEF Incorporated12-2-20001

ABC Worldwide Suppliers, Inc10-31-20001

NAMELOAD DTSCSID

ABC DEF Incorporated12-2-20001

ABC Worldwide Suppliers, Inc10-31-20001

NAMELOAD DTSCSID

123 World Drive12-5-20001

123 World Dr10-14-20001

ADDRESSLOAD DTSCSID

123 World Drive12-5-20001

123 World Dr10-14-20001

ADDRESSLOAD DTSCSID

12-5-200012-2-200012-5-20001

10-14-200012-2-200012-2-20001

10-31-2000

10-14-2000

NAME_LOAD_DTS

10-14-2000

10-14-2000

ADDRESS_LOAD_DTS

10-31-20001

10-14-20001

LOAD DTSCSID

12-5-200012-2-200012-5-20001

10-14-200012-2-200012-2-20001

10-31-2000

10-14-2000

NAME_LOAD_DTS

10-14-2000

10-14-2000

ADDRESS_LOAD_DTS

10-31-20001

10-14-20001

LOAD DTSCSID

A structure which sustains integrity of joins across time to

all the satellites that are connected to the hub.

Customer Name Satellite Customer Address Satellite

Hub Key

Load Date

{Sat Load DTS}

{Sat Load DTS}

{Rec Source}

Hub Key

Load Date

{Sat Load DTS}

{Sat Load DTS}

{Rec Source}


23/31


B

ridge

A single row table that contains the latest

Load DTS with multiple columns. A Bridgeis not a helper table.

Similar to a PIT Table except it spans or

applies to a subject-area or schema. A PITTable is HUB (LINK) and SATELLITE

specific.


24/31


Us

er GroupingLink

Primary Key

Load DTS

Record Source

Primary Key

Load DTS

Record Source

EXCEL10-2-2000Small Customers2

EXCEL10-12-2000Big Customers1

RCRD SRCLOAD DTSGrouping LabelID

EXCEL10-2-2000Small Customers2

EXCEL10-12-2000Big Customers1

RCRD SRCLOAD DTSGrouping LabelID

FINANCE10-14-2000ABC-1101

FINANCE10-14-2000ABC295882100

RCRD SRCLOAD DTSCustomer #ID

FINANCE10-14-2000ABC-1101

FINANCE10-14-2000ABC295882100

RCRD SRCLOAD DTSCustomer #ID

EXCEL10-14-20001011

EXCEL10-14-20001001

RCRD SRCLOAD DTSCustomer #Grp#

EXCEL10-14-20001011

EXCEL10-14-20001001

RCRD SRCLOAD DTSCustomer #Grp#

The User Grouping Link, allows users to state how they want

roll-ups to occur in situations where source data doesnt exist.

BASE TABLE:


25/31

Data Vault How is DPS using DVHub_Students

Student_ID

SIS_Code

Load_DTS

Rec_SRC

Hub_Students

Student_ID

SIS_Code

Load_DTS

Rec_SRC

Hub_Schools

School_ID

School_Number

Load_DTS

Rec_SRC

Hub_Schools

School_ID

School_Number

Load_DTS

Rec_SRC

Hub_Employees

Employee_ID

HR_Emp_ID

DPSID

Load_DTS

Rec_SRC

Hub_Employees

Employee_ID

HR_Emp_ID

DPSID

Load_DTS

Rec_SRC

Lnk_School_Enrollments

Sch_Enr_IDSchool_ID

Student_ID

Grade_Name

Load_DTS

Rec_SRC

Lnk_School_Enrollments

Sch_Enr_IDSchool_ID

Student_ID

Grade_Name

Load_DTS

Rec_SRC

Lnk_Teacher_Schools

Teacher_School_ID

School_ID

Employee_ID

Load_DTS

Rec_SRC

Lnk_Teacher_Schools

Teacher_School_ID

School_ID

Employee_ID

Load_DTS

Rec_SRC

The directionof the arrows

equate to

crows feet.


26/31

Data Vault Why is DPS using DV

Storage considerations.

Vertical partitioning of data (rate ofchange).

All the FACTS all the TIME.

Scalability and Extensibility.


27/31

Data Vault What was not covered.

How to apply Data Vault Modeling.

Best practices.

Lessons Learned. Dan Linstedts use of DECODE in

determining changed data capture.

Whos data is it? S

LAs?

The new regulations / compliance that willaffect all of us.


28/31

Data Vault Questions?


29/31

Data Vault - References

DATA VAULT OVERVIEW: THE NEXT EVOLUTION IN DATA MODELINGDan Linstedt - Core Integration Partners, Inc.http://www.tdan.com/i021hy01.htm

DATA VAULT OVERVIEW THE NEXT EVOLUTION IN DATA MODELING SERIES 2Dan Linstedt - Core Integration Partners, Inc.http://www.tdan.com/i023hy02.htm

DATA VAULT - SERIES 3 END-DATES AND BASIC JOINSDan Linstedt - Core Integration Partnershttp://www.tdan.com/i024hy02.htm

DATA VAULT - SERIES 4 LINK TABLESDan Linstedt - Core Integration Partnershttp://www.tdan.com/i027ht04.htm

DATA VAULTTM OVERVIEW THE NEXT EVOLUTION IN DATA MODELING SERIES 5 LOADING TABLESDan Linstedt - Core Integration Partnershttp://www.tdan.com/i027ht04.htm

Data Vault Modeling Class Materials and Notes; copyright 2002-2003Dan Linstedt Core Integration Partnershttp://www.coreintegration.com

Home of the Data Vault; www.danlinsedt.com

Audit the Data or Else. Un-audited Data Access Puts Business at High Risk; Bloor, Robinand Baroudi, Carol; Lumigent, Inc.; copyright 2004


30/31

Data Vault Contact Information

JEFFREY MEYER

[email protected]


31/31

Data Vault

RMOUG Training Days

2006Colorado Convention Center

Denver, Colorado

February 15-16

data vault whats the combo

Documents