designmind data warehouse

8/14/2019 DesignMind Data Warehouse

1/31

DESIGNMINDARCHITECTING A DATA WAREHOUSE

A CASE STUDY

Project: zBis

Carl Zeiss Vision North America

ELIZABETH DIAMONDDATA WAREHOUSE ARCHITECT, DESIGNMIND

SAN FRANCISCO SQL SERVER USER GROUPSEPTEMBER 9, 2009


2/31

Tonights Speaker

Elizabeth Diamond

Senior Data Warehousing and BI Consultant DesignMind in Emeryville, CA

Specialist in Enterprise Data Management Systems

Lead Data Warehouse Architect and ETL Developer

Author and Speaker Building the Connection Between Your Business and Your IT

Infrastructure

Developing Your Enterprise Data Warehouse Using Business

Processes

Tonights Topic: Architecting a Data Warehouse


3/31

The Journey Determined Need for Enterprise Data Warehouse

Worked with Business Users to Understand BusinessRequirements

Determined Software Requirements

MS SSIS (ETL Tool)

MS SSAS (Analytic Cube Tool)

MS SSRS & Excel (Reporting Tools)

SharePoint for Deploying Reports over CompanyIntranet

Designed and Developed zBis Data Warehouse


4/31

Z BIS = What We Will Deliver

This project team will deliver the following:

Consolidated reporting for Carl Zeiss VisionNorth America

Reporting that is consistent and from onedata warehouse

Reporting that is easy to use and easy toaccess

X

X

X

Toolset will be flexible and able to grow andchange with our business

Phase I rock solid download from ERP/Manf Providing ability to review lab informationas a lab network not individual silos withaccurate reporting across all products andservices

We will deliver the best product possible based on the information wecan place in our data warehouse!

X

X


5/31

Reporting from cubes off source systems only

No data warehouse

Disparate data systems with different results from

each

Most systems not balanced to GL

Reporting for each business unit only

No reporting across all business units


6/31

Transactional Cube of Approach

ETL Loads

Data MartFinance

Data MartInventory

Data MartSales & Marketing

Sales ReportsSales Queries

Corporate

Other Reports

Download

ODS/Staging

Operational Data Store

ETL Load

ETL Load

ERP Manufacturing Other


7/31

Aggregated

Data Mart

TBD

Finance

Data Mart

Inventory

Data MartSales

Data Mart

PerformancePoint Server

BI Tools/Analytics

SQL Analytics

Server (SSAS)

SQL

Reporting

Server

Static Reports

ActiveReports

Excel

SharePoint

ERP

ODS/StagingOperational Data Store

Data Warehouse

Manufacturing SW Other Data Sources

oa

ETL Load (SSIS)

ETL Load (SSIS)


8/31

Introduction to Data Warehousing

What is a Data Warehouse System

Why a Data Warehouse Vs. Cubes on Source Systems

Star Schema Vs. Transactional Data Warehouses

Star Schemas ease of system integrating Star Schemas provide substantial performance gains

Star Schemas hierarchy capabilities or Drill Down

Capabilities Ralph Kimball Developed Current Industry Standards for Star

Schema Dimensions and Facts


9/31

Data Warehouse Project Lifecycle

ProjectPlanning

Business

Requirement

Definition

Technical

Architecture

Design

Dimensional

Modeling

Product

Selection &

Installation

Physical

Design

Data Staging

Design &

Development

Deployment

Maintenance

Testing

ETL &

DW/DM

Specifications

Development

Testing

Project Management


10/31

4 + 1 StepsDimensional Design Process

Ralph Kimballs Process for Developing Star Schemas

1. Determine Business Process

Model business Processes

Each Process will determine 1 or more Facts

Design DW by Business Process Not Business Unit2. Identify the Grain of the Fact

What does 1 row in Fact table represent

Transactional or Summary

3. Design the DW Dimensions4. Design the DW Facts

+1 Determine Hierarchies


11/31

Business Driven vs. Data Driven

Design DW/BI System via Business Process

Develop DW/BI System via Data from Source Systems

ro e a a as ear y as poss e

Understand data and design DW using existing data

Design & Develop using both Business Process and available

Data if possible


12/31

Understanding Your Business Identify key business sponsors for DW project

Use Corporate Org Chart

Setu initial interviews with ke s onsors

Develop Business Process diagrams

Develop high level Use Case Diagrams

Determine Business Hierarchies


13/31

The Business Executive Interview What are the objectives of your organization?

What Business goals do you want to accomplish with the

development of zBis data warehouse System?

How do you measure success? How do you know you are doing

well? How often do you measure your corporate performance?

What are your key business issues that you are trying to solvefrom the zBis system? If these issues are not justified what is the

impact to your department and organization?


14/31

The Business Executive Interview How do you identify problems or know when you might be

headed for trouble?

How do you spot exceptions in your business? Whatopportunities exist to dramatically impact your business basedon improved access to information? What is the financial

If you could.., What would it mean to your business?

What is your vision to better leverage information within yourorganization?

How do you anticipate that your staff will interact directly withthis information?


15/31

The Business Manager Interview

What are the objectives of your department?

What are you trying to accomplish? How would do you go

about achieving your objectives?

W at are your success metr cs

How do you know you are doing well?

How often do you measure your department/team?

How do you anticipate that your staff will interact directly with

this information?


16/31

Business Process Diagrams

Understand Business Requirements for buildingDW/BI system.

Defines the Measures and Dimensions for data

warehouse


17/31

Determine Hierarchies

Customer Hierarchies Sales Channels Distribution Channels

Business Channels

ustomer anne s

Product Divisions

Sales Organizations

Sales Office

Buy Groups/Directly Purchase


18/31

Determine Hierarchies Product Hierarchy

Manufacturer

Brand

Product T e Each roduct t e had own Hierarch

Lens Service

Equipment

etc Design

Make/Model


19/31

Determine Hierarchies

Geo Hierarchy Sales Division

Sales Region

Sales Territory


20/31

Conformed Dimensions

Standardized dimensions across data warehouse

Dimensions are associated with multiple business

processes

Conformed Dimensions are shared and consistent

across fact tables


21/31

Use Data Warehouse BUS Matrix Use Data Warehouse BUS Matrix for

Understanding & mapping of Business Processes and

Dimensions

Team & Management Communications

Understand Business Process unions across the enterprise


22/31

Data Warehouse BUS MatrixDate Company Customer Product Geo Dist Ctr Promo

Company

SalesX X X X X X

Customer X X X X X X

ProductCost

X X X X X X X

Company

InventoryX X X

Dist CtrInventory X X X


23/31

Develop Dimensional Schema


24/31

Slow Changing Dimensions

Type 1 Overwrite existing Dimension Row

Use when dont need to keep history data row Can be used to correct bad data

Type 2 Create a new Dimension Row

Use date and/or active non-active fields to identify currentand inactive data rows

Type 3 Keep old and add new attributes in Dimension Row

Allow Alternate realities to exist simultaneously in one

Dimension Row

Slow Changing Dimensions are handled in the ETL


25/31

Type of Dimensions

Mini-Dimension

Junk Dimensions

Outrigger Dimensions

Lookup tables


26/31

Type of Facts

Transaction Fact Tables

Snapshot Fact Tables

Accumulating Snapshot Fact Tables

Consolidated or Aggregated Fact Tables


27/31

Bridge Tables


28/31

Bridge Tables


29/31

Recommended Reading list The Data Warehouse Toolkit: The Complete Guide to Dimensional

Modeling (Second Edition) by Ralph Kimball and Margy Ross

The MicrosoftData Warehouse Toolkit: With SQL Server2005 and the

MicrosoftBusiness Intelligence Toolset by Joy Mundy, Warren

Building a Data Warehouse: With Examples in SQL Server (Expert's Voice)

by Vincent Rainardi

The Data Warehouse Lifecycle Toolkit by Ralph Kimball, Margy Ross,

Warren Thornthwaite, and Joy Mundy

The Data Warehouse ETL Toolkit: Practical Techniques for Extracting,

Cleanin by Ralph Kimball and Joe Caserta


30/31

Elizabeth Diamond

Senior Data Warehouse Architect

DesignMindEmeryville, CA

www.designmind.com


31/31

www.bayareasql.org

To attend our meetings or inquire about speaking opportunities,please contact:

Mark Ginnebaugh, User Group Leader [email protected]

designmind data warehouse

Documents