avoid bi catastrophe

AVOIDING BI CATASTROPHE

Technology Focus on Backend BI

LeapFrogBI – Making Agile Data Warehousing Possible

Paul B. Felix512-748-0216paul@leapfrogbi.com

1998 201420062002 2010

“There is a common perception that the failure rate of data warehousing projects is 70 to 80 percent…”

“…one study reported a 90 percent failure rate.”

“(From now) through 2007, more than 50 percent of data warehouse projects will have limited acceptance, or will be outright failures…”

“Between 70 to 80 percent of corporate business intelligence projects fail, according to research by analyst firm Gartner.”

“During the panel session vendors were asked to estimate the failure rate of analytic projects. They generally agreed that more than 70 percent failed to meet expectations.”

“A staggering 60 percent of BI projects end in abandonment or failure…”

* - Reference Available by Request

?

TODAY’S AGENDA

Success & Failure

Agile ETL Design Pattern

15 Minute Data Mart (Video)

Open Discussion

~ 45 Minutes

SUCCESS & FAILURE

Defining the goal and the most common causes of failure.

A SUCCESSFUL BI SOLUTION

Add ValueValue = Revenue - Costs

1. Improve Decision Making by Providing Relevant Information

2. Respond Efficiently to a Changing Business Environment

SUCCESSFUL BACKEND BI SOLUTION

Data Structure that Supports Requirements

1. Acceptable Performance

2. Easy to Navigate

3. Responsive to Changing Requirements

4. Meet Data Persistence Needs

5. Resolve Data Quality Issues & Apply Business Rules

6. Consolidated Model (One Version of the Truth)

7. Resilient, Accurate, Timely, and Scalable

8. Controlled Ongoing Support/Maintenance Costs

BACKEND BI FAILURE

#1 - Failure to Plan

John Wooden – UCLA Coach (1910 – 2010)

"If you don’t have time to do it right, when will you have time to do it over?"

• Resource gaps – Technical & non-technical

• Cowboy coding – No standards• Building without requirements• No support & maintenance plan• No MDM or data quality plan• The short term solutions

BACKEND BI FAILURE

#2 – Low Level Development

Benjamin Disraeli – British Politician(1804 – 1881)

"There can be economy only where there is efficiency.”

• Expensive; recreate the wheel• Tightly coupled dependencies• Difficult to transfer knowledge• Lower usually equals slower• Difficult to extend; loss due to rework• Developer fatigue

BACKEND BI FAILURE

#3 – Flawed Design Patterns

Mark Twain – Author(1835 - 1910)

"I have never let my schooling interfere with my education.”

• Failure to consolidate• Reliance on natural keys• No separation between stage & load• Focus on optimization before design• Dimensions not conformed• No bus architecture

AGILE ETL DESIGN PATTERN

An architecture that supports agile data warehousing.

AGILE ETL MANIFESTO

Strive for Simplicity

Plan for Change

Default to the Foundation Pattern

Facilitate Custom Needs

Loosely Couple Dependencies

AGILE ETL DESIGN PATTERN

AGILE ETL DESIGN PATTERN

AGILE ETL DESIGN PATTERN

AGILE ETL DESIGN PATTERN

AGILE ETL DESIGN PATTERN

AGILE ETL DESIGN PATTERN

AGILE ETL DESIGN PATTERN

AGILE ETL DESIGN PATTERN

15 MINUTE DATA MART

LeapFrogBI platform demonstration.

METADATA DRIVEN ETL

ID Effective Expired Current CustomerID Name Age City1 1/1/1900 5/12/2010 0 4562 Jack 56 Austin2 5/13/2010 2/4/2013 0 4562 Jack 56 Dallas3 2/5/2013 1/1/9999 1 4562 Jack 56 Houston

Data Warehouse

Click Image to Download Video

QUICK RECAP

BI Success = Net Value Add

Avoid Common Causes of Failure

Select an Agile ETL Design Pattern

Consider a Metadata Driven Approach

OPEN DISCUSSION

Question, Comments, and Experiences.

LeapFrogBI – Making Agile Data Warehousing Possible