best practices for multi-dimensional design using cognos 8...
TRANSCRIPT
Copyright © Boeing 2007. All rights reserved
Best Practices for Multi-Dimensional Design using Cognos 8 Framework Manager Bruce ZornesThe Boeing CompanyBusiness Intelligence & Competency Center
Copyright © Boeing 2007. All rights reserved
Best Practice: Knowledge and Experience
Use Best Practice to reduce cost and produce quality
Best Practice means
Knowledge about the solution
Theory is OK if produces results
Use Patterns – re-apply Knowledge
Experience with the tools
How to, what to do, when to do it
People have Experience and Knowledge
Copyright © Boeing 2007. All rights reserved
Best Practice: Use Design Principals
Design quality in rather than Testing it in
Framework Manager model is great place to implement quality design
Understand the business case and drivers
Understand the data schema - 3NF, star, recursive
Understand the technology for optimization
Teaming – include DBA’s and Server Admins
Common model and keep it simple
Understand your customer – reports team too
Copyright © Boeing 2007. All rights reserved
Best practices: Measures for success - Project
Success criteria checklist – expectations, funding, resources
Complete solution – phase the release(s)
Our case: 11 business units, 20 cubes, one model, 4 rels
Follow a process – Dev Test Prod, CM, docs, SQA
Design reviews and open communication
Customer acceptance – requirements met
End-user acceptance – navigation and UI, portals
Cost and schedules – hidden cost, unplanned work
Reports performance – team collaboration
Support and operations team – training, procedures
Copyright © Boeing 2007. All rights reserved
Customer Requirements: Business case
How to produce quality – criteria for design?
Design choices:
Star schema
Narrow fact tables
Familiar navigation
Fast query response rqmt
Minimize load time ETL
Reporting mart
Three layer design
Nice theory
Too slow for large data?
Copyright © Boeing 2007. All rights reserved
Best Practice: Framework Model Design
Framework Model Three layer design
• [Physical Database Layer]
SQL data source objects
• [Business Model Layer]
Model objects and building blocks
Business rules and data logic
• [Dimensional Presentation Layer]
Multi-dimensional views and format
Copyright © Boeing 2007. All rights reserved
Data Design: Parent-Child to Denormal table
Denormalizedtable for Reporting
PARENT- CHILD tree
Note: Non-proprietary ‘scrubbed’ test data
Copyright © Boeing 2007. All rights reserved
Framework model: Construct model objects
Building blocks: parent-child Query Subjects (QS) and Query Sets
Namespace scope
Folders organize
Naming convention
User Naming for Reports
Shortcut to physical
Explicit relationship
Generation Specific QS
Placeholder Parents QS
QS building block
Leaf members
Query Set Union
Copyright © Boeing 2007. All rights reserved
Framework Model: Core Design Elements
Conformed star schema (multiple fact tables)
Parent-child to denormal dimension tables
Unbalanced hierarchies
Building block Query Subjects (QS)
Parent place-holders at each generation
Unions via Querysets: QSParent to QSGen
Create leaf member level – coalesce(case when)
Link dimensions to facts at leaf level
Narrow fact table - one key per dim
Create dimensional views from building blocks
Copyright © Boeing 2007. All rights reserved
Framework model: with SQL Generation
Building blocks: Generate Top-of-Tree SQL
with E123_SCENARIO21 as
(select E123_SCENARIO.ESB_FKID,E123_SCENARIO.ESB_PARENT,E123_SCENARIO.ESB_CHILD,E123_SCENARIO.ALIAS1,E123_SCENARIO.DATASTORE,E123_SCENARIO.GENERATION
from "FabFBI Datasource"..AFAB.E123_SCENARIO),
ScenarioGen2 as (select
E123_SCENARIO21.ESB_FKID as ESB_FKID,E123_SCENARIO21.ESB_PARENT as PARENT,E123_SCENARIO21.ESB_CHILD as CHILD,E123_SCENARIO21.ALIAS1 as ALIAS1,E123_SCENARIO21.DATASTORE as DATASTORE,E123_SCENARIO21.GENERATION as GENERATION
from E123_SCENARIO21
where (E123_SCENARIO21.GENERATION in ('2'))
),
Primary reference to PARENT-CHILD physical table
Schema owner
Top-of-Tree constraint
Cognos Connection
Copyright © Boeing 2007. All rights reserved
Framework model: SQL Magic for Placeholders
Building blocks: QuerySet union with placeholders SQLScenarioGen23Union517(ESB_FKID,
PARENT,CHILD,ALIAS1,DATASTORE,GENERATION) as
((select E123_SCENARIO15.ESB_FKID as ESB_FKID,E123_SCENARIO15.ESB_CHILD as PARENT,cast( null as varchar (80)) as CHILD,cast(null as varchar (80)) as ALIAS1,cast(null as varchar (80)) as DATASTORE,E123_SCENARIO15.GENERATION as GENERATION
from E123_SCENARIO15
where (E123_SCENARIO15.GENERATION in ('2'))
) UNION(select
E123_SCENARIO16.ESB_FKID as ESB_FKID,E123_SCENARIO16.ESB_PARENT as PARENT,E123_SCENARIO16.ESB_CHILD as CHILD,E123_SCENARIO16.ALIAS1 as ALIAS1,E123_SCENARIO16.DATASTORE as DATASTORE,E123_SCENARIO16.GENERATION as GENERATION
from E123_SCENARIO16
where (E123_SCENARIO16.GENERATION in ('3'))
) ),
Promote CHILD as PARENT
Placeholder Parents at Gen 2
UNION to
Members at Gen 3
‘NULL’ CHILD‘NULL’types coerced
Copyright © Boeing 2007. All rights reserved
Framework model: SQL tricks for Leaf members
Building blocks: coalesce(case(tree)) as LEAF_MEMBER
ScenarioLeaf as (select
coalesce(case when (ScenarioGen34Union18.CHILD is NULL) then null
else ScenarioGen34Union18.ESB_FKID end,case when (ScenarioGen23Union19.CHILD is NULL) then null
else ScenarioGen23Union19.ESB_FKID end ,case when (ScenarioGen2.CHILD is NULL) then null
else ScenarioGen2.ESB_FKID end ) as LEAF_FKID,
coalesce(case when (ScenarioGen34Union18.CHILD is NULL) then null
else ScenarioGen34Union18.PARENT end,case when (ScenarioGen23Union19.CHILD is NULL) then null
else ScenarioGen23Union19.PARENT end ,case when (ScenarioGen2.CHILD is NULL) then null
else ScenarioGen2.PARENT end ) as LEAF_PARENT,
coalesce(ScenarioGen34Union18.CHILD,ScenarioGen23Union19.CHILD,ScenarioGen2.CHILD
) as LEAF_CHILD,
….
Walk trees in reverse
3 2 1
Case switch to NOT promote placeholders
Build leaf members
Copyright © Boeing 2007. All rights reserved
Framework model: Construct leaf and case logic
Building blocks: final assembly – leaf model objects
Conditional members
Placeholders promote members
No cross joins allowed!
Model objects need relationships
Leaf links to fact tables
Coalesce Case switch
Coalesce is tree walker
Scan right to first non-null
Copyright © Boeing 2007. All rights reserved
Framework model: SQL loop joins
Building blocks: SQL loop joins w/placeholder filter
ScenarioLeaf as ….from
ScenarioGen2left outer join
ScenarioGen23Union517 ScenarioGen23Union19on (ScenarioGen2.CHILD = ScenarioGen23Union19.PARENT)left outer join
ScenarioGen34Union214 ScenarioGen34Union18on (ScenarioGen23Union19.CHILD = ScenarioGen34Union18.PARENT)
where (not (coalesce(ScenarioGen34Union18.CHILD
,ScenarioGen23Union19.CHILD,ScenarioGen2.CHILD)is NULL)) )
Join-Join causes repeating output
Put ‘em in & take ‘em out
Seems like extra work for unbalanced
My ‘tricky’SQL?
(really just ANSI sqlF041-03F041-05)
Copyright © Boeing 2007. All rights reserved
Framework model: Determinants fix multi-grain
Building blocks: use determinants on leaf data groups
Leaf model objects are generation specific
Naming suffix
1 2 3…
CHILD1 is top-of-tree
Uniqueness based on composite business keys
Grouped by generation
(nice to have: determinant filters)
Copyright © Boeing 2007. All rights reserved
Framework model: Dimensional construction
Presentation Layer: connecting the piecesDimensional model construction from Model objects
Hierarchy name
Member levels
Unique leaf level
Set businesskeyand caption
Build presentation layer from BP model building blocks
Drag drop & test
Copyright © Boeing 2007. All rights reserved
Framework model: Test in Query Studio
Published package: validate unbalanced structure
Note: Non-proprietary ‘scrubbed’ test data
Suppress first two layers in final publish
All reports connect to presentation layer
First look at dimensional output
Unbalanced member names promoted to leaf_level
Leaf members connect to facts
Copyright © Boeing 2007. All rights reserved
Framework model: Design challenges met
Interface to legacy system OLAP data cubes
Iterative design – not all data structure understood
ETL expensive and difficult
Minimize business rules in ETL
Narrow fact tables - faster than wide
Link all tables via SID (business keys)
High volume data (200 Million rows) partitioned
Users expect 2 - 4 seconds click report response
Copyright © Boeing 2007. All rights reserved
Framework model: Test in Analysis Studio
Verify to legacy system OLAP data cubes
Note: Non-proprietary ‘scrubbed’ test data - all aggregate values are row counts
Ready to build reports
Do aggregate values match
Mixed grains
User familiar names
Is it fast
In AS 10-30 secs
In QS 3-5 secs
Copyright © Boeing 2007. All rights reserved
Example 1 of Unbalanced Hierarchy
Unbalanced Dimensions: promotion of members to leaf level
Facts link to leaf member level
Fringe Dollar Adjustments
Regular Dollars
Measures dimension
Link by Root_child
Data Rollups
Mixed-grain data
Note: Non-proprietary ‘scrubbed’ test data
Copyright © Boeing 2007. All rights reserved
Example 2 of Unbalanced Hierarchy - SHARED
Unbalanced Dimensions: complex ‘network’ members
‘SHARED’ dimension members have root_child ‘Cloned’parents and child data
‘Cloned’parent
‘Cloned’ child data from root_child
Attached to different ancestor
Note: Non-proprietary ‘scrubbed’ test data
Copyright © Boeing 2007. All rights reserved
Design strategy: Model meets the needs
Who: model end-user vs. reports end-user
Adapt to Teams, Roles and Org maturityPriorities may change
Key success drivers
Compromise and validate
Rely on principals
Three layer design -only two
Developer model not end-user
3NF structure
Denormalize in rdbms
Copyright © Boeing 2007. All rights reserved
Best practices: Measures for success – check!
So how’d we do? Success criteria / performance indicatorsFirst release due: July 2006, Delivered first reports: March 2007
Followed process: Dev Test Prod, docs
Tech reviews: hold daily status with PMs
Customer and End-user acceptance: improved their data qualityReally appreciate common business model and Cognos rpts
Cost and schedules: re-planned to meet realities
Performance: designed-in, mat views added, DBA involved
Support and operations: OJT, handover challenges,teaming improvements, working together, find-a-way
Users: want more BI reports and faster deployments
Copyright © Boeing 2007. All rights reserved
Evaluation Forms
Each completed evaluation qualifies you to win one of five $100 American Express gift certificates given away daily.
Complete evaluations for every session you attendand qualify to win an additional $500! An overall conference survey will be available at Cognos Central on Thursday morning, and will also be emailed to you. Complete the survey by May 25, 2007, and you'll qualify to win $500!