wherescape, the pioneer in data warehouse automation software
TRANSCRIPT
THE PIONEER INDATA WAREHOUSE
AUTOMATION SOFTWARE
Datavaultautomationconference – Utrecht– 6October2011
Whythispresentation?
COMPANY BACKGROUND• Founded in 2000 in Auckland, New Zealand
by data warehouse experts• 120+ Employees• US – Global Head Office, Portland• UK – European Head Office, Reading• NZ – Development, Auckland
Supported Targets:
USA:Wells Fargo BankNIKEGE AviationIPC SubwayJuniper NetworksNordstromDelta Credit UnionMed Assets
EuropeTescoVodafoneAONXerox HSBCMAN InvestmentsNetwork Rail Sainsbury’sZurich Insurance Volkswagen
Asia PacificMGM MacauAbano InternationalSingtel OptusEricsson VietnamTelstra AUReserve Bank of AustraliaMinistry of DefenceWestfieldQBE InsuranceEnergy AustraliaAstraZenecaAir New ZealandTelecomVodafoneXero
Vodafone Qatar
ABSA Barclays Bank
GLOBAL CUSTOMER BASE700+ WORLDWIDE
SOME OF OUR CUSTOMERS
WHAT WE DO
Profiles, Plans, DesignsRight.
Builds, Documents, Automates Now.
VALUE PROPOSITION
• 1st integrated development environment (IDE) that manages the entire data warehouse lifecycle (Plan & Build)
• Builds data warehouses 10-100 times faster than a traditional approaches
• Data driven approach de-risks projects and automatically applies best practice whilst auto documenting…
DATA WAREHOUSE AUTOMATION
REQUIREMENTS
DW framework
Profile
Logical Model
Physical Model
DB Architecture
Storage Mgmt
Index Mgmt
OLAP Design
ETL Mapping
ETL Dev
Version Control
Workflow
Deployment
Maintenance
Word/Excel
Mainly in-house solutions
Informatica
Microstrategy
IBM Clear Case
Trillium
AbInitioDataStage
TOAD
PowerDesigner
Enterprise Architect
Cognos
JIRA
SSMS
SSIS
SSAS
SVN
IBM Clear Quest
Change tool
Change tool
Change tool
Change tool
Change tool
Change tool
Informatica
ERwin
DB Management Tools
DataStage
Documentation
TRADITIONAL APPROACH
Over complicatedInefficient
DW Framework
Profile
Physical Model
DB Architecture
Storage Mgmt
Index Mgmt
OLAP Design
ETL Mapping
ETL Dev
Version Control
Workflow
Deployment
Maintenance
Logical Model
Documentation
METADATA
WhereScape RED
WhereScape 3D
THE WHERESCAPE WAY
SimplificationAutomation
MANAGING AN EDW/BI ENVIRONMENTIS CHALLENGING
• Too many development tools & skills
• Siloed with a high cost of change
MANAGING AN EDW/BI ENVIRONMENTJUST GOT EASIER
• One tool, one skillset, low cost of change
• Non-proprietary & Non-disruptive
SQL
• I set-based SQL• Scale with your RDBMS or Big Data platform
• Easy migration path to other technologies
• You’re in control, no black boxes
• Don’t we all talk SQL?
DATA WAREHOUSE AUTOMATION
😀
PROCESS
• Different activities warrant different approaches• Model-driven development vs.• Data-driven development
• Bring the agile back in our profession• Break down the barriers between profiles• Don’t limit yourself to “coding only” activities
DATA WAREHOUSE AUTOMATION
“A 3 year project has now been completed in 6 months thanks to WhereScape and agile development.”
Discovery—Rapidlycreateworkingsolutions,whichareputinfrontofbusinessstakeholdersforreviewinsprintsnolongerthan30days.
Production Build—Becomesanassemblyprocess,closelyfollowingstandardsandcommoditizedimplementationpatterns
Redlineshowscurrentdate
THE DATA QUADRANT MODELRONALD D. DAMHOF
DevelopmentStyle
Systematic
Opportunistic
I II
III IV
Research,Innovation&
Design
“ShadowIT,Incubation,
Ad-hoc,Onceoff”
Push/Supply/Source driven Pull/Demand/Productdriven
DataPush/PullPoint
ContextFacts
SCOPE
• End-to-end• Full life cycle -> identify your bottlenecks
• Analysis: source profiling• DevOps / CI• Test Automation• Documentation
• Any modeling style?
DATA WAREHOUSE AUTOMATION
WHERESCAPE ARCHITECTURE
CLASSIC KIMBALLStar Schema
HISTORICAL ODS + CLASSIC KIMBALLData Store Star Schema
NORMALIZED EDW3NF Star Schema
RAW + BUSINESS DATAVAULTData Vault Data Vault
RAW DATAVAULT + STAR SCHEMAData Vault Star Schema
DATA LAKE + STAR SCHEMAStar Schema
COMPETITION
• Very few automation software packages• Traditional ETL suites• Set of templates• But typical: PPP
DATA WAREHOUSE AUTOMATION
• Teradata is an important platform
