The Big Green Thingy – A Case Study in Data Warehousing
Allison Lobato, DBAEnterprise Data Warehouse
Department of Technology ServicesDenver Public SchoolsDenver, Colorado
Agenda
DPS and the CIF Current Environment Overview
Staffing Hardware Architecture Software Architecture
Design, Development and Deployment Architecture
Installation and Configuration Notes The Big Green Thingy Overview Conclusion Questions
My Bio
Allison LobatoDatabase Administrator – 21 years w/
DPS in IT Applications Development and DBA Support
Standard Survey
Who are you? DBAs Data Warehouse Designers Data Warehouse Architects Managers
Experience Warehouse Builder (OWB), Discoverer, Designer, Reports Oracle 9iAS or Portal Data Warehousing
Less than 1 yr?1-3 yrs?Over 3 years?
DPS, Data Warehousing and the CIF
DPS has no shortage of data – inconsistent, disjointed and disparate
DPS knows value and importance of getting our hands around the data
Using the Corporate Information Factory (CIF) conceptual architecture Developed by Bill Inmon and Claudia
Imhoff
DPS District Information Factory
DPS, Data Warehousing and the CIF
Current EnvironmentCIF is a long term architecture strategy for
DPS (a.k.a. District Information Factory-DIF)Pilot mission:
Prove the value to get funding!Deliver current student profile information
Using an Operational Data Store (ODS)Student profile subject area
Target audience 1 high school1 middle school1 elementary school1 administrative department
DPS, Data Warehousing and the CIF
Current Environment (continued)Enterprise Data Warehouse is an unfunded
projectUsing existing IT resources
Hardware – existing servers and disk spaceStaff
3 full-time (re-allocated)Data warehouse architect (Supervisor)DBAETL programmer
2 part-time (shared)DBA & iAS administrator
Hardware Architecture
Three-tiered approachEnd User Layer (tier 1)
Workstations
Middle Layer (tier 2)Web and application services
Database Layer (tier 3) Database and storage services
Hardware Architecture
The Workstations
For developers’ machines: robust PCs are a must. They need lots of memory (>512 Mb)Fast processors
End users machines: they are easierBrowser capable desktop running a
supported browser version Macs and PCs
Middle Tier
9iAS – Web and Application ServicesProduction Server
Dell PowerEdge 2650– 4 Gb Memory2 – 36 Gb Mirrored Hard Drives1 – 36 Gb Hot Spare Hard DriveWindows 2000 Advance Server with SP3
Development ServerDell PowerEdge 2500 – 4 Gb Memory2 – 18 Gb Mirrored Hard DrivesWindows 2000 Advance Server with SP3
Database Servers and Storage Devices
Database ServersHP’s RP8400 class server
Production ODS (64-bit)HP’s RP7410 class server
Production repositories & development ODSHP-UX 11.11
Storage DeviceEMC Symmetric 8430 Using less than 150GB currently
Estimated need over 400GB
Software Architecture
End-to-end Oracle solution (Oracle 9iDS , 9iAS & RDBMS) on all 3 tiers (workstation, web, database) RDBMS Designer Warehouse Builder (OWB) Discoverer Reports Portal Single Sign On (SSO) Oracle Internet Directory (OID) Enterprise Manager (OEM and WebOEM) Workflow (Job scheduling and monitoring)
Version compatibility was key (and constantly changing)
Oracle Software Tool Versions
Tool Name ClientVersion
RepositoryVersion
DatabaseVersion
Server Type
Designer 9.0.2.3 9.0.2.90.10 9.2.0.3 HP RP7410
Warehouse Builder 9.0.4.8.21 9.0.4.0.27 9.2.0.3 HP RP7410
Reports 9.0.2.0.1 n/a n/a Dell PE2650
Discoverer 9.0.2.53.09 9.0.2.53.09 9.2.0.3 HP RP7410
Enterprise Manager n/a * 9.0.1.3.1 Dell PE2650
Portal n/a 9.0.2.2.22 9.0.1.3.1 Dell PE2650
SSO n/a * 9.0.1.3.1 Dell PE2650
OID n/a * 9.0.1.3.1 Dell PE2650
Design Phase Tasks
Create logical and physical data models using DesignerStaging area for source dataOperational Data Store (ODS)
Create transformation routines using OWBGUI ETL (Extract,Transform,Load) tool
Diagram inputs, outputs, and transformation routines for moving data into ODS from source
Generates PL/SQL code
Development Phase Tasks
Create the databases Using OEM or SQL*Plus
Deploy the data modelsUsing DDL from Designer
Deploy and test the transformations Using OWB
Development Phase Tasks
Install and configure 9iAS and the Reports server
Create the access portal/interface Using Oracle Portal
Create static, parameter driven reports Using Oracle Reports1st cut was generated from Designer
Development Phase Tasks
Create the various portlet providers Using Oracle Portal
Create the Business Areas (End User Layer)Using Discoverer Administration
Create the dynamic business intelligence interface Using Discoverer End User Edition
(workstation) or Discoverer Plus (web)
Development Phase Tasks
Create the Discoverer public connections Within 9iAS environmentUsing Web OEM
Deploy Discoverer portlets (worksheet & workbook)Using Oracle Portal
Develop additional PL/SQL scripts For automating the data loads
Develop Workflow Process Flows Using OWB vs. Workflow Builder
Discoverer Portlets
Management Software
OEM (Oracle Enterprise Manager)Manage the database – storage, users,
collecting stats, etc.Web OEM
Manages the 9iAS componentsOracle Workflow
Schedules and monitors the ETL mappings and load routines
Oracle Tool Repositories
REPOPROD database (meta data)Oracle DesignerOracle Warehouse BuilderOracle Enterprise Manager
ODS databaseOracle Discoverer (EUL)OWB runtimeOWF runtime
Oracle Tool Repositories
iAS databaseOracle PortalWeb OEMSSO (Single Sign On)OID (Oracle Internet Directory)
Repository Schema Locations
Installation and Configuration
Workstation notesCaution – numerous Oracle homesMemory, memory and more memory
9iAS (all components on the same server)Required to do the install 3 times
InfrastructureApplications Tools (if using the Portal Developer’s Kit-PDK)
Installation and Configuration
9iAS notes (continued)Infrastructure installation
9i database (IASDB) automatically builtVersion 9.0.1.3.1
Application installationReports server configuration will fail 1st time
Series of services started after each installDocumentation is “shaky” for all 9iAS
installation proceduresBe prepared to delete and start overDPS submitted to IOUG a 9iAS Rel 2 Survival Guide
for Windows for publication in SELECT magazine.
Installation and Configuration
Database notesAlways plan carefully
Check for operating system patches firstTest all components (interfaces, db links,
backups, etc…)
If installing under the same OS user-id All other Oracle databases must be stopped
due to the shared java components
Get a test server (if you can!)
The Big Green Thingy – Design Phase Components
The Big Green Thingy -Development Phase Components
The Big Green Thingy -Deployment Phase Components
Finally –The Big Green Thingy
Conclusion
Complex Setup Lot of work, research, trial and errorLimited published documentation
ResultThe foundation of our architecture is up and
runningIntegrated, single vendor solution
Will support our efforts to build our District Information Factory
Hopefully this information will add some clarity and make life easier when building the components of your own data warehouse
Contact Information
Allison Lobato• [email protected]