confidential | june 2003 | © 2003 ibm corporation transforming data into business intelligence - an...

Confidential | June 2003 | © 2003 IBM Corporation

Transforming Data into Business Intelligence - An Overview

State of Louisiana Department of Transportation

State of Louisiana | Confidential © 2003 IBM Corporation

Discussion Topics

BI Overview

Questions to consider

BI Case Studies

Q & A


Business Intelligence Overview


Aristotle Onassis said:

"The secret of business is to know something that nobody else knows.""The secret of business is to know something that nobody else knows."


Government organizations today are faced with an almost overwhelming set of questions. Legislation and constituents are driving a focus on accountability for dollars spent and results

What is the effectiveness of various government programs?

What is the cost effectiveness of various programs?

What is the breakdown of the client base by program, service, demographic makeup?

What is the adherence of the programs to state and federal regulations?

What are the administrative costs of running state, county, and community based programs?

...


Much of the data to answer these questions exists and more is available but there is a growing gap in the ability of organizations to analyze it

Knowledge Gap

Information Availability

AnalyticalPotential

1960 1970 1980 1990 2000

Dat

a

Source: Gartner Group


Challenges to accessing data to answer questions

Multiple sources of data and reporting systems

Difficulty in locating and addressing information

Unreliable information

Pressures to lower costs and improve services

Information requests take days or weeks to fulfill

Re-keyed data ensuring errors creep into results


The needed data exists in "silo" applications

Human Services - TANF, FS, Medicaid

DMVCriminal Justice

...

Labor - ES, UI, Workman's

Compensation

DOT - Workplan, Roadway Inventory, Transport, Fixed Assets

Treasury - Tax

Federal Systems- SSA, HCFA, IRS,

HHS


The solution is to derive business intelligence from relevant consolidated data

Cross Agency Client data

Population demographics

Federal program information

Service delivery encounter data

Contract data


Breakout Discussion

Questions for discussion:► As a business unit, what challenges you the most in your effort to make

informed decisions?

► What roadblocks do you have to cross in order to get the information that you need to do your job?


What is business intelligence?

Business Intelligence means using your data assets to make better business decisions.

Business intelligence involves the gathering, management, and analysis of data for the purpose of turning that data into useful information which is then used to improve decision making. Organizations can then make more strategic decisions about how to administer clients and programs. These practices can also reduce operating costs through more effective financial analysis, risk management, and fraud management.


Business Intelligence represents a fundamental shift in the purpose, objective and use of information

SNA Knowledge Management

Microcomputers

CASEInformation Engineering

ObjectTechnology

Client Server Technology Internet Technology

360Architecture

Relational Databases

e.content

Accounting SystemsBasic “sort and report” administrative and accounting systems to automate financial reporting processes

1

Operations SystemsIntegrated operations and planning systems providing centralized business controls and enterprise transaction processing

2

Intelligence SystemsDiscovery and analysis system used to enhance strategic and tactical decisions

3

Accounting Systems(1950s - 1970s)

Operations Systems(1970s - 1990s)

Intelligence Systems(1990s Forward)


Business Intelligence Solutions are...

"... solutions which enable an end-user to quickly and easily analyze organizational data to make intelligent decisions..."

"... systems that put information in the hands of end-users..."

"...the combination of data warehouse and end-user data access and analysis tools..."

"...an investment in information technology that can assist governments in managing limited fiscal resources..."


Business Intelligence solutions start with data warehouses and data marts

Analysis Complexity & Value

Statistical

Multidimensional

Data Mining

Optimization

Stage 1 Stage 2 Stage 3 Stage 4 Stage 5

Data MartData Warehouse

Discovery

Verification


Industry leaders agree that data warehousing is a process, a journey and takes a commitment

"Data warehousing is a process, not a place." META Group

"Data Warehousing is a journey, not a destination."

Data Warehousing Institute

"A data warehouse is an architecture, not a product."

Gartner Group

"You can't buy a data warehouse, you have to build it."

Computer World

"House Divided Speech" June 16, 1858

"If we could first know where we are, and whither we are tending, we could better judge what to do, and how to do it." - Abraham Lincoln


What is a data warehousing?

"Data Warehousing is the process whereby organizations extract meaning from their informational assets through the use of special stores called data warehouses."

- Ramon Barquin, The Data Warehouse Institute, Bethesda

"Data Warehousing is the process whereby organizations extract meaning from their informational assets through the use of special stores called data warehouses."

- Ramon Barquin, The Data Warehouse Institute, Bethesda


A data warehouse provides a logical single point where data from operational systems is brought together. It ultimately provides a single source of information needed by end users for decision-making

Reconciliation of dataWith different meaningsFrom many disparate sourcesWith different time dependenciesOnce and only once for each data elementInto a controlled and managed environmentBuilding a complete historical record

Derivation of informationAs needed by end usersFrom a certified and consistent sourceWithout a need to understand individual operational system data sources

Data Warehouse

1 5

68

9

10

11

Operational Systems

Data Marts &

Applications


This conceptual view of an enterprise data warehouse integrates data that is transformed into useful decision-making information

OperationalData

ExternalData

Enterprise Data Model

Transaction Data Meta-data

SummarizationAggregation

RepresentationReview

Specialized Analysis Datamarts / Decision Support Systems

Global Datamart

Data Mining Analysis

Enterprise Data Warehouse


And this is the Required Transformation

Data to Information to Decisions

Query & Reporting

Data Mining

On-line Analytical Processing

Summary and detail

Drill capability

On-line Updates

Batch Feeds

Operational Data Store

Data Warehouse

Data Mart

Data Transformation

Data Synchronization

AccessData Information Management


Levels of Query Analysis Capability

Spring Launch

Daily Inbound Call Volume by Marketing SourceCode

As Of: Sep 21, 1996Sun Mon Tues Wed Thur

9/15 9/16 9/17 9/18 9/19

Total 16 77 153 88 105

Umbrella Ad 0 0% 1 1% 6 4% 3 3% 2 2%

MF Single Proprietary- Local (5500) 0 0% 1 1% 0 0% 0 0% 0 0%- National (2004) 0 0% 1 1% 0 0% 0 0% 0 0%

MF Family Proprietary 0 0% 0 0% 0 0% 0 0% 0 0%- Local (7700) 0 0% 0 0% 0 0% 0 0% 0 0%- National (2005) 0 0% 0 0% 0 0% 0 0% 0 0%

Money Market 0 0% 0 0% 0 0% 1 1%Discount 0 0% 1

Non-Targeted

•

Spring Launch

Daily Inbound Call Volume by Marketing SourceCode

As Of: Sep 21, 1996Sun Mon Tues Wed Thur

9/15 9/16 9/17 9/18 9/19

Total 16 77 153 88 105

Umbrella Ad 0 0% 1 1% 6 4% 3 3% 2 2%

MF Single Proprietary- Local (5500) 0 0% 1 1% 0 0% 0 0% 0 0%- National (2004) 0 0% 1 1% 0 0% 0 0% 0 0%

MF Family Proprietary 0 0% 0 0% 0 0% 0 0% 0 0%- Local (7700) 0 0% 0 0% 0 0% 0 0% 0 0%- National (2005) 0 0% 0 0% 0 0% 0 0% 0 0%

Money Market 0 0% 0 0% 0 0% 1 1%Discount 0 0% 1

Non-Targeted

•

Site

Service

Geography(County)

Costs

Client

Clinician

Multi-Dimensional Analysis Capability

Diangosis

Query & Reporting W ith user friendly query and reporting tools, users can gain access to valuable data to answer basic business questions. Users can also produce customized reports, view standard reports and perform basic analysis.

Multidimensional Analysis/OLAPW ith user-friendly analytical capabilities, users can drill-down & view data by multiple dimensions. Ac-cording to Lee The, Editor of Datamation, "Think of an OLAP data structure as a Rubik's Cube of data that users can twist and twirl in different ways to work through 'what-if' and 'what-happened' scenarios." (May 1995)

glossaryAd-hoc Query- Is a business question that is 'conversational' in nature and not predetermined. From a technical perspective, this results in a query that cannot be determined prior to the moment the query is issued. A query that consists of dynamically constructed SQL, which is usually constructed by desktop-resident query tools.

Multi-dimensional (MDD) Analysis - MDD Analysis is the process of analysis that involves organizing and summarizing data in a multiple number of dimensions. A Multi-dimensional Data Base System captures and presents data as arrays that can be arranged in multiple dimensions. MDD often provides sophisticated visualization and other statistical reporting tools to help identified trends and "outliers" or data which does not fit expected patterns. MDD technology is often used for specialized business needs (areas).

OLAP (On-line Analytical Processing) - OLAP (On Line Analytic Processing) is a loosely defined set of principles that provide a dimensional framework for decision support. The term OLAP also is used to define a confederation of vendors who offer non-relational, proprietary products aimed at decision support.

Payer

MeasuresRevenue

PenetrationOutcomesUtlization

Cost

Which road projects in

Cuhahoga County were late and

overbudget FY 2000?

Show me the road projects by County, by contractor, by costs, by quarter, that were late

and overbudget?


Breakout Discussion

Questions for discussion:► What impact do you think there might be on the work that you perform

and the way you do business now versus in the future?

► What concerns do you have as a result of learning about this type of advanced technology?


There are key critical success factors for the successful implementation of a BI initiative

High-degree of executive sponsorship► Strong executive sponsorship is needed to demonstrate commitment to the project, provide resources, and

reinforce the linkage of the data warehouse to business issues and strategy.

Data warehouse is linked to a well defined business information problem► Without clear linkage to a well defined business problem, the value and benefits of the project to the

business will be difficult to quantify and the project will be very hard to justify.

User involvement throughout the process► Users must be involved in every step of the project to ensure that the effort remains focused on delivering

business value and that the system can and will be utilized by the user community.

An appreciation of the significant effort involved for legacy transformation/data management tasks► The difficulty of extracting, transforming, and combining data from disparate operational systems is one of

the most difficult aspects of the data warehouse project and complicates the testing and validation of the data warehouse system.

Focus on proper database design for effective end user access and performance► Good database design leads to performance that will encourage use.

Start small and plan for continued growth► Developing the warehouse in small increments allows the business to begin receiving benefit from the

project earlier and maintains project momentum.

Commitment to technical and user education and training► Technical education is needed to allow IT staff to maintain, manage, and enhance the data warehouse.

User education is required to develop expertise with the user interface, tools, and the data contents of the warehouse.


IBM has a proven approach for implementing data warehouses

BusinessDiscovery

Detailed Business

Requirements - Pilot

Tools Selection

ConceptualDesign - Pilot

Implementation Plans

Pilot Implementation

& Rollout Planning

Data Organization & Discovery

Data Architecture - Pilot, and Technical

Infrastructure


IBM's experience and data warehouse best practices indicate projects with manageable scope, and then growing the EDW iteratively are most successful

Threefold projects► Business function► Enterprise data modeling► Infrastructure development

Organic growth► Each project expands the existing

infrastructure

Outlook► First deliverable in 4 to 5 months► After 12-24 months:

– Many business deliverables– Modeling & Infrastructure continue

to grow with each release– Multiple releases that deliver value

to the business with each iteration

A complex, interdependent, long-term process

► Differs from most current I/T development efforts

Months 4 18

Infrastructure development

Enterprise modeling

Businessapplication 1

Data warehouse R2

Data warehouse R3

Data warehouse R4

Infrastructure development

Enterprise modeling




Data warehouse R1

8 12


Top "5" Ways to ensure that a Business Intelligence initiative will Fail ...

5. Build a data warehouse to "fix" all of the data quality issues in your operational data.



4. Aggregate data, but keep all of the codes from the operational system -- because "everyone" knows the codes.




3. Try to do everything in a single project.





2. Build a data warehouse and/or perform data mining with no business requirements or business involvement -- also known as the "build it and they will come" method.






1. Build a data warehouse without Executive Sponsorship from the functional side of the house.

2. Build a data warehouse and/or perform data mining with no business requirements or business involvement -- also known as the "build it and they will come" method.





Metadata (data dictionary) - or business rules - is a very important part of a data warehouse. IBM recognizes this and includes metadata strategy in our approach

Metadata is the term used to describe information about data, (e.g. data name, types, business definitions, or business rules)

Metadata enables administrators and business users to better understand the data environment and how to use the data in that environment for analytical purposes

Without metadata, business users are like tourists in a new city without any information or roadmap, and data warehouse administrators are like city planners who have no idea about the size of the city or how fast it is growing

This is VERY important in a fluid and changing environment driven by legislation


Geographic Information Systems (GIS)


Business Intelligence with Spatial Data: GIS Solutions Enable Visualization of Analysis Results

Source of maps: www.esri.com


Many government organizations are faced with questions that, at their root, are geographic in whole or in part

Where are my geographic based assets? (e.g., fire hydrants, facilities, state-maintained roads)

What is happening in the geographic areas under my jurisdiction? (e.g., crime, crashes, demographics, traffic volumes/flow, disease)

Where should I intervene during regular circumstances? (e.g., enforcement, maintenance, construction)

How can I plan for special circumstances? (e.g., emergency situations, extreme weather, future growth)


Components of a GIS Solution

Base Maps

Data Layers

Applications


Data Warehouses/Marts As The Repository for Spatially-Referenced Data Layers: Potential Subject Areas

Roadway Attributes:Railroad CrossingsBridgesHistorical Traffic VolumeGradeFunctional ClassOthers….

Roadway Events:Crashes/Crash AttributesMaintenance ProjectsConstruction ProjectsTraffic Flow/VolumeFloodingOthers….

Place/Other Attributes:ZoningDemographicsJurisdictionFunding LevelsRestrictionsOthers…

Nature of Data:Granular vs SummaryReal time vs HistoricalHierarchies for Drill Up/DownOthers…


Business Intelligence with Spatial Data: Bi-Directional

Source of maps: www.esri.com


• Integration of Crash, Roadway, and Base Map Data• Query and Report Functionality

Application Start Up


• Zoom in: Reference Marker layer is configured to be rendered automatically at a predefined zoom level

Scale-Based display


•Zoom in to automatic street labelling

Street labels


• Crashes denoted on map in red• Crash data shown on bottom gridQuery for a subset of crashes


• Select crash on map, see data in grid• Select crash on grid, see highlight on mapBi-directional selection


Business Intelligence Case Studies


Cleveland Municipal School District - making progress on a work in progress

Large urban district - 77,000 students, with many superintendents over the last two decades

The Cleveland Public School District, placed in state receivership in 1995 restructured to a Municipal School District governance in 1998 CEO Barbara Byrd-Bennett signs on in November of 1998

The Data Warehouse is a key focus for enhancing decision-making:► Release 1 (Mid-2000)

– Focus on Student, Test and School data for three academic years– Trained 20 "power" users over the summer

► Release 2 (December, 2000)– Add Curriculum, Certified Staff, more attendance and discipline with additional academic

school year– Trained 120 new users that have "browser" access– Continued work on data quality– Focusing on establishing organizational roles to support the warehouse– Mentoring and promoting training in the proper use of data for data-driven decisions

► Release 3 (current)– Assessing data interchange with other public agencies, such as Juvenile Justice– More current refreshes of attendance data- (weekly) & most recent school year all subjects– Structured skills transfer sessions

"How many students did not

have immunmizations?"


Education data warehouses focus on strategic stakeholder questions

OPERATIONAL (closed-ended questions)

INFORMATIONAL (open-ended questions)

How many knives were in school today? What trends regarding weapons in school exist?

Which students in which schools were absent today? Of those who were absent today, which were absent yesterday?

How do absences affect student achievement?

What professional development activities have certified reading teachers taken this year to date?

What effect does professional development have on the reading portion of the proficiency test?

What are all the reading initiatives in the district?

Which are the most effective reading programs? What best practices exist for reading programs?

How many students have 4 consecutive unexcused absences this month?

What are social factors contribute to unexcused absences (e.g., childcare, pregnancy)? What interventions can the county help with?

How many teachers are teaching math and science? By school?

What effect on student achievement does multiple teachers teaching a single subject have?


In collaboration with Gwinnett County Public Schools (GCPS), IBM met budget & scope for each phase

Gwinnett County Public Schools (GCPS)

Largest school district in Georgia with over 100,000 students

Subject Areas: School, Student, Test, Courses/Grades, Staff and Finance

Multi-phased project:► Phase 0: Enterprise Information Strategy - 10/98-12/98

► Release 1: Student, Test & School - 01/99-06/99

► Release 2: Student, Test, School (detail), Courses/Grades, HR - 07/99-10/99

► Release 3: Programs & Services, Curriculum - 11/99-05/00

► Release 4: Additional Programs & Services, Grants, Classified Staff - 05/00 - 12/00

► Release 5: Finance & Additional Programs & Services - 03/01 - 08/01

► Release 6: Additional Finance, Programs & Teacher Data Mart - 09/01 current

Gwinnett County Public Schools, 2000"Putting the right information in the hands of the appropriate people, including educators, makes all the difference in the world in the quality of education" -- J. Alvin Wilbanks, Superintendent, Gwinnett County Public Schools


Miami Dade Police is using integrated data and information to affect change

Miami-Dade Police Department supports the largest municipality in Florida.

IBM provided technical and project leadership in requirements gathering and development of the warehouse and led the development of the web enabled front end to the reporting environment

Deployed to the police department and local government officials, the warehouse supports analysis of crime and operational trends

Enables easy to use source of statistical information that allows executives to create crime-reducing strategies and identify crime patterns.

Allows more effective use of scarce resource to meet juvenile rehabilitative needs.

Align policy more effectively and plan for emerging community trends

Evaluate risk factors and improve interventions for at-risk juveniles


The goal of the First of a Kind at Farmers Insurance was to improve risk management in their property & casualty lines

Identify relevant risk factors

Segment insured population into homogenous groups with distinct risk characteristics

If (driver is male) and (age under 25) and (product is sports-car) --> claim frequency is 25%

Price segments profitably

Identify sub-niches

If (driver is male) and (product is EXPENSIVE sports-car) --> claim frequency is 5%

Manage business with underwriting rules that pre-identify segments and therefore risk

First-of-a-kind results, for 1 state, 1 year ► 6 out of 43 discoveries► Policy gain of +7,600► Profit gain of +$2 Million


IBM has worked with the State of Mississippi in building a multiphase, multi-agency data warehouse, focusing on their self-sufficiency

Environment and Requirements► Formulation of less than optimal public policy

– No comprehensive view of the state's own operations– Personnel-related info not linked to financial, economic and statistical systems

► Multiple data sources - a lot of data manipulation, difficult to locate► Delayed, inaccurate, inefficient reporting - internal and interagency

Solution Description► Highly successful partnership with IBM with deliverables across four phases ► Consulting services for design through implementation, hardware and software, warehouse

model► Side-by-side methodology and skills transfer, affording the state's staff to enhance their

warehouse

Benefits► Provide better financial management

– Current financial and five years of history readily available ► Executives summary reports with referrals to the data sources ► Business analysts access to detail data with deep drill down capabilities ► Fast, reliable information, exceptional data integrity ► Improved from several hours to immediate & several weeks to same day

"How many employees work at the state and

where?"


Breakout Discussion

Questions for discussion:► As a result of learning about BI, what are the top 3 areas of concern

most prevalent on the minds of the business community in your business function?


deeper insight

Thank you.deeper accountability

deeper knowledge

deeper expertisedeeper technology

deeper resourcesdeeper experience

deeper capabilitiesdeeper answers

deeper research

deeper solutions

deeper government

deeper integrationdeeper valuedeeper e-businessdeeper thinkingdeeper transformationdeeper innovation

IBM Business Intelligence for Governments

confidential | june 2003 | © 2003 ibm corporation transforming data into business intelligence - an...

Documents