Beyond Data Warehousing:Beyond Data Warehousing:What’s Next in Business Intelligence?What’s Next in Business Intelligence?

DOLAP 04DOLAP 04

Matteo Golfarelli, Stafano RizziMatteo Golfarelli, Stafano RizziDEIS – University of BolognaDEIS – University of Bologna

Iuris CellaIuris CellaGruppo Formula S.p.A.Gruppo Formula S.p.A.

Summary:Summary:1. Metric Driven Management1. Metric Driven Management

2. Business Performance Management2. Business Performance Management3. Proposed Architecture3. Proposed Architecture

4. Research and Technological issues4. Research and Technological issues5. Conclusions5. Conclusions

History table for DWingHistory table for DWing• Late ’80s: the request for efficiently and effectively Late ’80s: the request for efficiently and effectively

analyzing the enterprise data becomes a pressing matter analyzing the enterprise data becomes a pressing matter for managersfor managers

• Early ’90s: the birth of DWing and OLAP in the industrial Early ’90s: the birth of DWing and OLAP in the industrial worldworld

• Mid ’90s: academic world becomes interested in DwingMid ’90s: academic world becomes interested in Dwing

• Late ’90s: metrics driven management becomes a Late ’90s: metrics driven management becomes a pressing matter for managerspressing matter for managers

• Early ’00s: the birth of Business Performance Management Early ’00s: the birth of Business Performance Management in the industrial worldin the industrial world

• Mid ’00s: What happens in the academic world?Mid ’00s: What happens in the academic world?

Metric Driven ManagementMetric Driven Management II• Process-oriented management couples the organizational Process-oriented management couples the organizational

structure with a set of inter-division processes.structure with a set of inter-division processes.– The The organizational structureorganizational structure is a hierarchy of divisions aimed at is a hierarchy of divisions aimed at

defining duties and responsibilities.defining duties and responsibilities.

Tacticallevel

Strategiclevel

Operationallevel

Information System

At this level the global strategy of the enterprise is decided

It is composed by multiple divisions, each controlling a set of functions; the decisions taken here are related to the corresponding functions and must comply with the strategy defined at the upper level

Where the core activities are carried out; the decision power is limited to optimize specific production activities in accordance with the main strategy.

Metric Driven ManagementMetric Driven Management IIII• Process-oriented management couples the organizational Process-oriented management couples the organizational

structure with a set of inter-division processes.structure with a set of inter-division processes.– ProcessesProcesses identify a set of related tasks performed to accomplish identify a set of related tasks performed to accomplish

a defined goal. Tasks are carried out by different divisions and at a defined goal. Tasks are carried out by different divisions and at different levels.different levels.

Tacticallevel

Strategiclevel

Operationallevel

Information System

Processes focus on the global business goal rather than a single divisional task. Of course, employees involved in processes must share the business strategy in order to synchronize their behavior

Metric Driven ManagementMetric Driven Management IIIIII• Metrics (KPI) are used to:Metrics (KPI) are used to:

– continuously measure process performances.continuously measure process performances.– share the top-level business strategy with the employees in order share the top-level business strategy with the employees in order

to synchronize their behavior.to synchronize their behavior.

Tacticallevel

Strategiclevel

Operationallevel

Information System

actions/decisions

actions/decisions

target values for indicators

current values for indicators

DW

Daily export sales=10000$

Calls to prospects= 200 per dayOrder latency

= 2days

Strategy

Business Performance Business Performance Management (BPM)Management (BPM)• The term BPM defines the new approach to management. The term BPM defines the new approach to management.

The peculiarities that distinguish BPM from classical DW-The peculiarities that distinguish BPM from classical DW-based BI applications are:based BI applications are:– Users: Users: are decision-makers but at the tactical and operational are decision-makers but at the tactical and operational

levels. levels. – Delivery time: Delivery time: decision at the lower levels must be faster than the decision at the lower levels must be faster than the

strategic ones. BMP systems are not supposed to operate in real-strategic ones. BMP systems are not supposed to operate in real-time, but rather in right-time.time, but rather in right-time.

– Information coarseness and lifetime: Information coarseness and lifetime: information are usually more information are usually more detailed since concerns events related to specific tasks. Lifetime of detailed since concerns events related to specific tasks. Lifetime of information is limited according to right-time requirement.information is limited according to right-time requirement.

– User interface: User interface: information are mainly accessed via dashboards information are mainly accessed via dashboards and reports as well as through automated alerts activated by and reports as well as through automated alerts activated by business rules.business rules.

Architecturual sketch for BPMArchitecturual sketch for BPM• The technology implementing the BPM flow is often called: The technology implementing the BPM flow is often called: Business Activity Monitoring (BAM)Business Activity Monitoring (BAM)

metadata repository

Mining tools

RTI DDS

BAM

KPI manager

Rule engine

data streamsEAI

Dynamic Data Store(DDS):Dynamic Data Store(DDS): is a is a repository capable of storing short-repository capable of storing short-term data for fast retrievingterm data for fast retrieving

KPI Manager:KPI Manager: computes all the computes all the indicators necessary at different levels indicators necessary at different levels to feed dashboards and reportsto feed dashboards and reports

Right-Time Integrator (RTI):Right-Time Integrator (RTI): integrates data from operational integrates data from operational data sources at right-timedata sources at right-time

Mining tools:Mining tools: capable of extracting capable of extracting relevant patterns out of data streamsrelevant patterns out of data streams

Rule engine:Rule engine: that continuously monitors that continuously monitors the events filtered by RTI or detected by the events filtered by RTI or detected by mining tools to deliver timely alertsmining tools to deliver timely alerts

business domain

ERP

ODS

OLAP

ETL

user interface

report

dashboard

alert

DW

metadatarepository

other DBs

source data

DataWarehouse

Research and technological issues IResearch and technological issues IData latency reductionData latency reduction

• Right-time constraint makes classical ETL and ODS Right-time constraint makes classical ETL and ODS approaches unfeasibleapproaches unfeasible

• On-the-fly techniques are needed:On-the-fly techniques are needed:– Cleaning techniques devised so far rely on the presence of a Cleaning techniques devised so far rely on the presence of a

materialized integrated level.materialized integrated level.– Data stream manipulation still presents many challenges (i.e Data stream manipulation still presents many challenges (i.e

complex queries over data are performed offline, real-time queries complex queries over data are performed offline, real-time queries are usually restricted to simple filters).are usually restricted to simple filters).

Research and technological issues IIResearch and technological issues IIInformative powerInformative power

• Complex Complex KPIKPI systems have been devised in the economic field (e.g. systems have been devised in the economic field (e.g. balanced score card) but the BI community has only partially faced balanced score card) but the BI community has only partially faced the problem of their the problem of their modeling and handlingmodeling and handling

KP1KP2

KP5

KP3

KP4Input

1

Input2

Input3

• When the set of KPI is large (i.e. hundreds or thousand) it is hard to When the set of KPI is large (i.e. hundreds or thousand) it is hard to understand which their relationships are and if the targets are understand which their relationships are and if the targets are consistently defined. To this end consistently defined. To this end simulation toolssimulation tools are needed. are needed.

• System dynamic tools are used within the economic field but their System dynamic tools are used within the economic field but their exploitation within the BI tools has not been considered yet.exploitation within the BI tools has not been considered yet.

Research and technological issues IIIResearch and technological issues IIIInformative powerInformative power

• Most of the Most of the mining techniquesmining techniques devised so far are inapplicable in the devised so far are inapplicable in the right-time assumptionright-time assumption..

• Indicators and mining tools may need short-term information to Indicators and mining tools may need short-term information to compute complex information. Since simple buffering techniques are compute complex information. Since simple buffering techniques are not appropriated, not appropriated, main-memory databasesmain-memory databases (real-time databases) must (real-time databases) must be considered.be considered.

Research and technological issues IVResearch and technological issues IVInterfaceInterface

• The classical paradigms of DW systems, namely reporting and OLAP, are no The classical paradigms of DW systems, namely reporting and OLAP, are no more sufficient to give users a full picture of the trend of their business in the more sufficient to give users a full picture of the trend of their business in the short and medium term.short and medium term.

• New paradigms, with different characteristics, must be merged into a common New paradigms, with different characteristics, must be merged into a common interfaceinterface

InterfaceInterface StructureStructure FreshnessFreshness InteractionInteraction InformationInformation

ReportReport StaticStatic Short-timeShort-time PullPull Measures/IndicatorsMeasures/Indicators

OLAPOLAP DynamicDynamic Short-timeShort-time PullPull MeasuresMeasures

DashboardDashboard StaticStatic Right-timeRight-time PullPull IndicatorsIndicators

AlertAlert StaticStatic Right-timeRight-time PushPush EventEvent

Research and technological issues VResearch and technological issues VDesignDesign

• Right-time:Right-time: the designer must determine what is the meaning of right- the designer must determine what is the meaning of right-time for the specific business domaintime for the specific business domain

• Light-architecture:Light-architecture: the requirements for KPIs change quickly the requirements for KPIs change quickly depending on the pursued strategies that will bring to the foreground depending on the pursued strategies that will bring to the foreground new functions and behaviors monitored by KPI. new functions and behaviors monitored by KPI.

• Process design: Process design: with respect to DW, BPM also requires to understand with respect to DW, BPM also requires to understand processes and their relationships in order to find out relevant KPI and processes and their relationships in order to find out relevant KPI and Business rules.Business rules.

• KPI design: KPI design: capturing and modeling the relationships between capturing and modeling the relationships between different indicators has a primary role in this phase to ensure that different indicators has a primary role in this phase to ensure that effective and reliable information is delivered.effective and reliable information is delivered.

Conclusions and discussionConclusions and discussion• We summarized the requirements emerging from modern We summarized the requirements emerging from modern

companies and discussed how they meet into a new companies and discussed how they meet into a new architecture that promises to lead Business Intelligence architecture that promises to lead Business Intelligence beyond data warehousing.beyond data warehousing.

• We saw that different sophisticated technologies are We saw that different sophisticated technologies are involved in BPM. Most of these fields are not mature involved in BPM. Most of these fields are not mature enough in terms of commercial products.enough in terms of commercial products.

• Gartner group indicates BPM and BAM as the emerging Gartner group indicates BPM and BAM as the emerging market in Business Intelligence till 2008.market in Business Intelligence till 2008.

Is it just a passing fad ?Is it just a passing fad ?Is it a technology matter only ?Is it a technology matter only ?

Is it the second era in BI ?Is it the second era in BI ?