Integration and Insight Aren’t

Simple Enough

Integration and Insight Aren’t

Simple Enough

Laura HaasIBM Distinguished EngineerDirector, Computer ScienceAlmaden Research Center

Insight and Integration for the Enterprise

• Enterprises want business insight– Need to correlate information from

diverse sources– Need powerful analytics and views– Need to deliver results where, when,

how needed• Not easy

– Consumers may not understand sources or how to correlate them

– Different consumers have radically different needs and abilities

– Analytics and application subtle• Goals

– automatically create a searchable source of “all” enterprise information (understand and integrate)

– enable a variety of actionable search and analytic capabilities over it (analyze and deliver)

ContentMetadata

Business objects

Enterprise Repository

Order

Account

Customer

Semantic Analysis

OLAP, Predictive analytics

Reporting, Visualization

…

Info ConsumersSolutions, (mobile) users

ERP, CRM, reports, spreadsheets, SCM, and other apps, databases, and documents, E-Mail, etc

Analyze, IntegrateCrawl, ETL

Insight and Integration for the Enterprise

• Understand and Integrate– Find relevant data sources– Ingest large volumes of information

• Structured, semi-structured and unstructured data

• Real-time data streams and static warehouses– Automatically discover semantics,

identify important “objects”– Automatically cleanse and reconcile data– Identify data relating to the same entities

from different sources• Analyze and Deliver

– Provide simple ways to request complex analytics

• Keyword search, faceted search, natural language, new user interactions?

• Leverage semantics, find what is really desired– Provide results in a form appropriate to

the requestor, when/where needed• Reports or visualizations or objects or …• Feed mashups, traditional programs, users in

mobile or traditional environments– New analytics, often solution-specific

ContentMetadata

Business objects

Enterprise Repository

Order

Account

Customer

Semantic Analysis

OLAP, PredictiveAnalytics

Reporting,Visualization

…

Info ConsumersSolutions, (mobile) users

ERP, CRM, reports, spreadsheets, SCM, and other apps, databases, and documents, E-Mail, etc

Analyze, IntegrateCrawl, ETL

Back to the Future? Nonprocedural Data Access is What We Need (More Of)

• Relational dbms provided nonprocedural access –Relational data model and declarative query language–Relational calculus and algebra–Optimizer and efficient algorithms for execution

• What has changed today?–Heterogeneous data in overlapping heterogeneous sources–Many if not most applications need data from multiple sources–Users aren’t technical gurus; lack knowledge of sources and query languages–Needs are for deeper analysis, not “just” integration

• The tools we have today are far from relational simplicity–Multiple integration engines with different capabilities–Integration specification depends on engine, rarely declarative–Operations defined by engines’ capabilities, no formal model