enterprise search and its evolution dusan rnic vp emea

Enterprise Search and its Evolution

Dusan RnicVP EMEA

Copyright©2009 Endeca Technologies, Inc. All rights reserved. Proprietary and confidential.

Motivation

Why are YOU here ?

Efficiency – Metrics from Butler Group …

Knowledge workers are 60% of labour Up to 20 hours a week spent ‘searching’

Compliance: control & audit (who saw/sees what)

Rationalisation of existing Search Engines


Growth is phenomenal ...

Information Created, Captured, and Replicated57% Compound Annual Growth Rate (CAGR)

Source: IDC, 2007 “The Expanding Digital Universe” (1 Exabyte = 1 Million Terabytes)


Heterogeneity of Information Compounds The Problem

Emails

File Systems

Content Management Systems

Purpose Built Databases

Intranet Portals

Collaboration Tools

…….and how and when people want to view the information


The genesis of today’s information management

ACM Fellow,First recipient of theSIGIR Award for Information Retrieval (1983)

1964 SMART(Salton’s Mechanical Analysis & Retriever of Text)

IBM Fellow,Turing Award winner (1981)

1970 "A Relational Model of Data for Large Shared Data Banks"

Edgar Codd

Gerard Salton


The User Interface 40 years ago


More Buried Information

Implementing Search and Discovering The Long TailPopular Searches

Buried Information

Future of Business Is Selling Less of MoreChris Anderson, 2004

… those that attract attention


What information consumes is rather obvious: it consumes

the attention of its recipients. Hence a wealth of

information creates a poverty of attention, and a

need to allocate that attention efficiently among the

overabundance of information sources that might consume it.

Information overload is not the problem. A scarcity of attention

to find the right information in it is the problem.

…… We need better Information Visibility

Volume and Variety of Information …

Source: Computers, Communications and the Public Interest, pages 40-41, The Johns Hopkins Press, 1971.

Herbert Simon

Nobel Prizewinner &

Turing Award recipient


Enterprises can learn from eCommercePopular Searches

Buried Information

Spotlight Related Content

Dynamically Classify

Summarise & Aggregate

Visualise

Suggest

Realisation that enabling a Discovery based User Experience driven by increased Information Visibility is key.

... Search is just one part of this

eCommerce historically viewed Search as the key to users finding products / information

Cross Content Navigation

Landing PagesAll these techniques are required to deliver Information Visibility


Discovering the “Black Swan”

Nassim Nicholas Taleb : The Black Swan (Published 2007)…. regards many Discoveries as Black Swans

“Undirected" and “Unpredicted”

Comes from the ancient belief that: 'All swans are white'.

Black Swan is a metaphor for something not known to exist.


If I didn’t predict and didn’t direct

E.g. Dynamic Classification:

Colour: White Black Black & White

If I did not know Black Swans existed – how could I predict

Therefore why would I direct the search to a Black Swan

More information needs to be visible to discover Black Swans


Facilitating Discovery

Search

Banners

Features

Notices

Classify

Categorise

Aggregate

Spotlight

Relate &

Merchandise

All these elements work in Harmony to make information visible

They are not discrete pieces of logic working autonomously

A user action in one place – causes a contextual response in anotherFacilitating Discovery that is Undirected & Unpredicted by the user

The overall User Experience is the Design Point


Does making information visible matter ?

70 executives from leading enterprises attended a Strategy Day

Event facilitated by Michael E Porter – World renowned business professor from Harvard

Competitive Strategy The Five Forces The Value Chain

Concluded that …– Individual discretion

is essential to improve top-line revenue and bottom-line profit drives operational efficiency & strategic differentiation

– Daily decisions require clearer visibility into information – this requires a ‘disruptive’ approach


Driving Competitive Advantage - Highlights

Strategy means competing on the basis of what makes a firm different– Improving operational efficiency - IS NOT strategy – Conducting activities that competitors don’t - IS strategy

Activities in the value chain contribute to differentiation or a low–cost position

Activities can be optimized by– Standardization – Individual Discretion


Driving Competitive Advantage - Highlights

Strategy means competing on the basis of what makes a firm different– Improving operational efficiency - IS NOT strategy – Conducting activities that competitors don’t - IS strategy

Activities in the value chain contribute to differentiation or a low–cost position

Activities can be optimized by – Standardization – Individual Discretion

Technology supporting individual discretion enables strategy

discretionary decision-making is unpredictable it happens to take advantage of a specific context


A new way for IT to deliver competitive advantage

• Formalize the sequence of steps

• Standardize the tasks

• Provide tools to execute tasks

• Allow spontaneous sequences

• Encourage choice and creativity

• Provide tools for exploration

ProcessStandardization

IndividualDiscretion

Source Of Optimization

Defined by the firm At the Discretion of the Individual

Predictable and routine Contextual and tailored

UNPREDICTABLE


A new way of thinking about information

First …Arranged by the constraints of physical objects

Then …Arranged by the constraints of anticipated use

Now …Arranged at the point of use, at the moment of use

Actionable intelligence : Just in Time not Just in Case

- David Weinberger, “Everything Is Miscellaneous”, 2007.


The Analysts …

Information AccessToday is a recasting of searchFuture – we believe it will encompass ad hoc BI and more

Information FabricTwo components:

Interactive query over customizable web services, API emulation, etc.

Advanced federation, similar to EII

Converged IndexA hybrid index of structured and unstructured to support

information access and analysis

Information VisibilityViewed through the lenses of the business benefit for the

verticals they cover


Takeaways

Search alone is not likely to meet your strategic objectives

Search alone will require follow on projects to expose the long tail– are you prepared for that additional investment ?

Learn from the parallels in eCommerce

Think in terms of Information Visibility– Search is just one part of that

Think in terms of the User Experience

Think in terms of Discovery

Think in terms of Enabling Strategy


Thank You

enterprise search and its evolution dusan rnic vp emea

Documents

information slide

buried information

information visibility

information retrieval

right information

wealth of information

information overload

attention slide