In association with:
The “Search” for a Unified
View of Private and Public
Information.
In association with: Presented by: Helen L. Mitchell
Agenda
1. Definitions
2. Content Findability
3. Search Technologies & Examples
4. Challenges & Benefits
5. Search Trends
6. Best Practices
7. Keys to Success
8. Technologies to Watch
9. Questions & Answers
In association with: Presented by: Helen L. Mitchell
What is Cloud Computing?
•Using the internet to access technology-enabled
services which must be 'massively scalable‘. Ex. Each time I
log into Facebook, or search for flights online, I am taking advantage of cloud computing.―
•Using a 3rd party service to perform computing
needs on a publicly accessible Internet (cloud) basis.
•Incorporates SaaS, Web 2.0 and other well-known
technology trends.
•Common theme is reliance on the Internet for
satisfying the computing needs of the users...
In association with: Presented by: Helen L. Mitchell
Terms
1. Definition by AIIM Market IQ
2. Definition by CMS Watch
3. A Federated Search Primer – Part II
4. Deep Web Technologies
• Reliable and complete retrieval of content based on user need, i.e. everything relevant is recalled (recall) while simultaneously returning only that content relevant to the user‟s focus (precision), thus eliminating the review of irrelevant content by the user.
1
Findability
• Systems…within an organization…seeking information held internally…in a variety of formats and locations, including databases, document management systems, and other repositories.
2
• Content is pre-indexed, simultaneously searched, and displayed to authorized users.
Enterprise Search (ES)
• The process of performing a simultaneous real-time search of multiple diverse and distributed sources from a single search page, with the federated search engine acting as intermediary.
3
Federated
Search (FS)
• The set of web-sites and their documents that cannot be accessed via crawler-type search engines such as Google. Deep web content typically lives inside of databases, and is accessed through search forms.
4 It is also referred to as the
Hidden or Invisible Web.
Deep Web
• SW written to access a content source that must know the URL of the source, how to send search commands, its search syntax, & how to process the search results returned from a source.
5Connector
In association with: Presented by: Helen L. Mitchell
Social Computing – Web 2.0 Era
(Source: “Leveraging Web 2.0 in Government” - E-Government Technology Series, IBM Center for the Business of Government, by Ai-Mei Chang and P. K. Kannan)
In association with: Presented by: Helen L. Mitchell
What is a Unified View?
Unified Search
Internet Intranet
Databases
Network Shares
IT Apps, Data Whse
Publications
Subscription DBs
Portals
Audio, Video
CMS, DMS
Social Media,
Wikis, Blogs
Cloud
Centralized &
Integrated View
of Content
Sources
In association with: Presented by: Helen L. Mitchell
Why Consider Enterprise Search?
Provide an integrated query solution with a simplified UI across
disparate content (DBs, File Shares, eMail, Web, Social Media…)
Improve access, ‘findability’ & analysis of relevant content
Integrate search across applications based on business needs
Facilitate research, alerts, marketing products & services & ideas
Integrate established nomenclature & data standards
Add capabilities to further Mission/Vision
Taxonomies, Categorization, Collaboration & BI Tools
Limit access based on network security
In association with: Presented by: Helen L. Mitchell
Findability Issues
AIIM Market IQ Research on Findability (of 528 end users):
50% believe Findability in their organization is ―Worse to Much Worse‖
than their consumer-facing web sites
49% have no formal goal for Enterprise Findability within their
organizations
49% ―Agreed or Strongly Agreed‖ that finding the information to do
their job is difficult and time consuming
69% believe less than 50% of their organization's information is
searchable online
36% reference five or more systems in any given week
Source: AIIM Market Intelligence, 2008
In association with: Presented by: Helen L. Mitchell
Managing Information Overload
Technologies to consider to analyze & manage information:
Enterprise Search
Federated Search
Vertical Search
Expert Search
Social Media – Collaboration & Transparency
Geo-spacial search
Mashups
Sentiment Analysis
Taxonomy Management
Context Summaries
Aggregated Search
Business Intelligence
NOTE: Presentation of Results and Relevancy Ranking is Critical
In association with: Presented by: Helen L. Mitchell
Typical Silo Approach Search
Courtesy of MuseGlobal, Inc.
In association with: Presented by: Helen L. Mitchell
Aggregation Examples
Aggregate search results from multiple search engines by
running a single query: Joongel
In association with: Presented by: Helen L. Mitchell
Portal Aggregation Example
For Portal Devices
Courtesy of IBM Websphere
In association with: Presented by: Helen L. Mitchell
Federated „Master Index‟ Search
Index content from multiple data sources into a single master
search index
Queries & results come from that one master index
Many Enterprise Search products integrate Federated Search via
‗connectors‘ to accomplish this (ex., FAST, Autonomy, Endeca)
Source: New Idea Engineering, Inc.
In association with: Presented by: Helen L. Mitchell
Federated „Data Silos‟ Search
‗Search federator‘ process queries of each data source silo
Transforms the users search terms to match each content source's
requirements
Submits the query to each of the sources simultaneously
Merges each source‘s results together - a single look and feel
Maintains no indices of its own, relies upon the capabilities of all the linked
systems
Source: New Idea Engineering, Inc.
In association with: Presented by: Helen L. Mitchell
Federated Search Example(uses Deep Web Technologies)
In association with: Presented by: Helen L. Mitchell
Federated Search Example
Courtesy of MuseGlobal, Inc.
In association with: Presented by: Helen L. Mitchell
Surface vs. Deep Web Search
Popular search engines (Google, Yahoo…) ―crawl‖ surface web
FS can drill down to the deep web where specialized content (i.e., scientific and technical databases) reside
Deep Web FS Examples:www.completeplanet.com -70,000+ searchable DBs &
specialty search engines
www.science.gov- federates U.S. federal agency science
informationhttp://imlsdcc.grainger.uiuc.edu/- Institute of Museum & Library
Services (IMLS) - Digital
Collections & Content
w/descriptions of digital
resources developed by IMLS
granteesSource: Juanico-Environmental Consultants, Ltd.
In association with: Presented by: Helen L. Mitchell
Vertical Search Engine
Searches for a particular niche i.e., a specific industry, topic, type
of content (e.g., scientific research, travel, movies, images,
blogs)
Example: www.vetseek.info - is a search engine focusing on veterinary science and related
topics
In association with: Presented by: Helen L. Mitchell
Vertical Search Example(uses FAST ESP )
In association with: Presented by: Helen L. Mitchell
Generate Summaries Example
Context Organizer instantly generates summaries of web
pages, documents, search results and Outlook
In association with: Presented by: Helen L. Mitchell
Improve Relevancy
Develop a consistent structure to more effectively manage &
utilize content assets to improve business processes
Use a Taxonomy Management Tool to increase Precision &
Recall
Enhances Personalization of Results
Ex. Concept Searching‘s Concept Classifier Taxonomy Management Tool
In association with: Presented by: Helen L. Mitchell
Barriers to Change
Legal/Policy – implications publishing drafts, informal
conversations, FOIA, governance framework, free tool
offers
Leadership/Champions - funding, reinforcement,
behavior support, recurring engagement w/audiences
Resources – shortage of Subject Matter Experts (SMEs)
Culture – punishes risk-takers, innovators not rewarded
Divergent Roles - don‘t recognize need for Subject
Matter Experts
Multiple Missions – silos, legacy systems
Operational Diversity – lack infrastructure standards
Changing Technology – inflexible complex systems
Adapt to Change Limited – no strategy plan, vision
In association with: Presented by: Helen L. Mitchell
Challenges
Lack of data standardization
Each source has a unique access method & needs translation
Metadata mapping between search engine and underlying
systems
Access methods to sources may change
Requires an interface rewrite or modification
Rules for error handling
Ex. Query term not available—exclude the query, the
repository, or proceed without the term?
Ex. Timeouts or connection problem
Complex searches usually not available
Fielded searches
In association with: Presented by: Helen L. Mitchell
FDA Case Study Success(Federated „Master Index‟ Search System)
ACTIONS RESULT
Started small with high ‘pain points’
Increased productivity & popularity
Modified business processes*
Standardized nomenclature increased efficiencies
Users across organizationcould find content in silos
Produced more timely and QUALITYwork products
Indexed structured & unstructured content repositories with documentlevel security
Grew from 1 repository of 500 documents to 50 repositories with 30+ million documents & data. Usersaccess based on ‘need to know’.
Introduced standardized search web services into applications
Decreased development time and costs, increased management & user acceptance, integrated in more applications
Increased user awareness through training, newsletters and meetings
Used more & content added. Search requirements gathered at BEGINNING of project development.
In association with: Presented by: Helen L. Mitchell
Gov 2.0 Search Trends
Integrate Enterprise Search (ES) w/ECM & KM Promote Collaboration
Personalize Search for ‗Communities of Practice‘
High Precision & Recall
Integrate ES & IT Applications via web services
Integrate Information Discovery & Collaboration
Tools
Provide Trend Analyses
Build Federated Search w/3rd Party Social Media
products
Integrate Knowledge, Resource DBs, & People
to Forecast Staffing
In association with: Presented by: Helen L. Mitchell
Felon Search – SharePoint Webparts
In association with: Presented by: Helen L. Mitchell
Best Practices
Strategically rethink how to deliver your mission, start small and just DO IT!
Recruit and train talented N-Geners
Use Search Technologies on internal and/or external sites, cloud, social media
Develop Governance: Overhaul Organization Websites, Integration Policies, UI, etc. if needed
Leverage lessons learned to reduce project cycles, increase trust and empower communities
In association with: Presented by: Helen L. Mitchell
Keys to Success
Educate the organization
Initiate a pilot project
Train & Engage Decision Makers & Stakeholders
Evaluate existing enterprise technologies compatibility with
Aggregation strategy
Create policies that maximize the benefits of adopting
Strategy Plan
Embrace a culture of collaboration by continually evolving
how interaction happens with internal and external
stakeholders
Measure results by establishing key performance indicators
In association with: Presented by: Helen L. Mitchell
Top Technologies to Watch
According to Gartner the 2010 technologies are:
1. Cloud Computing
2. Advanced Analytics
3. Client Computing
4. Reshaping the Data Center
5. Social Computing
6. Security
7. Flash Memory
8. Virtualization for availability
9. Green IT
10. Mobile Applications
In association with: Presented by: Helen L. Mitchell
Contact Info
THANK YOU!
Helen L. Mitchell, Principal
Team Technology Inc.
819 7th St., N.W., Suite 300
Washington, D.C. 20001
(202) 842-3340 (office)
(202) 842-3361 (fax)
(202) 422-6770 (mobile)
SBE & SBA 8(a) HUBZone, MBE: VA &
MD; LDSBE