oscon 2010 - panning for gold: a web prospector's guide to mining and filtering community data
TRANSCRIPT
Visit Us @ Booth #506 www.CommonPlaces.com © 2010 CommonPlaces e-Solutions, LLC
Welcome to“Panning for Gold”
A “Web” Prospector’s Guide to Mining and Filtering Community Data
Visit Us @ Booth #506 www.CommonPlaces.com © 2010 CommonPlaces e-Solutions, LLC
Introductions
Ben Bassi, President & CEOSeasoned Internet Pioneer
At the forefront of online innovation
for almost two decades…
Bassi was a part of the founding management team of Lycos.com responsible for creating many of the first communities and portals for companies like Microsoft, CompuServe, Prodigy, MTV and GTE.
EVP of the Firefly Network (acquired by Microsoft asthe basis for the Microsoft Passport for MSN).
CommonPlaces e-Solutions
• Founded in 1998 in Cambridge, MA• Open Source Development since 2003• We create rewarding and intelligent
experiences for online communities…
– CMS Powered Online Communities
– Subscription-Based e-Commerce Applications
– CMS Integration (single sign on, SFA, back-office)
– UG Media / Multimedia Sites
– Advanced Search / Data Collaboration
Visit Us @ Booth #506 www.CommonPlaces.com © 2010 CommonPlaces e-Solutions, LLC
Let’s Get Started!
“Panning for Gold”
A “Web” Prospector’s Guide to Mining and Filtering Community Data
Visit Us @ Booth #506 www.CommonPlaces.com © 2010 CommonPlaces e-Solutions, LLC
Panning for Gold
There are 4 Steps to Mining For Gold(and as it turns out, for mining for useful
data from Web and Internal Data)
1. Prospecting 2. Mining 3. Extracting 4. Refining
Visit Us @ Booth #506 www.CommonPlaces.com © 2010 CommonPlaces e-Solutions, LLC
1. Prospecting
You Know There's Nuggets of Info Out There Businesses today have access to an unimaginable amount of user data andcontent, both structured and unstructured.
Social Media Support DeskCRMEmailsDatabases
UG and Product Reviews
Blogging / Micro-Blogging
Visit Us @ Booth #506 www.CommonPlaces.com © 2010 CommonPlaces e-Solutions, LLC
Traditional Prospecting Methods
• Relying on What You Know Already
– “Like relying on outdated maps”, predetermined keywords, product names, and preconceptions
• Not the best way to discover new ideas
• Best ROI is found on “Discovered” intelligence
• No Competitive Edge
Visit Us @ Booth #506 www.CommonPlaces.com © 2010 CommonPlaces e-Solutions, LLC
Plenty of Commercial Search Tools
• Automate the Search Process • Provide Sentiment Data • Expensive • You have to know what you’re searching for first
Visit Us @ Booth #506 www.CommonPlaces.com © 2010 CommonPlaces e-Solutions, LLC
Plenty of Free, Open Source Applications
• Automate the Search Process • Tell You Who Your Influencers Are• You have to know what you’re searching for first
Visit Us @ Booth #506 www.CommonPlaces.com © 2010 CommonPlaces e-Solutions, LLC
These Tools Still Aren’t “Discovery” Tools
The frustration is that most can’t quite reach or understand the value of the data with the search tools they’re currently using.
• You’re Still Searching For What You Already Know
• Like searching for treasure on the beach – the ‘beep’ (or information returned in a report) could be Gold, or a nickel. You have no way of aggregating the data across platforms to know what’s really useful – so they’re really “blind” searches.
• A lot of work for very small ROI.
Visit Us @ Booth #506 www.CommonPlaces.com © 2010 CommonPlaces e-Solutions, LLC
2. Mining
vs.
Traditional Search Data Discovery
Visit Us @ Booth #506 www.CommonPlaces.com © 2010 CommonPlaces e-Solutions, LLC
3. Extracting
Social Media
Support DeskCRMEmailsDatabases
Blogging / Micro-BloggingUG and Product Reviews
Visit Us @ Booth #506 www.CommonPlaces.com © 2010 CommonPlaces e-Solutions, LLC
4. Refining
• Here’s where the available Search tools fall down
• What can you do with the data once its collected?
• How to keep Context
• How automated is the process (million of spreadsheets? Interns?)
Visit Us @ Booth #506 www.CommonPlaces.com © 2010 CommonPlaces e-Solutions, LLC
CP’s Data Discovery Technology Delivers
Timely actionable information discovery for impactfulbusiness results
• Detailed Voice of Customer insights
• Market intelligence to stay ahead of competitors
• Develop new strategies for products and services for revenue growth
• Improve customer satisfaction to increase loyalty and up-sell
Visit Us @ Booth #506 www.CommonPlaces.com © 2010 CommonPlaces e-Solutions, LLC
What Clients Tell Us They Need
Timely Results• Quickly see what is in the information• Guide “me” to what I should look at - Go beyond traditional search listings• Relevant and interesting information
For their users:
Real Information• Customer/user verbatim: Can I see what they are actually saying? • See citations from documents• Show me how information is connected
Help “Me”• Address competitive realities and
strategies• Identify new product/process
opportunities• Identify operational
improvements• Quickly find emerging trends for
opportunities/risks on the long tails of information
• Maximize the use of my information
For their business:
Discover relevant information in a timely basis – uncover actionable insights.
Empower “Me”• Enable “me” to do my own exploration• Twist, spindle, and manipulate “my” data at will and see how it actively responds• See clusters, long tail info, and trends
Visit Us @ Booth #506 www.CommonPlaces.com © 2010 CommonPlaces e-Solutions, LLC
Why is it Different?Search & Data Mining Exploration & Discovery
Howit
Works
Uses search tools where users may key in words or data for which they are interested in matching results.
Creates free-form interactive visual displays of information and its characteristics, connections and details.
Attributesof the
Approach
• Requires that the user decide what they want to find before they find it.
• Provides no means of narrowing a large set of potentially irrelevant results.
• People develop understanding of large bodies of information before reading in detail.
• Allows people to narrow in to the most important relevant results.
• Requires no presupposition of the outcome prior to application.
Discovery Limited to database exposure Utilizes various data inputs- internal, internet, social media, consumer generated content, subscription services all combined for data discovery
Result People find only what they decided they wanted to find ahead of time, minimal surprises or novel information.
People discover the unexpected. New and emerging ideas are discovered without knowing how to search for them in advance.
Sample Data Sources
Blogs, Polls, Surveys
Call Center Data
Data Discovery Utilizes Both Structured and Unstructured Data
Why is Discovery Technology Different?
Take Data “As-Is” with no need to
first clean source files
No Pre-Defined Dictionary Required
No Need To Know What is In Your Data
Before Beginning Exploration
No Need To Know Data Relationships –
Technology Identifies Them for You Real
Time
Works on the“Long Tail” of Information
Enabling Detection of Emerging
Opportunities and Risks
Solution Enables “Lenses” Which are Reusable
Knowledge Digital Assets That Can be Utilized Throughout the Organization For Discovery in
other Fields of Interest
The Connection Engine Platform that Enables Real Time Data Exploration and
Discovery
Sample Applications
Regulatory Analysis
Operational Analysis
Strategic & Competitive Intelligence
Market Research
Call Center Operations
Warranty Claims
Product Development
Litigation Support
E -Discovery
Patent Research
Pricing and Packaging
Claims Processing
SurveysWEB/BLOG
Forum
Subscribed Content
Customer/UserProfiling
Social Media Exploration
Voice of Customer
Fraud RelatedDiscovery
Visit Us @ Booth #506 www.CommonPlaces.com © 2010 CommonPlaces e-Solutions, LLC
Prospecting with CP Data Discovery• Interactive Exploration- no programming for
user…drag and drop • Pre-defined dictionary not required • Cluster & Long Tail Exploration for Advanced
Early Detection• Ability to utilize anywhere internet access
availability exists…..mobile field real time applications
• Can set up threshold and alerting system for automated processing of information
Visit Us @ Booth #506 www.CommonPlaces.com © 2010 CommonPlaces e-Solutions, LLC
Mining/Extracting with CP Data Discovery
• Information Connection Discovery• Multiple Simultaneous Data Sets • Can be utilized as an overlay enhancement
to existing programs• Structured and Unstructured Data
Correlation• Social Media Exploration • Multi-language capabilities
Visit Us @ Booth #506 www.CommonPlaces.com © 2010 CommonPlaces e-Solutions, LLC
Refining with CP Data Discovery
• Structured and Unstructured Data Correlation• Development of exception reporting based on
results of exploration• Word Proximity and Phrase Analysis • Information Connection Discovery• Trend Visualization • Lens Processing - user created knowledge
assets to share, repurpose and drive new discoveries
Visit Us @ Booth #506 www.CommonPlaces.com © 2010 CommonPlaces e-Solutions, LLC
Data Discovery Demo
Visit Us @ Booth #506 www.CommonPlaces.com © 2010 CommonPlaces e-Solutions, LLC
Thank You! Questions?
CommonPlaces e-Solutions Follow us on LinkedIn
Ben BassiPresident & CEO
[email protected]@BenBassi
Michael ReichDirector of Business Development
@CPOpenSource Follow us on Twitter
www.CommonPlaces.com603.329.6760