Enabling Opinion-Driven Decision MakingKavita GanesanFounder, FindiLike LLC.

State of the Web… Unstructured opinions:-e-commerce sites-online directories-blog articles-review sites-social networking sites-travel sites, more…

State of the Web…

Abundance of opinions…but rarely used to help users make better & faster decisions!

Unstructured opinions:-e-commerce sites-online directories-blog articles-review sites-social networking sites-travel sites, more…

Classic example: Looking for a hotel…

Shortlist hotels by date availability

Shortlist hotels by price

Find hotels with specific amenities

• pool & spa

Find hotels that fulfill specific

opinion criteria

• “Safe neighborhood”• “Clean”• “Comfortable beds”

• Have to read 100s of reviews of different hotels

• May visit different websitesHotels.com TripAdvisor Yelp Hotels.com

Users: Highly tedious process!• Users perform own data mining• Personally done it…not fun!• No draw for users to come back to your site

decide to use other sites next time

Companies: Users are straying away from your site• Taking too long to get information needed

use some other method of booking

• Worse: Found deals elsewhere while reading reviews especially true with modern day contextual advertising

Problematic to users & organizations!

Need to use available opinions in a more intelligent & efficient way to enable decision making!

In the research world … Many methods to leverage opinions for decision making Not really used by industry!...YET

Opinion-driven decision making tools

Opinion-driven decision making tools

Opinion-Driven Search

Opinion Summarization

Opinion Trend Visualization

• Help users find entities (e.g. products, people, businesses) based on opinion preferences

• Limits set of choices in consideration from a large number options

Opinion-driven decision making tools

Opinion-Driven Search

Opinion Summarization

Opinion Trend Visualization

• Help users understand underlying opinions in large amounts of unstructured text

• Whole range of formats to abstract out opinions

Opinion-driven decision making tools

Opinion-Driven Search

Opinion Summarization

Opinion Trend Visualization

• Helps users understand change in opinion over a period of time

• E.g. Smartphone:• started out with positive sentiments• sentiments changed after

recent firmware upgrade

Opinion-driven decision making tools

Opinion-Driven Search

Opinion Summarization

Opinion Trend Visualization

Goal of task: • Rank/recommend entities • Based on how well a users “opinion requirements”

match the unstructured opinions about a set of entities

Opinion-driven search

“lightweight, responsive screen”

opinion requirements• User provided• System suggested• Various interface options

Smartphones

Opinions

New task in literature [Ganesan & Zhai IRJ 2012] [Ganesan PhD Thesis, 2013] [Choi et. al WWW’12] [Choi et. al CIKM’12]

Referred to as “opinion-based entity ranking” Practical Implementation

1. Put search over results of opinion summarization Fairly involved requires work on search & summarization Can be more accurate than other methods

2. Treat as information retrieval problem [Ganesan & Zhai 2012] [Ganesan 2013] [Choi et. al 2012a] [Choi et. al 2012b]

Extend robust IR models for this task Easily scales up & can be used with different domains

Opinion-driven search

Preference based search: find entities based on preferences• Buy LCD TV: user preferences - “rich picture” ,“no glare”• System takes in preferences and lists TV’s matching criteria

Search filters: users filter results by specific opinions• Laptop search: limit to “lightweight” laptops

Restaurant search: limit to “authentic” restaurants• Can be used in conjunction with your current search engine

Opinions can be “suggested” or “user enters”• Can also think about this as another form of faceted navigation

How can we use opinion-driven search?

Search filters: users filter results by specific opinions

How can we use opinion-driven search?

Babies R Us

Car Seat

Contextual advertising: recommend entities with similar or different opinionsShopping for Laptop:

• Recommend similar laptops to one being viewed• Viewing laptop with negative opinions: recommend

laptops with positive opinions• Brings users closer to the preferred type of laptop

How can we use opinion-driven search?

Big area of study in the research world• Sentiment classification• Aspect-based summarization• Text summarization• Co-reference resolution

Many different formats [Kim et al 2011]

• Can range from simple sentiment summary to unstructured textual summaries

Opinion Summarization

Sentiment Summary

----+

+++

source2

source1

source3

source4

source5

source6

source7

Final sentiment summary: (+ve/–ve) or (score 3/5)

-whole documents-passages-sentences-phrases

--

Gives users high level overview of underlying sentiments• Similar to overall ratings see in e-commerce sites

Widely studied in literature [Pang et al. 2002] [Pang and Lee 2004; 2005] [Dave et al. 2003] [Turney 2002] [Turney and Littman 2003]…

• Heuristic based approaches to supervised methods

Fairly easy to implement • With supervised machine learning models + training data

Issues to consider:• Lacks sufficient details

E.g. “hotel may get overall positive rating with extremely bad service”

• To make it useful: pair with other summarization methods

Sentiment Summary

Aspect-Based Summaries

Aspects Ratings

Design

Screen

Sound

Battery

4.0/5

2.0/5

3.0/5

4.5/5

smartphone xyz

Aspect-Based Summaries

Feature/Aspect IdentificationDesign, batterysound, screen

Sentiment PredictionBattery life is great… +veLong battery life… +veHorrible sound quality -ve

Star rating?

Score based approach? 4/5, 2/5

Percentage? 60% 40%

Presentation

Each step can be implemented in different ways with different levels of sophistication

aggregate

Gives more details than overall sentiment ratings Widely studied in literature – hot topic for several

years! [Lu et al 09 ] [Titov & McDonald 2008] [Hu & Liu 2004a] [Hu & Liu 2006] [Ku et al. 2006] [Popescu & Etzioni 2005] [Zhuang et al. 2006]

• Different options for implementation• Tweaked as needed to fit domain needs

Aspect-Based Summaries

Issues to consider:• Finding aspects or features in each domain

Varies from domain to domain E.g. Electronics: features for television vs. smartphones How to find features in a general and scalable way?

• Lacks detailed reasons E.g. Smartphone screen received score of 2/5? Why? Screen too small? Screen non-responsive? no way of

knowing! To give more info: complement this textual summaries

Aspect-Based Summaries

Gives more details than structured summaries Easiest implementation method:

• Traditional text summarization • Select few sentences from text to make up summary

Textual Opinion Summaries

The xyz smartphone is easy to use and has a very bright screen. I mostly love that screen is so responsive.

Not recommended for opinions! [ Ganesan et al COLING’10 ]

• Can introduce bias when selecting sentences Can select sentences that don’t represent major opinions

“Battery life is great (10), but this phone is too small for me (1)” Miss out key opinions when not selecting enough sentences

• Not suitable for smaller screens verbose if you keep adding sentences

Textual Opinion Summaries

Over last few years, people have been looking into micro-summarization approaches [Yatani et al CHI’11] [Ganesan et al COLING’10] [Ganesan et al WWW’12] [Khabiri 2013, PhD Thesis] [Potthast & Becker ECIR’10]

Concise summaries that represent key opinions in large amounts of text

Micropinion Summaries

Micropinion Summaries [Ganesan et al COLING’10, Ganesan et al WWW’12] http://www.findilike.com/demo.jsp

Summary generated on reviews of Acura 2007 from Edmunds.com

-Concise and readable-Picks up on aspects naturally-Contains important details -Displays how many people said it / how many times appeared

Micropinion Summaries [Ganesan et al COLING’10, Ganesan et al WWW’12] http://www.findilike.com/demo.jsp

-display snippets-display whole passages-up to application

Highly flexible• Adjust summaries to screen size

longer summaries for larger screens shorter summaries for smaller screens

• Limiting duplicates, increasing diversity

• Don’t need to know aspects/features in advance Generate summaries for arbitrary aspects You can limit to certain aspects if needed (e.g. battery, sound)

• Recent user study showed with micro-summaries:[Yatani et al CHI’11] Users took significantly less time to decide on a restaurant

compared to reading full reviews Micro-summaries shown to be effective in decision making

Micropinion Summaries

Practical Implementation• Not too hard to implement with n-gram methods • More sophisticated approaches – keyphrase extraction,

multi-sentence compression [Fillipova COLING’10] [Ganesan et al COLING’10] [Ganesan et al WWW’12] [Boudin & Morin 2013]

Issues to consider:• Simple methods: chances of generating junk• E.g. iPhone 5s:

“the battery life is”, “the iPhone 5s” meaningless to user• You want: “battery life is short”, “iPhone 5s is reliable”

Micropinion Summaries

Comparative summaries• compare contradicting opinions

Entity based summarization

Other summarization formats

Decision making tools shown:• Are powerful, but have their limitations • Just as in any new state-of-the art methods

Pair techniques wisely + take edge cases into consideration powerful solution!

How FindiLike Hotel Search uses opinion-driven decision making tools

Example Implementation

Search system: Helps users find hotels by preferences• Unstructured opinion preferences

“friendly service”, “clean”, “good views” • Common structured preferences

price [$0-$100], distance [5 miles from campus]

Beyond search: Support for analysis of hotels• Micropinion summaries • Multi-word buzz phrases of reviews

Developed to showcase published ideas: related to enabling opinion-driven decision making [Ganesan et al COLING’10, Ganesan & Zhai IRJ, Ganesan & Zhai WWW’12, Ganesan 2013 PhD. Thesis]

FindiLike Hotel Search

Let us say…

• Visiting: Los Angeles• Main opinion criteria: hotels said to be clean & safe

Input requirements as natural keywords

Hotels ranked by how well preferences are matched in opinions

• Preferences entered• How well preferences

matched (stars)• Snippets to show why hotel

was selected

More reviews on why the hotel matched ‘clean’

• Can add more preferences: ‘cheap’, ‘good room service’

• Remove preferences • Combine with structured

preferences – price, distance

Benefits of opinion-driven search: • Limits number of hotels in consideration • No need to read reviews to find hotels matching

preferences

Micropinion summaries:• Concise opinion summaries • Highlights key opinions about

hotel

Benefit of micropinion summaries: • Helps understand underlying opinions within reviews• Further refine choices by knowing key opinions

Benefit of multi-word buzz phrase: • Helps users explore the opinion space. Learning

through exploration• Extremely suitable when opinions are sparse

Multi-word buzz phrases:• Highlights common phrases • Weighted by frequency and

readability

snippets related to “parking lot”

Benefit of multi-word buzz phrase: • Helps users explore the opinion space. Learning

through exploration• Extremely suitable when opinions are sparse

Improves user productivity • Eliminates need to read large number of opinions

Don’t need to perform own data mining

Helps user retention• Users can make better, faster and more informed

decisions on your site Likely to come back

Improves conversion rates• Users can make faster decisions

More likely to complete transaction on your site

The need for opinion-driven applications

Are users asking for this? Yes!

QUORA: Are there any service to summarize yelp reviews? I want to check the reviews of a roofing company but there are hundreds of them. Can I get some kind of summary (and maybe bar graph) without reading all of them?

Contact: Kavita Ganesan Email: kganes@findilike.comPersonal web: kavita-ganesan.comCompany web: findilike.com

Thank you! Questions?