redefining sentiment analysis

Download Redefining Sentiment Analysis

Post on 26-Jan-2015

103 views

Category:

Technology

0 download

Embed Size (px)

DESCRIPTION

Why current sentiment analysis theory and practice is fundamentally flawed and what to do about it. Talk at the research session of the Sentiment Symposium in San Francisco on Nov 8, 2011. Based on joint work with Dr. Ken Bloom.

TRANSCRIPT

  • 1. Redening(Sentiment(Analysis1 Shlomo ArgamonSubtext3, Inc. Illinois Institute of Technology

2. Strong(foundations1Sentiment Analysis Symposium, San Francisco 8 November, 2011 3. are(critical!1Sentiment Analysis Symposium, San Francisco 8 November, 2011 4. The(Archetype1Review classification the earliest, and still stereotypical sentiment analysis application5.0$out$of$5$stars$Great!11.0$out$of$5$stars$Buy*cheap,*buy*twice2This(Nikon(is(great.(It(takes(perfect(Im(absolutely(livid(with(myself(for(buying(this(pictures(and(I(tried(the(shaking(eect;(it( camera.(IFve(gone(through(a(divorce(and(been(left(was(the(bom.(I(shaked(the(camera(while( with(nothing((including(the(loss(of(my(SLR(taking(a(picture(and(the(picture(came(cameras)(and(this(was(the(one(I(could(aord(that(out(picture(perfect.((promised(to(be(the(most(similar(to(the(SLRs.(I(have(Great(product!(1a(2(year(old(son,(and(armed(with(only(this(camera,(I(have(missed(documenting(the(last(6(months(of(his(5/5$stars$$.time(with(me.(Every(picture(is(blurredNN(granted(Opera*with*subtitles?**Who*knew?* heFs(2(and(moves(a(lot,(but(even(the(slightest(I(had(the(great(fortune(to(aZend(a(dress( movement(by(the(photographer(blurs(the(image.(rehersal(for(The(Damnation(of(Faust.((Blurred.(EVERYTHING.(I(wish(I(had(saved(up(and(This(updated(version(was(amazing.((1bought(something(decent.(I(will(take(my(loss(on(A(M(A(Z(I(N(G.((( this(one(for(an(Amazon(gift(card,(but(DONFT(The(theatre(is(gorgeous.((The(acoustics(WASTE(YOUR(MONEY.(*sigh*(1are(incredible.(Go.((Go(now.1Sentiment Analysis Symposium, San Francisco8 November, 2011 5. Assumptions1Text(has(a(single(main(sentiment( about(a(single(main(topic1Sentiment(is(graded(on(a(linear(scale1Usually$positive/negative.Words(carry(sentiment(information1 Analysis(=(Classication1 Very(liZle(semantics((if(any)1Sentiment Analysis Symposium, San Francisco8 November, 2011 6. Lets*look*a*li=le*more*closely2Sentiment Analysis Symposium, San Francisco 8 November, 2011 7. Do(they(t(the(archetype?1This(setup(using(the(CD(was(about(as(easy(as(learning(how(to(open(a(refrigerator(door(for(the(rst(time.(1 Easy2 Sentiment Analysis Symposium, San Francisco 8 November, 2011 8. Do(they(t(the(archetype?1This(setup(using(the(CD(was(about(as(easy(as(opening(a(wooden(crate(with(your(ngernails.(1 Hard2 Sentiment Analysis Symposium, San Francisco 8 November, 2011 9. Do(they(t(the(archetype?1This(setup(using(the(CD(was(about(as(easy(as(learning(This(setup(using(the(CD(was(about(as(easy(as(how(to(open(a(refrigerator(door(for(the(rst(time.(1opening(a(wooden(crate(with(your(ngernails.(1There(are(a(few(extremely(sexy(new(features(in(Final*Cut*Pro*7.(1 Sentiment Analysis Symposium, San Francisco 8 November, 2011 10. Do(they(t(the(archetype?1This(setup(using(the(CD(was(about(as(easy(as(learning(This(setup(using(the(CD(was(about(as(easy(as(how(to(open(a(refrigerator(door(for(the(rst(time.(1opening(a(wooden(crate(with(your(ngernails.(1There(are(a(few(extremely(sexy( but(its(generally(lousy?1new(features(in(Final*Cut*Pro*7.(1 Sentiment Analysis Symposium, San Francisco8 November, 2011 11. Do(they(t(the(archetype?1This(setup(using(the(CD(was(about(as(easy(as(learning(This(setup(using(the(CD(was(about(as(easy(as(how(to(open(a(refrigerator(door(for(the(rst(time.(1opening(a(wooden(crate(with(your(ngernails.(1There(are(a(few(extremely(sexy( The(rest(of(it,(however,(1new(features(in(Final*Cut*Pro*7.(1 Sentiment Analysis Symposium, San Francisco8 November, 2011 12. Do(they(t(the(archetype?1This(setup(using(the(CD(was(about(as(easy(as(learning(This(setup(using(the(CD(was(about(as(easy(as(how(to(open(a(refrigerator(door(for(the(rst(time.(1opening(a(wooden(crate(with(your(ngernails.(1There(are(a(few(extremely(sexy(new(features(in(Final(Cut(Pro(7.(1When(the(tent(was(nally(completed,(the( Positive?2result(was(much(nicer(than(that(Kiva(tent((1 1 Sentiment Analysis Symposium, San Francisco 8 November, 2011 13. Do(they(t(the(archetype?1This(setup(using(the(CD(was(about(as(easy(as(learning(This(setup(using(the(CD(was(about(as(easy(as(how(to(open(a(refrigerator(door(for(the(rst(time.(1opening(a(wooden(crate(with(your(ngernails.(1There(are(a(few(extremely(sexy(new(features(in(Final(Cut(Pro(7.(1When(the(tent(was(nally(completed,(the( Negative?2result(was(much(nicer(than(that(Kiva(tent( but(not(as(nice(as(the(Arctic(oven1 Sentiment Analysis Symposium, San Francisco 8 November, 2011 14. Do(they(t(the(archetype?1This(setup(using(the(CD(was(about(as(easy(as(learning(This(setup(using(the(CD(was(about(as(easy(as(how(to(open(a(refrigerator(door(for(the(rst(time.(1opening(a(wooden(crate(with(your(ngernails.(1There(are(a(few(extremely(sexy(new(features(in(Final(Cut(Pro(7.(1When(the(tent(was(nally(completed,(the(result(was(much(nicer(than(that(Kiva(tent( but(not(as(nice(as(the(Arctic(oven1Jurassic(Park(was(a(great(book(Positive?2but(a(lousy(movie.(1 Negative?2 Sentiment Analysis Symposium, San Francisco 8 November, 2011 15. Do(they(t(the(archetype?1This(setup(using(the(CD(was(about(as(easy(as(learning(This(setup(using(the(CD(was(about(as(easy(as(how(to(open(a(refrigerator(door(for(the(rst(time.(1opening(a(wooden(crate(with(your(ngernails.(1There(are(a(few(extremely(sexy(new(features(in(Final(Cut(Pro(7.(1When(the(tent(was(nally(completed,(the(result(was(much(nicer(than(that(Kiva(tent( but(not(as(nice(as(the(Arctic(oven1Jurassic(Park(was(a(great(book(but(a(lousy(movie.(1Loud(but(nice(room(at(a(great( Generally*positive2 price(with(bad(view.1 Sentiment Analysis Symposium, San Francisco 8 November, 2011 16. Do(they(t(the(archetype?1This(setup(using(the(CD(was(about(as(easy(as(learning(This(setup(using(the(CD(was(about(as(easy(as(how(to(open(a(refrigerator(door(for(the(rst(time.(1opening(a(wooden(crate(with(your(ngernails.(1There(are(a(few(extremely(sexy(new(features(in(Final(Cut(Pro(7.(1When(the(tent(was(nally(completed,(the(result(was(much(nicer(than(that(Kiva(tent( but(not(as(nice(as(the(Arctic(oven1Jurassic(Park(was(a(great(book(but(a(lousy(movie.(1Loud(but(nice(room(at(a(great( Negative*on*sound2 price(with(bad(view.1 Sentiment Analysis Symposium, San Francisco 8 November, 2011 17. Do(they(t(the(archetype?1This(setup(using(the(CD(was(about(as(easy(as(learning(This(setup(using(the(CD(was(about(as(easy(as(how(to(open(a(refrigerator(door(for(the(rst(time.(1opening(a(wooden(crate(with(your(ngernails.(1There(are(a(few(extremely(sexy(new(features(in(Final(Cut(Pro(7.(1When(the(tent(was(nally(completed,(the(result(was(much(nicer(than(that(Kiva(tent( but(not(as(nice(as(the(Arctic(oven1Jurassic(Park(was(a(great(book(but(a(lousy(movie.(1Loud(but(nice(room(at(a(great( Positive*on*price2 price(with(bad(view.1 Sentiment Analysis Symposium, San Francisco 8 November, 2011 18. Do(they(t(the(archetype?1This(setup(using(the(CD(was(about(as(easy(as(learning(This(setup(using(the(CD(was(about(as(easy(as(how(to(open(a(refrigerator(door(for(the(rst(time.(1opening(a(wooden(crate(with(your(ngernails.(1There(are(a(few(extremely(sexy(new(features(in(Final(Cut(Pro(7.(1When(the(tent(was(nally(completed,(the(result(was(much(nicer(than(that(Kiva(tent( but(not(as(nice(as(the(Arctic(oven1Jurassic(Park(was(a(great(book(but(a(lousy(movie.(1Loud(but(nice(room(at(a(great( Negative*on*view2 price(with(bad(view.1 Sentiment Analysis Symposium, San Francisco8 November, 2011 19. Does(it(maZer?1 Pa=ern2Results*(Google)2 Percentage2X is good 4,820,000,000X is as good as Y 2,530,000,00052%X is better than Y2,120,000,00044%X is bad1,730,000,000X is worse than Y 437,000,00025%X is a good Y14,160,000,000X is a better Y than a Z781,000,000 6%X at a good price 888,000,000X at a good price but ... bad Y29,400,000 3% Sentiment Analysis Symposium, San Francisco8 November, 2011 20. Annotation(oddities1We(believe(in(the(sincerity(of(the(United(States(in(promising(not(to( mix(up(its(counterNterrorism(drive(with(the(Taiwan(Strait(issue1AZitude1 the(United(States(1the(sincerity(of(the(United(States(in(promising(not(to(mix(up(its(counterNterrorism(drive(with( the(Taiwan(Strait(issue1Sentiment Analysis Symposium, San Francisco 8 November, 2011 21. Annotation(oddities1what(is(important(is(that(your(Internet(connection(will(never(even(reach(the(speed(capabilities(of(this(router...1AZitude1 Topic1 important1your(Internet(connection(will(never(even( reach(the(speed(capabilities(of(this(router1Sentiment Analysis Symposium, San Francisco8 November, 2011 22. Annotation(oddities1Itll(make(life(a(lot(easier,(and(preclude(you(from(having(to(give( this(product(a(negative(review.((1 AZitude1Topic1 positive1product1Sentiment Analysis Symposium, San Francisco8 November, 2011 23. Annotation(oddities1However,(this(product(performed(well(below(my(expectation.((1 AZitude1Topic1 negative1 router1Sentiment Analysis Symposium, San Francisco 8 November, 2011 24. So(whats(really(going(on?1 Her(appearance(and(demeanor(are(excellently(suited(to(her(role.1 Positive2 SheFs(the(most(heartless(coqueZe(in(France!1Negative2 Classication+Sentiment Analysis Symposium, San Francisco 8 November, 2011 25. The(Structure(of(Sentiment1 Her(appearance(and(demeanor(are(excellently(suited(to(her(role.1 Positive2 SheFs(the(most(heartless(coqueZe(in(France!1Negative2 A-itude+Sentiment Analysis Symposium, San Francisco 8 November, 2011 26. The(Structure(of(Sentiment1 Her(appearance*and*demeanor*are(excellently(suited(to(her(role.1 SheFs(the(most(heartless(coqueZe(in(France!1 Target+Sentiment Analysis Symposium, San Francisco 8 November, 2011 27. The(Structure(of(Sentiment1 Her(appearance*and*demeanor*are(excellently(suited(to(her(role.1 SheFs(the(most(heartless(coqueZe(in(France!1Superordinate+Sentiment Analysis Symposium, San Francisco 8 November, 2011 28. The(Structure(of(Sentiment1 Her*appearance*and*demeanor*are(excellently(suited(to(her(role.1 SheFs(the(most(heartless(coqueZe(in(France!1 Aspect+Sentiment Analysis Symposium, San Francisco 8 November, 2011 29. The(Sentiment(Complex1target11aZitude1target21the(result(was