sentiment analysis

Character-basedHybridSentimentAnalysisJITINDUA,PRANAVDIXIT

Whysentimentanalysis?◦ Movie:Isthisreviewpositiveornegative?◦ Products:WhatdopeoplethinkaboutthenewiPhone?◦ Publicsentiment:Howisconsumerconfidence?◦ Politics:WhatdopeoplethinkaboutTrump?◦ Prediction:Predictmarkettrends

ChallengesØ Shortinlength- lesscontextualdatatopredictthesentiment

Ø Informalwithmisspellings(oftenintentional,likedifferentspellingsof”cool”:coool,coolll,coooolll!!),newwords,slangs,andURLs

Ø Newwords,newtrendsinusingabbreviations- leadstotheproblemofout-of-vocabularywords

Ø Specialsymbolslikeemoticonsandhashtags

RelatedWork:1. RecurrentNeuralNetwork

a. Wordbasedb. Characterbased

2. ConvolutionalNeuralNetworka. Wordbasedb. Characterbased

3. Hybrid(CNN-RNN)a. Wordbasedb. Characterbased(ourapproach)

RNNbasedmodelsRNNtakeswordsinasentenceinasequentialorderandisabletolearnthelong-termdependencies oftextsratherthanlocalfeatures.

CNNbasedmodelsSimilartothetranslation,rotationandscaleinvarianceofimagesinCNN,CNNisabletolearnthelocalfeaturesfromwordsorphrasesindifferentplacesoftexts.

Usesdifferentsizefilterstogetn-grams,ex.unigram,bigram,etc.◦ contiguoussequenceofnitemsfromagivensequence

Example:“thecountryofmybirth”computesn-gramsasfollows:

thecountry,countryof,ofmy,mybirth,thecountryof,countryofmy,ofmybirth,thecountryofmy,countryofmybirth

Kim, Y. Convolutional Neural Networks for Sentence Classification. In Proceedings of EMNLP. 2014.

CNNbasedmodels

Kim, Y. Convolutional Neural Networks for Sentence Classification. In Proceedings of EMNLP. 2014

WordBasedWordEmbedding

Ø Expressesawordasarealnumbervectoroflowdimension(about200dimensionsorhigher)

Ø Wordswithsimilarmeaningcorrespondstoclosevectorandobtainmeaningfulresults(e.g.king– man+women=queen)byadding(subtracting)vectors.

Ø Types:Ø TrainfromdataØ Pre-trainedmodels(static,non-static)

Ø Popularpre-trainedmodels:Ø Googleword2vecØ Glove

http://suriyadeepan.github.io/2016-06-28-easy-seq2seq/

Wordbasedmodels:Issues•Wordbasedmodelsareblindtosub-wordinformation(e.g.morphemes– eventful,eventfully,uneventful)

•Mostofthetrainingparametersarefromthewordembedding

•Word2Vecisusuallyverybiginsize• Googleword2vec:3.5GB

•Wordbasedmodelscannothandle• Newwords• Typos

CharCNNØ Detectssub-wordinformation(e.g.morphemes)

Ø Doesnotrequiremorphologicaltaggingasapre-processingstep

Ø Handlestypos,slangsandnewwords

Ø Asmostoftheparametersarefromthewordembeddings,theproposedmodelhassignificantlyfewerparametersØ Makesitattractiveforapplicationswheremodelsizemaybeanissue(e.g.cellphones).

CharCNNCharacterbasedwordembedding

Ø Createsaword-levelembeddingusingone-hotrepresentationofcharacters

Ø Usesmultiplefiltersofvaryingwidthstoobtainthefeaturevectorforaword

Ø EachfilteroftheCharCNN isessentiallylearningtodetectparticularcharactern-grams

Ø IfwehaveatotalofhfiltersH1,...,Hh,thenyk =[y1k,...,yhk ]istheinputrepresentationofwordk.

YoonKim,Yacine Jernite,DavidSontag,andAlexanderMRush.Character-awareneurallanguagemodels.(AAAI2016)

z = t ⊙ g(WH y + bH ) + (1 − t) ⊙ y

t = σ(WTy + bT )

YoonKim,Yacine Jernite,DavidSontag,andAlexanderMRush.Character-awareneurallanguagemodels.(AAAI2016)

HybridapproachCNN+ RNNarchitecture

Takesadvantageofthecoarse-grainedlocalfeaturesgeneratedbyCNNandlong-distancedependencies learnedviaRNNforsentimentanalysis.

Word2Vec CNN(localfeatures)

RNN(long-distancedependencies)

Words Sentiment

Wang,Jiang,Luo.CombinationofConvolutionalandRecurrentNeuralNetworkforSentimentAnalysisofShortTexts .Dec2016.

NovelapproachCharCNN +Hybrid-CNN-RNN

Ø Advantagesofcharacterbasedmodels

Ø Coarse-grainedlocalfeaturesgeneratedbyCNN

Ø Long-distancedependencies learnedviaRNNforsentimentanalysis

CharCNN(word-embedding)

CNN(localfeatures)

RNN(long-distancedependencies)

Characters Sentiment

Futureofsentimentanalysis:

BetterT-800!..

References:1. YoonKim,Yacine Jernite,DavidSontag,andAlexanderMRush.2016.Character-awareneural

languagemodels.(AAAI2016)

2. X Wang,WJiang,ZLuo.2016.Combinationof Convolutional andRecurrent NeuralNetworkforSentimentAnalysisof ShortTexts.

3. YXiao,KCho.2016.Efficient Character-level Document Classification byCombiningConvolution andRecurrent Layers.arXiv preprint arXiv:1602.00367.

4. Kim,Y.2014.Convolutional NeuralNetworksforSentence Classification.InProceedings ofEMNLP.

Thanks..

sentiment analysis

Science