senticircles for contextual and conceptual semantic sentiment analysis of twitter
DESCRIPTION
Lexicon-based approaches to Twitter sentiment analysis are gaining much popularity due to their simplicity, domain independence, and relatively good performance. These approaches rely on sentiment lexicons, where a collection of words are marked with fixed sentiment polarities. However, words' sentiment orientation (positive, neural, negative) and/or sentiment strengths could change depending on context and targeted entities. In this paper we present SentiCircle; a novel lexicon-based approach that takes into account the contextual and conceptual semantics of words when calculating their sentiment orientation and strength in Twitter. We evaluate our approach on three Twitter datasets using three different sentiment lexicons. Results show that our approach significantly outperforms two lexicon baselines. Results are competitive but inconclusive when comparing to state-of-art SentiStrength, and vary from one dataset to another. SentiCircle outperforms SentiStrength in accuracy on average, but falls marginally behind in F-measure.TRANSCRIPT
SentiCirclesfor Contextual and Conceptual Semantic Sentiment Analysis of
Hassan Saif, Miriam Fernandez, Yulan He and Harith Alani
The Eleventh Extended Semantic Web Conference (ESWC2014)May 2014
OutLine
oSentiment Analysis
oApproaches
oSentiCircles
oEvaluation
oConclusion
“Sentiment analysis is the task of identifying positive and negative opinions, emotions and evaluations in text”
3
Opinion OpinionFact
Sentiment Analysis
yes, It is sunny, but also very humid :(
The weather is great today :)
I think its almost 30 degrees today
Sentiment Analysis
oRich
o Formal Language {Well
Structured Sentences}
oDomain Specific
Conventional Text
Twitter Data
o Short (140-Chars)
oNoisy {gr8, lol, :), :P}
oOpen Environment
Sentiment Analysis
Approaches
Lexicon-Based Approach
Machine Learning Approach
Machine LearningAp
proa
ch
Lexicon-based Ap
proa
ch
I had nightmares all night long last night :(
Negative
Sentiment Lexicon
Text Processing Algorithm
great successsad
pretty
down
wronghorrible
beautiful
mistake
love
good
o Requires Labeled Twitter Corpora Labor Intensive TaskDistant Supervision (Noisy Labeling)
o Domain Specific Re-Training with new domains
Machine Learning Approach
On Twitter?
Traditional Lexicons- MPQA & SentiwordNet, etc
- Not tailored to Twitter noisy data:- lol, gr8, wow, :), :P
- Fixed number of words
Lexicon-based ApproachOn Twitter?
Sentiment Lexicon
great successsad
pretty
down
wronghorrible
beautiful
mistake
love
good
grt8lol:)
:P
Twitter-specific Lexicon-based Methods
- Such as SentiStrength- Rule-base method for sentiment analysis
on social web
- Uses Thelwall-Lexicon- Built to specifically work on social data - Contain lists of emoticons, slangs, abbreviations,
etc.
• Fixed Number of words
• Offer Context-Insensitive Prior Sentiment Orientations and Strength of words
Great
Problem Smile
Positive
Thelwall-Lexicon & SentiStrength
Sentiment Lexicon
great successsad
pretty
down
wronghorrible
beautiful
mistake
love
good
We Need.. Unsupervised Approach
Understands the Semantic of Words
Captures their Contexts
Updates Sentiment
SentiCircles
SentiCircles
Lexicon-based Approach
Builds Dynamic representation of words
Captures Contextual & Conceptual Semantics of words
Updates words’ sentiment orientation and strength accordingly
Contextual Semantics “Words that occur in similar context tend to have similar meaning”
Wittgenstein (1953)
“You Shall know the word by the company it keeps”Firth (1955)
GreatProblem
Look SmileConcert
Song
WeatherLoss
Game Taylor Swift
AmazingGreat
Capturing Contextual Semantics
Term (m) C1 C2 Cn….
Context-Term Vector
Degree of Correlation
Prior SentimentSentiment Lexicon
(1)
(2)Great
Smile Look
(3)
Contextual Sentiment Strength
Contextual Sentiment Orientation
Positive, Negative Neutral
[-1 (very negative)+1 (very positive)]
Term (m) C1
Degree of Correlation
Prior Sentiment
Great
Smile
SentiCircles Model
X = R * COS(θ)
Y = R * SIN(θ)
Smile
X
ri
θi
xi
yi
Great
PositiveVery Positive
Very Negative Negative
+1
-1
+1-1 Neutral Region
ri = TDOC(Ci)
θi = Prior_Sentiment (Ci) * π
Capturing Contextual Semantics
SentiCircles (Example)
Overall Contextual Sentiment
Ci
X
ri
θi
xi
yi
m
PositiveVery Positive
Very Negative Negative
+1
-1
+1-1 Neutral Region
Senti-Median of SentiCircle
Sentiment Function
SentiCircles & Conceptual Semantics
Enriching SentiCircles with Conceptual Semantics
Sushi time for fabulous Jesse's last day on dragons den
@Stace_meister Ya, I have Rugby in an hour
Dear eBay, if I win I owe you a total 580.63 bye paycheckCompany
Person
Sport
Enriching SentiCircles with Conceptual Semantics
Cycling under a heavy rain.. What a #luck!
Weather Condition
Wind
Snow
Humidity
SentiCircles for
Tweet-level Sentiment Analysis
Detecting the overall Sentiment of a given tweet message (positive vs. negative)
SentiCircles forTweet-level Sentiment Analysis
(1) The Median Method
Cycling under a heavy rain.. what a #luck!
S-Median S-Median S-Median S-Median S-Median S-Median
The Median of Senti-Medians
Tweet-level Sentiment Analysis
(2) The Pivot Methodlike1
X
Yr1
θ1
PositiveVery Positive
Very Negative Negative
new2
pj r2
θ2
like1 new2 iPadj Wn
Sj1
Sj2
Tweet tk
...
I like my new iPad
Experiments
Experimental Setup
(1) Datasets
(2) Sentiment Lexicons- SentiWordNet [3]- MPQA Subjectivity Lexicon [4]- Thelwall-Lexicon [5]
Experimental Setup
(3) Baselines
1. Lexicon-Labeling (MPQA & SentiWordNet)Average of positive & negative words in a tweet.
2. SentiStrength (State-of-the-art)- Lexicon-based method built for Twitter- Apply a set of syntactic rules
Results
Sentiment Detection with Contextual Semantics
SentiCircles vs. Lexicon-Labeling Methods
MPQA-Lex SentiWNet-Lex SentiCircle40.00
45.00
50.00
55.00
60.00
65.00
70.00
75.00
80.00
52.35 52.74
74.96
52.34 52.30
68.06
Accuracy F-Measure
SentiCircle vs. SentiStrengthDatasets Accuracy F1
OMD SentiCircle SentiCircle
HCR SentiCircle SentiStrength
STS-Gold SentiStrength SentiStrength
Average SentiCircle SentiStrength
Why Such Variance..• The sentiment class distribution in our datasets– SentiCircle produces, on average, 2.5% lower recall
than SentiStrength on positive tweet detection– Our datasets contain more negative tweets than
positive ones
• Topic Distribution in the three datasets
• More research is required
Sentiment Detection with Conceptual Semantics
Win/Loss in Accuracy and F-measure of incorporating conceptual semantics into SentiCircles, where Mdn:
SentiCircle with Median method, Pvt: SentiCircle with Pivot method.
Conclusion• We proposed a novel semantic sentiment approach called
SentiCircle
• SentiCircles captures context and update sentiment accordingly
• We showed how SentiCircle can be applied for Tweet-level sentiment analysis
• SentiCircles outperformed other lexicon labeling methods and overtake the state-of-the-art SentiStrength approach in accuracy, with a marginal drop in F-measure.
SentiCircles for Sentiment Analysis
1. Tweet-level Sentiment Analysis
2. Entity-Level Sentiment Analysis
3. Sentiment Lexicon Adaptation
4. Dynamic Stopwords Generation
5. Sentiment Patterns Discovery
Saif et al. (2014) at ESWC Conference. Greece, Crete
Saif et al. (2014), IPM Journal
Saif et al. (2014) at ESWC Conference
Saif et al. (2014) at LREC Conference. Reykjavik, Iceland
Saif et al. (2014) submitted to ISWC Conference.
Thank YouEmail: [email protected]: hrsaifWebsite: tweenator.com