sentiment of emojis -...
TRANSCRIPT
![Page 1: Sentiment of Emojis - VideoLectures.nettranslectures.videolectures.net/site/normal_dl/tag=1065925/single... · Why Sentiment of Emojis? •As features for sentiment models •As labels](https://reader033.vdocuments.mx/reader033/viewer/2022052916/5a7924747f8b9a00168dac9e/html5/thumbnails/1.jpg)
Sentiment of Emojis Petra Kralj Novak
4th Conference on CMC and Social Media Corpora for the Humanities
![Page 2: Sentiment of Emojis - VideoLectures.nettranslectures.videolectures.net/site/normal_dl/tag=1065925/single... · Why Sentiment of Emojis? •As features for sentiment models •As labels](https://reader033.vdocuments.mx/reader033/viewer/2022052916/5a7924747f8b9a00168dac9e/html5/thumbnails/2.jpg)
- EMOJI FACTS - OPINION MINING - EMOJI SENTIMENT
Outline
2
![Page 3: Sentiment of Emojis - VideoLectures.nettranslectures.videolectures.net/site/normal_dl/tag=1065925/single... · Why Sentiment of Emojis? •As features for sentiment models •As labels](https://reader033.vdocuments.mx/reader033/viewer/2022052916/5a7924747f8b9a00168dac9e/html5/thumbnails/3.jpg)
Emoticon • Facial expression composed of
keyboard characters • Emotion + icon (English)
O:-) Angel X-( Angry ~:0 Baby :-D Big Grin :-# Braces </3 Broken Heart =^.^= Cat *<:o) Clown O.o Confused B-) Cool :'( Crying …
Emoji • Pictographs (pictorial symbols): things,
emotions, activities, …
• Presented in a colorful form
• E ‘picture’ + moji ‘letter, character’ (Japanese)
3
![Page 4: Sentiment of Emojis - VideoLectures.nettranslectures.videolectures.net/site/normal_dl/tag=1065925/single... · Why Sentiment of Emojis? •As features for sentiment models •As labels](https://reader033.vdocuments.mx/reader033/viewer/2022052916/5a7924747f8b9a00168dac9e/html5/thumbnails/4.jpg)
Unicode® Emoji
• Single Unicode characters emoji counts
• Many more emojis are composed of sequences of two or more characters (e.g. regional indicators in pairs represent ≈ flags)
4
Version Emoji count Year 1.0.0 76 (+76) 1991 3 78 (+2) 1999 3.2 86 (+8) 2002 4 94 (+8) 2003 4.1 108 (+14) 2005 5.1 112 (+4) 2008 5.2 139 (+27) 2009 6 855 (+716) 2010 6.1 868 (+13) 2012 7 972 (+104) 2014 8 1,013 (+41) 2015 9 1,085 (+72) 2016
Emojis were first defined in Unicode 6.0, and pre-6.0 characters were only defined as emoji in 6.0 or later.
![Page 5: Sentiment of Emojis - VideoLectures.nettranslectures.videolectures.net/site/normal_dl/tag=1065925/single... · Why Sentiment of Emojis? •As features for sentiment models •As labels](https://reader033.vdocuments.mx/reader033/viewer/2022052916/5a7924747f8b9a00168dac9e/html5/thumbnails/5.jpg)
167 new Emoji Unicode 9.0.0 (July 2016)
5
![Page 6: Sentiment of Emojis - VideoLectures.nettranslectures.videolectures.net/site/normal_dl/tag=1065925/single... · Why Sentiment of Emojis? •As features for sentiment models •As labels](https://reader033.vdocuments.mx/reader033/viewer/2022052916/5a7924747f8b9a00168dac9e/html5/thumbnails/6.jpg)
Sequence of Emojis
6
![Page 7: Sentiment of Emojis - VideoLectures.nettranslectures.videolectures.net/site/normal_dl/tag=1065925/single... · Why Sentiment of Emojis? •As features for sentiment models •As labels](https://reader033.vdocuments.mx/reader033/viewer/2022052916/5a7924747f8b9a00168dac9e/html5/thumbnails/7.jpg)
Unicode® Emoji
7
Version Emoji count Year 1.0.0 76 (+76) 1991 3 78 (+2) 1999 3.2 86 (+8) 2002 4 94 (+8) 2003 4.1 108 (+14) 2005 5.1 112 (+4) 2008 5.2 139 (+27) 2009 6 855 (+716) 2010 6.1 868 (+13) 2012 7 972 (+104) 2014 8 1,013 (+41) 2015 9 1,085 (+72) 2016
Emojis were first defined in Unicode 6.0, and pre-6.0 characters were only defined as emoji in 6.0 or later.
![Page 8: Sentiment of Emojis - VideoLectures.nettranslectures.videolectures.net/site/normal_dl/tag=1065925/single... · Why Sentiment of Emojis? •As features for sentiment models •As labels](https://reader033.vdocuments.mx/reader033/viewer/2022052916/5a7924747f8b9a00168dac9e/html5/thumbnails/8.jpg)
Emoji Popularity:
8 http://instagram-engineering.tumblr.com/post/117889701472/emojineering-part-1-machine-learning-for-emoji
![Page 9: Sentiment of Emojis - VideoLectures.nettranslectures.videolectures.net/site/normal_dl/tag=1065925/single... · Why Sentiment of Emojis? •As features for sentiment models •As labels](https://reader033.vdocuments.mx/reader033/viewer/2022052916/5a7924747f8b9a00168dac9e/html5/thumbnails/9.jpg)
Why do we Emoji?
Oxford Dictionaries named (Face With Tears of Joy) its 2015 Word of the year.
9
![Page 10: Sentiment of Emojis - VideoLectures.nettranslectures.videolectures.net/site/normal_dl/tag=1065925/single... · Why Sentiment of Emojis? •As features for sentiment models •As labels](https://reader033.vdocuments.mx/reader033/viewer/2022052916/5a7924747f8b9a00168dac9e/html5/thumbnails/10.jpg)
10 Mehrabian, Albert (1971). Silent Messages (1st ed.). Belmont, CA: Wadsworth.ISBN 0-534-00910-7.
![Page 11: Sentiment of Emojis - VideoLectures.nettranslectures.videolectures.net/site/normal_dl/tag=1065925/single... · Why Sentiment of Emojis? •As features for sentiment models •As labels](https://reader033.vdocuments.mx/reader033/viewer/2022052916/5a7924747f8b9a00168dac9e/html5/thumbnails/11.jpg)
Emoticons in Mind
When we look at a smiley face, the same parts of the brain are activated as when we look at a real human face.
11 Churches O, Nicholls M, Thiessen M, Kohler M, Keage H (2014) Emoticons in mind: An event-related potential study. Social Neuroscience
9(12)doi:10.1080/17470919.2013.873737
![Page 12: Sentiment of Emojis - VideoLectures.nettranslectures.videolectures.net/site/normal_dl/tag=1065925/single... · Why Sentiment of Emojis? •As features for sentiment models •As labels](https://reader033.vdocuments.mx/reader033/viewer/2022052916/5a7924747f8b9a00168dac9e/html5/thumbnails/12.jpg)
- EMOJI FACTS - OPINION MINING - EMOJI SENTIMENT
Outline
12
![Page 13: Sentiment of Emojis - VideoLectures.nettranslectures.videolectures.net/site/normal_dl/tag=1065925/single... · Why Sentiment of Emojis? •As features for sentiment models •As labels](https://reader033.vdocuments.mx/reader033/viewer/2022052916/5a7924747f8b9a00168dac9e/html5/thumbnails/13.jpg)
Opinion Mining
• Opinion mining (or sentiment analysis) is the computational study of people’s opinions, sentiments, emotions, and attitudes.
13
Approaches
Machine learning
Large corpus annotated with
sentiment
Lexical
Sentiment lexicon
Hybrid
Annotated corpus
Sentiment lexicon
Semantic parsing
![Page 14: Sentiment of Emojis - VideoLectures.nettranslectures.videolectures.net/site/normal_dl/tag=1065925/single... · Why Sentiment of Emojis? •As features for sentiment models •As labels](https://reader033.vdocuments.mx/reader033/viewer/2022052916/5a7924747f8b9a00168dac9e/html5/thumbnails/14.jpg)
Sentiment Lexicons
• Lists of words associated to scores of sentiment polarity
• Well developed for English
– Bing Liu's Opinion Lexicon
– MPQA Subjectivity Lexicon
– Harvard General Inquirer
– SentiWordNet assigns to each synset of WordNet three sentiment scores: positivity, negativity, objectivity.
14 Baccianella S, Esuli A, Sebastiani F. "SentiWordNet 3.0: An Enhanced Lexical Resource for Sentiment Analysis and Opinion Mining.“
LREC. Vol. 10. 2010.
![Page 15: Sentiment of Emojis - VideoLectures.nettranslectures.videolectures.net/site/normal_dl/tag=1065925/single... · Why Sentiment of Emojis? •As features for sentiment models •As labels](https://reader033.vdocuments.mx/reader033/viewer/2022052916/5a7924747f8b9a00168dac9e/html5/thumbnails/15.jpg)
- EMOJI FACTS - OPINION MINING - EMOJI SENTIMENT
Outline
15 Kralj Novak P, Smailović J, Sluban B, Mozetič I (2015) Sentiment of Emojis. PLoS ONE 10(12): e0144296.
doi:10.1371/journal.pone.0144296
![Page 16: Sentiment of Emojis - VideoLectures.nettranslectures.videolectures.net/site/normal_dl/tag=1065925/single... · Why Sentiment of Emojis? •As features for sentiment models •As labels](https://reader033.vdocuments.mx/reader033/viewer/2022052916/5a7924747f8b9a00168dac9e/html5/thumbnails/16.jpg)
Why Sentiment of Emojis?
• As features for sentiment models
• As labels to generate samples of negative and positive texts
• Sentiment conveyed by emoticons is domain and topic independent
Emoticons in opinion mining
• B-)
Emojis are replacing emoticons in social media
16
![Page 17: Sentiment of Emojis - VideoLectures.nettranslectures.videolectures.net/site/normal_dl/tag=1065925/single... · Why Sentiment of Emojis? •As features for sentiment models •As labels](https://reader033.vdocuments.mx/reader033/viewer/2022052916/5a7924747f8b9a00168dac9e/html5/thumbnails/17.jpg)
Dataset
• Over 1.6 million tweets in 13 European languages (Apr. 2013 – Feb. 2015)
• Labeled by 83 native speakers: negative (-1) < neutral (0) < positive (+1)
• 4% of our tweets contain emojis (69,546)
17 Mozetič I, Grčar M, Smailović J (2016) Multilingual Twitter Sentiment Classification: The Role of Human Annotators. PLoS ONE 11(5):
e0155036. doi:10.1371/journal.pone.0155036
![Page 19: Sentiment of Emojis - VideoLectures.nettranslectures.videolectures.net/site/normal_dl/tag=1065925/single... · Why Sentiment of Emojis? •As features for sentiment models •As labels](https://reader033.vdocuments.mx/reader033/viewer/2022052916/5a7924747f8b9a00168dac9e/html5/thumbnails/19.jpg)
Sentiment Score
19
Discrete distribution
Discrete probability distribution
Sentiment Score
negative: 3 neutral: 17 positive: 75
negativity p- = 0.04 neutrality p0 = 0.18 positivity p+= 0.78
Sentiment score = 0.74
BOUQUET
95 occurrences
![Page 20: Sentiment of Emojis - VideoLectures.nettranslectures.videolectures.net/site/normal_dl/tag=1065925/single... · Why Sentiment of Emojis? •As features for sentiment models •As labels](https://reader033.vdocuments.mx/reader033/viewer/2022052916/5a7924747f8b9a00168dac9e/html5/thumbnails/20.jpg)
Sentiment Score
Laplace estimate to estimate the probability
20
Discrete distribution
Discrete probability distribution
Sentiment Score
Sentiment score is the mean of the discrete probability distribution
![Page 21: Sentiment of Emojis - VideoLectures.nettranslectures.videolectures.net/site/normal_dl/tag=1065925/single... · Why Sentiment of Emojis? •As features for sentiment models •As labels](https://reader033.vdocuments.mx/reader033/viewer/2022052916/5a7924747f8b9a00168dac9e/html5/thumbnails/21.jpg)
Sentiment Bar
21
BOUQUET • Negative: 0.04
• Neutral: 0.18
• Positive: 0.78
• Sentiment Score: 0.74
![Page 22: Sentiment of Emojis - VideoLectures.nettranslectures.videolectures.net/site/normal_dl/tag=1065925/single... · Why Sentiment of Emojis? •As features for sentiment models •As labels](https://reader033.vdocuments.mx/reader033/viewer/2022052916/5a7924747f8b9a00168dac9e/html5/thumbnails/22.jpg)
Emoji Sentiment Lexicon http://kt.ijs.si/data/Emoji_sentiment_ranking/
22
![Page 23: Sentiment of Emojis - VideoLectures.nettranslectures.videolectures.net/site/normal_dl/tag=1065925/single... · Why Sentiment of Emojis? •As features for sentiment models •As labels](https://reader033.vdocuments.mx/reader033/viewer/2022052916/5a7924747f8b9a00168dac9e/html5/thumbnails/23.jpg)
Language Independent?
23
Compare the ordered list with Emoji Sentiment Lexicon
Order emojis by their sentiment score
Create emoji sentiment lexicon for each language separately
![Page 24: Sentiment of Emojis - VideoLectures.nettranslectures.videolectures.net/site/normal_dl/tag=1065925/single... · Why Sentiment of Emojis? •As features for sentiment models •As labels](https://reader033.vdocuments.mx/reader033/viewer/2022052916/5a7924747f8b9a00168dac9e/html5/thumbnails/24.jpg)
Emoji Sentiment in Different Languages
• Pearson’s correlation and Spearman’s rank correlation are relatively high, and significant for all the languages, except Albanian.
• Emoji Sentiment Ranking can be considered a universal, language-independent resource, at least for European languages.
24
![Page 25: Sentiment of Emojis - VideoLectures.nettranslectures.videolectures.net/site/normal_dl/tag=1065925/single... · Why Sentiment of Emojis? •As features for sentiment models •As labels](https://reader033.vdocuments.mx/reader033/viewer/2022052916/5a7924747f8b9a00168dac9e/html5/thumbnails/25.jpg)
Emoji Sentiment Map
25
![Page 26: Sentiment of Emojis - VideoLectures.nettranslectures.videolectures.net/site/normal_dl/tag=1065925/single... · Why Sentiment of Emojis? •As features for sentiment models •As labels](https://reader033.vdocuments.mx/reader033/viewer/2022052916/5a7924747f8b9a00168dac9e/html5/thumbnails/26.jpg)
Emoji Sentiment Map
26
![Page 27: Sentiment of Emojis - VideoLectures.nettranslectures.videolectures.net/site/normal_dl/tag=1065925/single... · Why Sentiment of Emojis? •As features for sentiment models •As labels](https://reader033.vdocuments.mx/reader033/viewer/2022052916/5a7924747f8b9a00168dac9e/html5/thumbnails/27.jpg)
Emoji Sentiment Map: Zoom Positive Sentiment Score
27
![Page 28: Sentiment of Emojis - VideoLectures.nettranslectures.videolectures.net/site/normal_dl/tag=1065925/single... · Why Sentiment of Emojis? •As features for sentiment models •As labels](https://reader033.vdocuments.mx/reader033/viewer/2022052916/5a7924747f8b9a00168dac9e/html5/thumbnails/28.jpg)
Emoji Sentiment Map: Zoom Negative Sentiment Score
28
![Page 29: Sentiment of Emojis - VideoLectures.nettranslectures.videolectures.net/site/normal_dl/tag=1065925/single... · Why Sentiment of Emojis? •As features for sentiment models •As labels](https://reader033.vdocuments.mx/reader033/viewer/2022052916/5a7924747f8b9a00168dac9e/html5/thumbnails/29.jpg)
Emoji Sentiment Map: Zoom Neutral Sentiment Score
29
![Page 30: Sentiment of Emojis - VideoLectures.nettranslectures.videolectures.net/site/normal_dl/tag=1065925/single... · Why Sentiment of Emojis? •As features for sentiment models •As labels](https://reader033.vdocuments.mx/reader033/viewer/2022052916/5a7924747f8b9a00168dac9e/html5/thumbnails/30.jpg)
Emoji Occurrence and Sentiment
30
![Page 31: Sentiment of Emojis - VideoLectures.nettranslectures.videolectures.net/site/normal_dl/tag=1065925/single... · Why Sentiment of Emojis? •As features for sentiment models •As labels](https://reader033.vdocuments.mx/reader033/viewer/2022052916/5a7924747f8b9a00168dac9e/html5/thumbnails/31.jpg)
Tweets With and Without Emojis
31
• Does the sentiment distribution differ between tweets with and without emojis?
Sentiment
• Do annotators agree on the sentiment of the tweet any better if tweet contains emojis?
Annotator agreement
![Page 32: Sentiment of Emojis - VideoLectures.nettranslectures.videolectures.net/site/normal_dl/tag=1065925/single... · Why Sentiment of Emojis? •As features for sentiment models •As labels](https://reader033.vdocuments.mx/reader033/viewer/2022052916/5a7924747f8b9a00168dac9e/html5/thumbnails/32.jpg)
Tweets With and Without Emojis: Sentiment
32
• Tweets with emojis are more positive.
![Page 33: Sentiment of Emojis - VideoLectures.nettranslectures.videolectures.net/site/normal_dl/tag=1065925/single... · Why Sentiment of Emojis? •As features for sentiment models •As labels](https://reader033.vdocuments.mx/reader033/viewer/2022052916/5a7924747f8b9a00168dac9e/html5/thumbnails/33.jpg)
Tweets With and Without Emojis
33
• Does the sentiment distribution differ between tweets with and without emojis?
Sentiment
• Do annotators agree on the sentiment of the tweet any better if it contains emojis?
Annotator agreement
![Page 34: Sentiment of Emojis - VideoLectures.nettranslectures.videolectures.net/site/normal_dl/tag=1065925/single... · Why Sentiment of Emojis? •As features for sentiment models •As labels](https://reader033.vdocuments.mx/reader033/viewer/2022052916/5a7924747f8b9a00168dac9e/html5/thumbnails/34.jpg)
Tweets With and Without Emojis: Annotator Agreement
34
• The presence of emojis has a positive impact on the emotional perception of the tweets by humans.
![Page 35: Sentiment of Emojis - VideoLectures.nettranslectures.videolectures.net/site/normal_dl/tag=1065925/single... · Why Sentiment of Emojis? •As features for sentiment models •As labels](https://reader033.vdocuments.mx/reader033/viewer/2022052916/5a7924747f8b9a00168dac9e/html5/thumbnails/35.jpg)
Where in Tweet?
35
Beginning of tweet End of tweet
More emotionally loaded emojis, either negative or positive, tend to occur towards the end of the tweets.
![Page 36: Sentiment of Emojis - VideoLectures.nettranslectures.videolectures.net/site/normal_dl/tag=1065925/single... · Why Sentiment of Emojis? •As features for sentiment models •As labels](https://reader033.vdocuments.mx/reader033/viewer/2022052916/5a7924747f8b9a00168dac9e/html5/thumbnails/36.jpg)
Applications
• Opinion mining and sentiment analysis
– Automated sentiment classification
– Automated dataset generation
• Fiverr.com is developing a system to automatically identify dissatisfaction, which is using our Emoji sentiment lexicon
36
![Page 37: Sentiment of Emojis - VideoLectures.nettranslectures.videolectures.net/site/normal_dl/tag=1065925/single... · Why Sentiment of Emojis? •As features for sentiment models •As labels](https://reader033.vdocuments.mx/reader033/viewer/2022052916/5a7924747f8b9a00168dac9e/html5/thumbnails/37.jpg)
Caution
37 Miller, H., Thebault-Spieker, J., Chang, S., Johnson, I., Terveen, L., and Hecht, B. (2016) “Blissfully happy” or “ready to fight”: Varying Interpretations of
Emoji. Proceedings of ICWSM 2016. Menlo Park, CA: AAAI Press.
![Page 38: Sentiment of Emojis - VideoLectures.nettranslectures.videolectures.net/site/normal_dl/tag=1065925/single... · Why Sentiment of Emojis? •As features for sentiment models •As labels](https://reader033.vdocuments.mx/reader033/viewer/2022052916/5a7924747f8b9a00168dac9e/html5/thumbnails/38.jpg)
Conclusion
• Emojis are ubiquitous
• Most of the most frequently used emojis are positive
• Tweets with emojis are more positive
• Annotators perceive sentiment better if emojis are present
• Emotionally loaded emojis tend to appear at the end of the tweet
• Emoji sentiment map
• Emoji sentiment lexicon
– Language independent
– Clarin.si
38
![Page 39: Sentiment of Emojis - VideoLectures.nettranslectures.videolectures.net/site/normal_dl/tag=1065925/single... · Why Sentiment of Emojis? •As features for sentiment models •As labels](https://reader033.vdocuments.mx/reader033/viewer/2022052916/5a7924747f8b9a00168dac9e/html5/thumbnails/39.jpg)
THANK YOU.
Thanks to coauthors
Jasmina Smailović, Borut Sluban and Igor Mozetič
39