sarcasm game

of 33/33
+ Did you get it? Sanath Bhandary Akhil Bhiwal #Self Organization in Humans #Crowd Sourcing #Game With a Purpose #Human Computation #Computational Linguistics

Post on 13-Jul-2015

175 views

Category:

Technology

0 download

Embed Size (px)

TRANSCRIPT

  • +Did you get it? Sanath Bhandary Akhil Bhiwal

    #Self Organization

    in Humans

    #Crowd Sourcing

    #Game With a

    Purpose

    #Human

    Computation

    #Computational Linguistics

  • +Self Organization in Humans

    n As in natural phenomenon like ants, people too organize themselves into patterns to get tasks done.

  • +Self Organization in Humans

    n As in natural phenomenon like ants, people too organize themselves into patterns to get tasks done.

  • +Motivation

  • +Self Organization in Humans

    n Millions of people play games and spent countless hours for entertainment.

    n To give you an example:

    Time spent in 1 year > 9 Billion Time took to built this = 7 million hours

  • +Self Organization in Humans

    n Millions of people play games and spent countless hours for entertainment.

    n To give you an example:

    Time spent in 1 year > 9 Billion Time took to built this = 20 million hours

  • +n Could we use this human time to do something useful, while people still get entertained.

    n We think, yes!

    n So, we designed a game which people play purely for entertainment.

    n As a side effect of their playing this game, they solve a problem which computers currently cant do easily.

    Self Organization in Humans

  • +Problems hard for computer Easy for Humans

    n Machine translation.

    n Identifying objects in given image (Computer vision).

    n Detecting sarcasm in given text.

    n And many more

  • +Detecting Sarcasm in Text

    n Sarcasm transforms the polarity of an apparently positive or negative utterance into its opposite.

    n Why its difficult: Some of the best approaches for computational linguistics relies on machine learning techniques which require large dataset. However, the currently available datasets are small so limits training of algorithms.

    n Our aim in this project is to construct a corpus of text for computational linguistics researcher to train their existing algorithms or create more accurate computer linguistic algorithms.

  • +Did you get it? Game Rules

    n Web-based multiplayer game

    n 2 users to play a single instance of game.

    n Players have limited communication with each other and cant know their partners identity.

    n Each user gets a small paragraph (at max 4 sentences).

    n User categorizes it as positive, negative or sarcastic. User can also give two words which she finds useful to identify that. Each matching word with other partner fetches bonus score.

  • +Username:

    Login

  • +Did you get it?

    Play

    Pack 1 1-5 Points

    Pack 3 10-15 Points

    Pack 2 5-10 Points

    Pack 4 15-25 Points

    Your Score 480

    Highest Score 5150

  • +Did you get it?

    Words

    Sarcastic

    Positive Negative

    Pass Submit

    Sometimes I need what only you can provide: your absence.

    Time Left 1:30

    Score 155

    This text is:

    Key

    Points: 4

  • +Did you get it?

    Words

    Sarcastic

    Positive Negative

    Pass Submit

    Marriage is the chief cause of divorce!

    Time Left 0:22

    Score 985

    This text is:

    Key

    Points: 6

  • +Did you get it?

    Words

    Sarcastic

    Positive Negative

    Pass Submit

    The 100% American is 99% idiot.

    Time Left 2:55

    Score 65

    100% 99%

    This text is:

    Key

    Points: 5

  • +Did you get it? Game Rules: Reiteration

    n Web-based multiplayer game

    n 2 users to play a single instance of game.

    n Players have limited communication with each other and cant know their partners identity.

    n Each user gets a small paragraph (at max 4 sentences).

    n User categorizes it as positive, negative or sarcastic. User can also give two words which she finds useful to identify that. Each matching word with other partner fetches bonus score.

  • +The failure points!

    n Language apart from English.

  • +Did you get it?

    Words

    Sarcastic

    Positive Negative

    Pass Submit

    Por favor, treme un vaso de agua.

    Time Left 9:58

    Score 0

    This text is:

    Key

    Points: 23

  • +The failure points!

    n Language apart from English.

    n Multiple recurrence of same word.

  • +Did you get it?

    Words

    Sarcastic

    Positive Negative

    Pass Submit

    Where she sits she shines, and where she shines she sits.

    Time Left 5:15

    Score 480

    This text is:

    Key

    Points: 25

  • +How it works?

    n User starts with a initial score of zero.

    n User chooses the pack he wants to play with. Pack 1 being the easiest and Pack 4 being the most difficult.

    n Each time user agrees with his/her partner on a particular text, their score increases by points mentioned below the text.

    n If one user clicks on Pass, other user also have to do the same. He cant choose any other option until he passes that question.

  • +How it works?

    n Bonus Score: n Lets say the score of given text is X points. n If players have submitted one key word same, both gets a bonus

    score of X.

    n If players submitted both key words same, they get a bonus score of 3X.

    n Key words are useful metadata to train computational linguistics algorithm.

  • +Did you get it?

    Words

    Sarcastic

    Positive Negative

    Pass Submit

    The 100% American is 99% idiot.

    Time Left 2:55

    Score 65

    100% 99%

    This text is:

    Key

    Points: 5

  • +How it works?

    n Dataset: n Twitter dataset with userid (@userid) and hashtag (#hashtag)

    removed.

    n In practice, any sort of data can be used. Eg. Product reviews, opinions, etc.

    n Implementation: n JavaScript and Nodejs (>90%) + Python (

  • +How it works?

    n When N players agrees that a specific text to be of positive/negative/sarcastic type, we tag that text with specified type.

    n When users submit result for specific text, the points associated with text gets recalculated by: n Points = Points + Pheromone * weighted sum

    n When users agree, pheromone = 1 else pheromone = -1.

    n Weighted sum adjusts the points of the text.

    In our game:

    N = 10

    Weighted sum = 0.02

  • +Self Organization of Players

    n Task: Constructing corpus for computational linguistics researchers

    n Agents: Players

    n Program: Incentive structure Points and Entertainment

    n Patterns formation: Clusters around different difficulty packs

    n Communication: Pheromone based model

  • +Self Organization of Players

  • +n Inspired from Pheromone based Model

    n Pheromones are values that are used to alter the points associated with each sentence on the basis of the following formula

    n Pt = Pt-1 + (pheromone intensity)*W Pt : Points associated with sentence at time t Pt-1 : Points associated with sentence at time t-1 Pheromone intensity =1 when players disagree Pheromone intensity =-1 when players agree W : Weighted constant 0

  • +Agents

    n Users are synonymously referred as agents

    n Agents have two operations : sense_sentence_score(sentence_id) modify_sentence_score(sentence_id)

  • +Clustering

    n Driven by incentive model

  • +Self Organization: Different Perspective

    Robot Based Model Human Based Model

    Task Moving a rock Creating metadata for ML algorithms

    Agent Robots Humans

    Program Set of rules Incentive structure (more points or entertainment or both)

    Communication mechanism

    Electromagnetic radiation Pheromone deposit

    Patterns Local interactions causing global behavior

    People forming clusters

    Fault-Tolerant Yes Yes. System is independent of agents

    Dependence on Initial Conditions

    It depends Yes

  • +Conclusion

    n Play game.

    n Better still, collaborate with us to further develop this game. Email us for accessing code via GitHub.

    n When robots become dominant, they will still need humans. So, our species is safe.

  • +Thank you!