+

Did you get it? Sanath Bhandary Akhil Bhiwal

#Self Organization

in Humans

#Crowd Sourcing

#Game With a

Purpose

#Human

Computation

#Computational Linguistics

+Self Organization in Humans

n  As in natural phenomenon like ants, people too organize themselves into patterns to get tasks done.

+Self Organization in Humans

n  As in natural phenomenon like ants, people too organize themselves into patterns to get tasks done.

+

Motivation

+Self Organization in Humans

n  Millions of people play games and spent countless hours for entertainment.

n  To give you an example:

Time spent in 1 year > 9 Billion Time took to built this = 7 million hours

+Self Organization in Humans

n  Millions of people play games and spent countless hours for entertainment.

n  To give you an example:

Time spent in 1 year > 9 Billion Time took to built this = 20 million hours

+

n  Could we use this human time to do something useful, while people still get entertained.

n  We think, yes!

n  So, we designed a game which people play purely for entertainment.

n  As a side effect of their playing this game, they solve a problem which computers currently can’t do easily.

Self Organization in Humans

+Problems hard for computer – Easy for Humans

n  Machine translation.

n  Identifying objects in given image (Computer vision).

n  Detecting sarcasm in given text.

n  And many more…

+Detecting Sarcasm in Text

n  Sarcasm transforms the polarity of an apparently positive or negative utterance into its opposite.

n  Why it’s difficult: Some of the best approaches for computational linguistics relies on machine learning techniques which require large dataset. However, the currently available datasets are small so limits training of algorithms.

n  Our aim in this project is to construct a corpus of text for computational linguistics researcher to train their existing algorithms or create more accurate computer linguistic algorithms.

+Did you get it? Game Rules

n  Web-based multiplayer game

n  2 users to play a single instance of game.

n  Players have limited communication with each other and can’t know their partner’s identity.

n  Each user gets a small paragraph (at max 4 sentences).

n  User categorizes it as positive, negative or sarcastic. User can also give two words which she finds useful to identify that. Each matching word with other partner fetches bonus score.

+

Username:

Login

+

Did you get it?

Play

Pack 1 1-5 Points

Pack 3 10-15 Points

Pack 2 5-10 Points

Pack 4 15-25 Points

Your Score 480

Highest Score 5150

+

Did you get it?

Words

Sarcastic

Positive Negative

Pass Submit

Sometimes I need what only you can provide: your absence.

Time Left 1:30

Score 155

This text is:

Key

Points: 4

+

Did you get it?

Words

Sarcastic

Positive Negative

Pass Submit

Marriage is the chief cause of divorce!

Time Left 0:22

Score 985

This text is:

Key

Points: 6

+

Did you get it?

Words

Sarcastic

Positive Negative

Pass Submit

The 100% American is 99% idiot.

Time Left 2:55

Score 65

100% 99%

This text is:

Key

Points: 5

+Did you get it? Game Rules: Reiteration

n  Web-based multiplayer game

n  2 users to play a single instance of game.

n  Players have limited communication with each other and can’t know their partner’s identity.

n  Each user gets a small paragraph (at max 4 sentences).

n  User categorizes it as positive, negative or sarcastic. User can also give two words which she finds useful to identify that. Each matching word with other partner fetches bonus score.

+The failure points!

n  Language apart from English.

+

Did you get it?

Words

Sarcastic

Positive Negative

Pass Submit

Por favor, tráeme un vaso de agua.

Time Left 9:58

Score 0

This text is:

Key

Points: 23

+The failure points!

n  Language apart from English.

n  Multiple recurrence of same word.

+

Did you get it?

Words

Sarcastic

Positive Negative

Pass Submit

Where she sits she shines, and where she shines she sits.

Time Left 5:15

Score 480

This text is:

Key

Points: 25

+How it works?

n  User starts with a initial score of zero.

n  User chooses the pack he wants to play with. Pack 1 being the easiest and Pack 4 being the most difficult.

n  Each time user agrees with his/her partner on a particular text, their score increases by points mentioned below the text.

n  If one user clicks on Pass, other user also have to do the same. He can’t choose any other option until he passes that question.

+How it works?

n  Bonus Score: n  Let’s say the score of given text is X points.

n  If players have submitted one key word same, both gets a bonus score of X.

n  If players submitted both key words same, they get a bonus score of 3X.

n  Key words are useful metadata to train computational linguistics algorithm.

+

Did you get it?

Words

Sarcastic

Positive Negative

Pass Submit

The 100% American is 99% idiot.

Time Left 2:55

Score 65

100% 99%

This text is:

Key

Points: 5

+How it works?

n  Dataset: n  Twitter dataset with userid (@userid) and hashtag (#hashtag)

removed.

n  In practice, any sort of data can be used. Eg. Product reviews, opinions, etc.

n  Implementation: n  JavaScript and Nodejs (>90%) + Python (<10%)

n  Mongodb for storing data

n  Platform: Heroku

+How it works?

n  When N players agrees that a specific text to be of positive/negative/sarcastic type, we tag that text with specified type.

n  When users submit result for specific text, the points associated with text gets recalculated by: n  Points = Points + Pheromone * weighted sum

n  When users agree, pheromone = 1 else pheromone = -1.

n  Weighted sum adjusts the points of the text.

In our game:

N = 10

Weighted sum = 0.02

+Self Organization of Players

n  Task: Constructing corpus for computational linguistics researchers

n  Agents: Players

n  Program: Incentive structure – Points and Entertainment

n  Patterns formation: Clusters around different difficulty packs

n  Communication: Pheromone based model

+Self Organization of Players

+

n  Inspired from Pheromone based Model

n  Pheromones are values that are used to alter the points associated with each sentence on the basis of the following formula

n  Pt = Pt-1 + (pheromone intensity)*W Pt : Points associated with sentence at time t Pt-1 : Points associated with sentence at time t-1 Pheromone intensity =1 when players disagree Pheromone intensity =-1 when players agree W : Weighted constant 0<W<1

Communication Scheme

+Agents

n  Users are synonymously referred as agents

n  Agents have two operations : sense_sentence_score(sentence_id) modify_sentence_score(sentence_id)

+Clustering

n  Driven by incentive model

+Self Organization: Different Perspective

Robot Based Model Human Based Model

Task Moving a rock Creating metadata for ML algorithms

Agent Robots Humans

Program Set of rules Incentive structure (more points or entertainment or both)

Communication mechanism

Electromagnetic radiation Pheromone deposit

Patterns Local interactions causing global behavior

People forming clusters

Fault-Tolerant Yes Yes. System is independent of agents

Dependence on Initial Conditions

It depends Yes

+Conclusion

n  Play game.

n  Better still, collaborate with us to further develop this game. Email us for accessing code via GitHub.

n  When robots become dominant, they will still need humans. So, our species is safe.

+Thank you!