natural language processing (csep 517): …...natural language processing (csep 517): computational...

Natural Language Processing (CSEP 517):Computational Pragmatics

Chenhao Tanc© 2017

University of [email protected]

May 22, 2017

1 / 67

What do we use language for?

Communicating with other humans

I exchanging emails

I talking to friends

I writing

I giving lectures

I ...

2 / 67

What do we use language for?

Communicating with other humans

I exchanging emails

I talking to friends

I writing

I giving lectures

I ...

3 / 67

Throw back Monday

Can you pass me the salt?

4 / 67

Pragmatics

The study of meaning as communicated by a speaker to a listener (Yule, 1996).

Or, contextual meaning

Pragmatics is important for building conversational agents, understanding humandecision making, understanding language, etc.

5 / 67

Pragmatics




6 / 67

Pragmatics




7 / 67

Pragmatics vs. Syntax, Semantics (Yule, 1996)

I Syntax: the relationships between linguistic forms, how they are arranged insequences, and which sequences are well-formed.

I Semantics: the relationships between linguistic forms and entities in the world.

I Pragmatics: the relationships between linguistic forms and the users of thoseforms.

8 / 67

Outline

Speech act theory

The effect of wording choices (big data pragmatics)

Modeling conversations: dialogue act categorization

Rational speech acts model

9 / 67

Outline

Speech act theory




10 / 67

Speech act theory

We do not simply produce utterances containing grammatical structures; we performactions via those utterances.

Actions performed via utterances are generally called speech acts (Austin, 1975).

11 / 67

Speech act theory

We do not simply produce utterances containing grammatical structures; we performactions via those utterances.Actions performed via utterances are generally called speech acts (Austin, 1975).

12 / 67

Speech act theory

I locutionary act (the actual utterance and its ostensible meaning)

I illocutionary act (its real, intended meaning)

I perlocutionary act (its actual effect, whether intended or not)

13 / 67

Outline

Speech act theory




14 / 67

Wording mattersMotivate voter turnout (Bryan et al., 2011)

“How important is it to you to be a voterin the upcoming election?”

“How important is it to you to vote in theupcoming election?”

15 / 67

Wording mattersMotivate voter turnout (Bryan et al., 2011)

“How important is it to you to be a voterin the upcoming election?”

“How important is it to you to vote in theupcoming election?”

16 / 67

Large-scale natural experiments

A large number of social interactions in the format of texts

⇓

Potential opportunities for natural experiments

17 / 67


A large number of social interactions in the format of texts

⇓

Potential opportunities for natural experiments

18 / 67


The effect of wording on message propagation on Twitter (Tan et al., 2014)

19 / 67


The effect of wording on message propagation on Twitter (Tan et al., 2014)

20 / 67


I Millions of topic-author controlled pairsI Ranking within a pair (classification)

I Evaluation: the accuracy of predicting which one was retweeted more(random → 50%)

I Classifier: logistic regression

21 / 67

Features

Pronouns

first person singular (i)

——–

first person plural (we)

——–

second person (you)

——–

third person singular (she, he)

↑↑ ↑↑

third person plural (they)

↑ ↑↑↑

Referring to other people helps

22 / 67

Features

Pronouns

first person singular (i) ——–first person plural (we) ——–second person (you) ——–third person singular (she, he) ↑↑ ↑↑third person plural (they) ↑ ↑↑↑


23 / 67

Features

Pronouns

first person singular (i) ——–first person plural (we) ——–second person (you) ——–third person singular (she, he) ↑↑ ↑↑third person plural (they) ↑ ↑↑↑


24 / 67

Features

Generality

indefinite articles (a,an)

↑↑↑ ↑

definite articles (the)

——–

Generality helps

25 / 67

Features

Generality

indefinite articles (a,an) ↑↑↑ ↑definite articles (the) ——–

Generality helps

26 / 67

Features

Generality

indefinite articles (a,an) ↑↑↑ ↑definite articles (the) ——–

Generality helps

27 / 67

Features

Language model scores

I similarity with overall Twitter users

twitter unigram

↑↑↑ ↑

twitter bigram

↑↑↑ ↑

28 / 67

Features



twitter unigram ↑↑↑ ↑twitter bigram ↑↑↑ ↑

29 / 67

Features




I similarity with personal history

personal unigram

↑↑↑ ↑

personal bigram

——–

Be like the community & be true to yourself

30 / 67

Features




I similarity with personal history

personal unigram ↑↑↑ ↑personal bigram ——–

Be like the community & be true to yourself

31 / 67

Baseline without “natural experiments”

Supervised classification without control

I most-retweeted tweets vs. least-retweeted tweets

32 / 67

Prediction performance

Human performance

I Controlling for context is importantI Big data can help understand pragmatics

https://chenhaot.com/retweetedmore

33 / 67


Prediction performance

Human performance

I Controlling for context is importantI Big data can help understand pragmatics


34 / 67


Beyond retweeting

I Persuasive arguments (Tan et al., 2016)

I Memorable (movie) quotes (Danescu-Niculescu-Mizil et al., 2012a)

I Power dynamics (Danescu-Niculescu-Mizil et al., 2012b; Prabhakaran et al., 2014)

I Newsworthiness of research articles and political speeches (Zhang et al., 2016)

35 / 67

Outline

Speech act theory




36 / 67

Dialogue act classification/tagging

Define categories and label corpora (Stolcke et al., 2000)

I statement

I question

I backchannel

I agreement

I apology

I ...

37 / 67

Dialogue act classification/tagging

Supervised classification

I SVM

I logistic classification

Structure prediction (sequence tagging)

I Hidden Markov model

I Conditional random field

38 / 67

Speech act theory




39 / 67

Cooperative Principle

Make your contribution as is required, when it is required, by the conversation in whichyou are engaged (Grice, 1975).

40 / 67

Conversational Implicatures

I Maxims of quality(Do not say what you believe to be false; do not say that for which you lackadequate evidence)e.g., Noah is a nice person

⇒ I believe that Noah is a nice person

I Maxims of quantityMake you contribution as informative as is required (for the current purposes ofthe exchange); do not make your contribution more informative than is required

I I have two hands ⇒ I have no more than two hands

41 / 67


I Maxims of quality(Do not say what you believe to be false; do not say that for which you lackadequate evidence)e.g., Noah is a nice person ⇒ I believe that Noah is a nice person



42 / 67




I I have two hands

⇒ I have no more than two hands

43 / 67





44 / 67

Rational Speech Acts Model

Reference games (Wittgenstein, 1953; Frank and Goodman, 2012)

I Speaker. Imagine you are talking to someone and want to refer to the middleobject. Would you say “blue” or “circle”?

I Listener. Someone uses the word “blue” to refer to one of these objects. Whichobject are they talking about?

45 / 67


Reference games (Wittgenstein, 1953; Frank and Goodman, 2012)

I Speaker. Imagine you are talking to someone and want to refer to the middleobject. Would you say “blue” or “circle”?

I Listener. Someone uses the word “blue” to refer to one of these objects. Whichobject are they talking about?

46 / 67


Literal listener (l0)Pl0(s | u) ∝ P (s)JuK(s)

I P (s): the prior over states

I JuK(s): a mapping from states of the world to truth values

∀s, P (s) = 1/3

blue 0.5 0.5 0green 0 0 1square 0.5 0 0.5circle 0 1 0

47 / 67



I P (s): the prior over states

I JuK(s): a mapping from states of the world to truth values

∀s, P (s) = 1/3


48 / 67








50 / 67








51 / 67

Rational Speech Acts ModelLiteral listener (l0)

Pl0(s | u) ∝ P (s)JuK(s)Pragmatic speaker (s1)


α = 1

cost(u) =

{0, u ∈ {blue, green, circle, square}∞, otherwise

blue green square circle

0.5 0 0.5 0

0.33 0 0 0.67

0 0.67 0.33 0

52 / 67



Pragmatic speaker (s1)


pragmatic speaker

blue green square circle

0.5 0 0.5 0

0.33 0 0 0.67

0 0.67 0.33 0

vs.

literal listener


53 / 67




Pragmatic listener (l1)Pl1(s | u) ∝ P (s)Ps1(u | s)


54 / 67






55 / 67





pragmatic listener


vs.

literal listener


Live demo: http://gscontras.github.io/ESSLLI-2016/

56 / 67

http://gscontras.github.io/ESSLLI-2016/





pragmatic listener


vs.

literal listener


Live demo: http://gscontras.github.io/ESSLLI-2016/57 / 67

http://gscontras.github.io/ESSLLI-2016/


Literal listener (l0): utterance meaning × state priorPragmatic speaker (s1): literal listener - utterance costsPragmatic listener (l1): pragmatic speaker × state prior

58 / 67


Literal speaker (s0): utterance meaning - utterance costsPragmatic listener (l1): literal speaker × state priorPragmatic speaker (s1): pragmatic listener - utterance costs

59 / 67

Experiments

Rational speech acts model is a powerful tool for understanding the pragmatic meaningof language.

60 / 67

Experiments

Rational speech acts model is a powerful tool for understanding the pragmatic meaningof language.

61 / 67

Extensions and critiques

I Learning based approach by featurizing utterances and states (Monroe and Potts,2015)

I Neural rational speech acts model (Monroe et al., 2017)

I Exceptions: sarcasm, irony, hedging, etc

I Cultural differences

62 / 67

Extensions and critiques

I Learning based approach by featurizing utterances and states (Monroe and Potts,2015)

I Neural rational speech acts model (Monroe et al., 2017)

I Exceptions: sarcasm, irony, hedging, etc

I Cultural differences

63 / 67

Summary

I Wording matters; we can learn useful insights from social interaction dataavailable nowadays

I Modeling conversations by categorizing speech acts

I Rational speech acts model can achieve pragmatics understanding

64 / 67

Computational Pragmatics

Questions?https://chenhaot.com

[email protected]

65 / 67

https://chenhaot.com

References I

John Langshaw Austin. How to do things with words. Oxford university press, 1975.

Christopher J Bryan, Gregory M Walton, Todd Rogers, and Carol S Dweck. Motivating voter turnout byinvoking the self. Proceedings of the National Academy of Sciences, 108(31):12653–12656, 2011. URLhttp://www.pnas.org/content/108/31/12653.abstract.

Cristian Danescu-Niculescu-Mizil, Justin Cheng, Jon Kleinberg, and Lillian Lee. You Had Me at Hello: HowPhrasing Affects Memorability. In Proceedings of ACL, 2012a. URLhttp://dl.acm.org/citation.cfm?id=2390524.2390647.

Cristian Danescu-Niculescu-Mizil, Lillian Lee, Bo Pang, and Jon Kleinberg. Echoes of Power: Language Effectsand Power Differences in Social Interaction. In Proceedings of the 21st International Conference on WorldWide Web, 2012b. URL http://doi.acm.org/10.1145/2187836.2187931.

Michael C Frank and Noah D Goodman. Predicting Pragmatic Reasoning in Language Games. Science, 336(6084):998–998, 2012.

H Paul Grice. Logic and conversation. 1975, pages 41–58, 1975.

Will Monroe and Christopher Potts. Learning in the rational speech acts model. In Proceedings of the 20thAmsterdam Colloquium, 2015.

Will Monroe, Robert X.D. Hawkins, Noah D. Goodman, and Christopher Potts. Colors in Context: A PragmaticNeural Model for Grounded Language Understanding. In Proceedings of ACL, 2017.

Vinodkumar Prabhakaran, Ashima Arora, and Owen Rambow. Staying on Topic: An Indicator of Power inPolitical Debates. In EMNLP, 2014.

66 / 67

http://www.pnas.org/content/108/31/12653.abstract

http://dl.acm.org/citation.cfm?id=2390524.2390647

http://doi.acm.org/10.1145/2187836.2187931

References II

Andreas Stolcke, Klaus Ries, Noah Coccaro, Elizabeth Shriberg, Rebecca Bates, Daniel Jurafsky, Paul Taylor,Rachel Martin, Carol Van Ess-Dykema, and Marie Meteer. Dialogue act modeling for automatic tagging andrecognition of conversational speech. Computational linguistics, 26(3):339–373, 2000.

Chenhao Tan, Lillian Lee, and Bo Pang. The effect of wording on message propagation: Topic- andauthor-controlled natural experiments on Twitter. In Proceedings of ACL, 2014.

Chenhao Tan, Vlad Niculae, Cristian Danescu-Niculescu-Mizil, and Lillian Lee. Winning Arguments: InteractionDynamics and Persuasion Strategies in Good-faith Online Discussions. In Proceedings of WWW, 2016. URLhttp://dx.doi.org/10.1145/2872427.2883081.

Ludwig Wittgenstein. Philosophical investigations. 1953.

George Yule. Pragmatics. Oxford, 1996.

Ye Zhang, Erin Willis, Michael J Paul, Nomie Elhadad, and Byron C Wallace. Characterizing the (Perceived)Newsworthiness of Health Science Articles: A Data-Driven Approach. JMIR medical informatics, 4(3), 2016.

67 / 67

http://dx.doi.org/10.1145/2872427.2883081

natural language processing (csep 517): …...natural language processing (csep 517): computational...

Documents