computational models of emotion inference in theory of ...· aa*star artificial intelligence...

Download Computational models of emotion inference in Theory of ...· aA*STAR Artificial Intelligence Initiative,

Post on 08-Apr-2019

215 views

Category:

Documents

1 download

Embed Size (px)

TRANSCRIPT

Running Head: EMOTION IN THEORY OF MIND

Computational models of emotion inference in Theory of Mind:

A review and roadmap

Desmond C. Onga,b, Jamil Zakic, and Noah D. Goodmanc,d aA*STAR Artificial Intelligence Initiative, Agency for Science, Technology and Research

(A*STAR), Singapore

bInstitute of High Performance Computing, Agency for Science, Technology and Research

(A*STAR), Singapore cDepartment of Psychology, Stanford University

dDepartment of Computer Science, Stanford University

In Press at: Topics in Cognitive Science

Special Issue on: Computational Approaches to Social Cognition

Final version dated: 1 July 2017 (Minor edits to funding acknowledgements)

Address Correspondence to:

Desmond C. Ong A*STAR Artificial Intelligence Initiative Agency for Science, Technology and Research, Singapore 138632 desmond.c.ong@gmail.com Author Contributions: D.C.O., J. Z., and N. D. G. wrote the paper. The authors declare no conflict of interest. Keywords: Emotion; Affective Cognition; Inference; Theory of Mind

EMOTIONINTHEORYOFMIND 2

ABSTRACT

Research on social cognition has fruitfully applied computational modeling approaches to

explain how observers understand and reason about others mental states. By contrast, there has

been less work on modeling observers understanding of emotional states. We propose an

intuitive theory framework to studying affective cognition---how humans reason about emotions-

--and derive a taxonomy of inferences within affective cognition. Using this taxonomy, we

review formal computational modeling work on such inferences, including: causal reasoning

about how others react to events, reasoning about unseen causes of emotions, reasoning with

multiple cues, as well as reasoning from emotions to other mental states. Additionally, we

provide a roadmap for future research by charting out inferences---such as hypothetical and

counterfactual reasoning about emotions---that are ripe for future computational modeling work.

This framework proposes unifying these various types of reasoning as Bayesian inference within

a common intuitive Theory of Emotion. Finally, we end with a discussion of important

theoretical and methodological challenges that lie ahead in modeling affective cognition.

EMOTIONINTHEORYOFMIND 3

Recentdevelopmentsincomputationalcognitivemodelinghaveallowed

researcherstospecifyandtestprecisehypothesesabouthowpeoplemakeinferences

abouttheirsocialworld.Theseincludeinferencesaboutwhatothersdesireandbelieve

abouttheworld(e.g.,Baker,Jara-Ettinger,Saxe,&Tenenbaum,2017;Goodmanetal,

2006),whatothersmeanwhentheyuselanguagetocommunicate(Goodman&Frank,

2016;Goodman&Stuhlmller,2013;Frank&Goodman,2012),andwhatfuturedecisions

othersmightmake(Jara-Ettinger,Gweon,Schulz,&Tenenbaum,2016;Jern&Kemp,2015).

Theserecentdevelopmentscomprisethefocusofthecurrentspecialissue.However,these

modelshaveoftenneglectedhowpeoplereasonaboutoneofthemostessentialelements

ofhumanpsychology:emotions.

Emotionsplayacentralroleinoursociallives.Theysignalpeoplesimmediate

reactionstoeventsintheworld(e.g.,Ellsworth&Scherer,2003),andcausemany

behaviors---bothintentionalandunintentional(Lerner,Li,Valdesolo,&Kassam,2015;

Loewenstein&Lerner,2003).Giventheimportanceofemotionsinsocialinteractions,itis

hardlysurprisingthatpeoplearenaturallyattunedtoperceivingandunderstanding

emotionsinthosearoundthem(Harris,1989;Zaki&Ochsner,2011).

Inthispaper,weconsiderthepresentandfutureofcomputationalmodelingin

understandinghowpeoplereasonaboutemotionalstates---whatwetermaffective

cognition(Ong,Zaki,&Goodman,2015).Weadoptanintuitivetheoryframeworktothe

studyofaffectivecognition,usingwhichwederiveataxonomyofaffectivecognitive

inferences(inthespiritofAdelson&Bergen,1991,andKemp&Jern,2013,who

respectivelyderivedtaxonomiesofvisualfunctionsandinductiveproblems).This

taxonomyencompassesawiderangeoflayreasoningaboutemotionalstates,suchasusing

EMOTIONINTHEORYOFMIND 4

observedbehaviororsituationcontexts(orcombinationsthereof)toinferemotionsand

othermentalstates(e.g.,DeMelo,Carnevale,Read,&Gratch,2014;Wu,Baker,Tenenbaum,

&Schulz,2018;Zaki,2013).Finally,weprovidearoadmapforfutureresearch,by

highlightinginferencesthathaveyettobestudiedcomputationally,andbydiscussing

importanttheoreticalandmethodologicalchallengesthatlieahead.

Layingoutanintuitivetheoryofemotions

Peoplehaverichintuitivetheoriesofhowothersaroundthemthinkandbehave,

allowingthemtoinferothersmotivationsandexplainothersbehavior(e.g.,Heider,1958;

Gopnik&Meltzoff,1997;Ross,1977).Anintuitivetheoryofotherpeopleconsistsoffirst,a

structuredontologyofconcepts---forexample,personality,goals,behavior---andsecond,

thecausalrelationshipsrelatingtheseconcepts(Gerstenberg&Tenenbaum,2017).These

intuitivetheoriesallowlaypeopletomakesenseofothersaroundthem,inasimilarfashion

tohowscientifictheoriesallowscientiststoexplainthephysicalworld(Carey,2009;

Wellman&Gelman,1992).

Peoplealsopossessarichintuitivetheoryofemotionsthatcomprisesconceptual

knowledgeaboutdifferentemotionalstates(e.g.,anger,happiness)andhowtheyare

relatedtotheircausesandeffects.People(observers)usetheirintuitivetheoryof

emotiontoreasonabouttheemotionalstatesofothers(agents)aroundthem,and

therebydecidehowbesttorespondinsocialsituations.Importantly,theseintuitive

theoriescomprisetheobserversbeliefsabouthowothersemotionswork,whichdepend

ontheobserverspasthistoryandtheirsubjectivebeliefs.Thoughtheobserversbeliefs

maynotnecessarilyreflecttherealityofhowemotionsactuallywork,thesebeliefs

EMOTIONINTHEORYOFMIND 5

neverthelessformthebasisforhowtheobserverunderstandsandinteractswiththose

aroundthem(e.g.,Gopnik&Meltzoff,1997;Ross,1977).

Anintuitivetheoryofemotioncontainstwoimportanttypesofcausalrelationships.

Thefirstconnectsemotionstotheircauses:what,intheobserversmind,causesanagent

tofeelanemotion?Thesecondinvolvestheeffectsofemotion:what,intheobservers

mind,doesanagentsemotioncausethemtodo?Thesecomponentsandtheirrelationships

arerepresentedinFigure1.

Figure1:Amodelofanintuitivetheoryofemotion,unifyingideasfromdeMeloetal(2014),Ong,Zaki,&Goodman(2015),SaxeandHoulihan(2017),andWuetal(2018).Weusestandardgraphicalmodelnotation:shadedcirclesrepresentobservablevariables,whileunshadedcirclesrepresentlatentvariables.Werendervariablesatthesubsequenttime-steptranslucent.Arrowsrepresentadirectedcausalrelationship,andboldedlettersdenotetheabbreviationsusedinequations.Theobserverappliesathird-personappraisalprocesstoreasonabouthow(i)theoutcomeofaneventthatanagentexperiences,and(ii)theagentsmentalstates(beliefsanddesires),togetherresultintheagentexperiencingemotions.Theagentsemotionsinturncausetheagenttodisplayemotionalexpressions,andtakeintentionalactionsthatleadtonewoutcomesandupdatedmentalstates(andnewemotions).

Intuitivecausesofemotions

Peopleintuitivelyexpectthatothersemotionsariseasareactiontomotivationally

salientevents.Inadditiontotheoutcomeoftheseevents,peoplealsousetheirknowledge

aboutothersmentalstatessuchasothersbeliefsanddesirestoreasonabouthow

othersfeel.Thisintuitiveunderstandingofhoweventoutcomesandmentalstatesimpact

emotionsemergesearlyinlife(Gross&Ballif,1991;Harris,1989;Lagattuta,Wellman,&

EMOTIONINTHEORYOFMIND 6

Flavell,1997;Repacholi,Meltzoff,Hennings,&Ruba,2016;Wu,Muentener,&Schulz,

2017):toddlersandinfantsunderstandthatanagentsdisplayofhappinessreflectsa

satisfieddesire,whileanagentsdisplayofsadnessreflectsathwartedgoal(Repacholi&

Gopnik,1997;Skerry&Spelke,2014;Wellman,Phillips,&Rodriguez,2000).Bypreschool,

childrenassociateasurprisedemotionalexpressionwithamismatchbetweenrealityand

theagentspriorbeliefsabouttheworld(Wellman&Banerjee,1991;Wu&Schulz,2017),

andfactoragentsexpectationsintoattributionsoftheagentsemotions(Ong,Asaba,&

Gweon,2016).Adultssimilarlydrawrichinferencesfromanagentsemotionalreactionsto

theirlatentbeliefsanddesires(Scherer&Grandjean,2008;Wuetal,2018;VanKleef,De

Dreu,&Manstead,2010).Thus,theintuitivetheoryofemotionrelatesanagentsmental

states(specifically,theirbeliefsabouttheworld,andtheirdesires),andtheoutcomeof

aneventthattheagentexperiences,totheagentsemotions(Fig.1).

Howdopeopleunderstandthewayeventoutcomesandtheagentsmentalstates

giverisetoemotions?Oneideaisthatpeoplemayengageasimilarreasoningprocessto

howtheythemselvesexperienceeventsfirsthand.Accordingtoappraisaltheoriesof

emotion(Arnold,1960;Ellsworth&Scherer,2003;Moors,Ellsworth,Scherer&Frijda,

2013;Ortony,Clore,&Collins,1988;Smith&Lazarus,1993),anagentsemotionsarise

fromanevaluation(appraisal)ofoutcomesalongvariousself-relevantdimensions,such

aswhethertheoutcomescontributetoordetractfromtheagentsgoals(goal-

conduciveness).IfSallyjudgesherjobofferasfacilitatinghercareergoals,shewould

likelyfeelhappy.

Manyresearchershaveproposedthatlaypeopleperformasimilarappraisal-like

processtoreasonaboutothersemotions(DeMeloetal,2014;Ong,Zaki,&Goodman,

EMOTIONINTHEORYOFMIND 7

2015;Seimer&Reisenzein,2007;Skerry&Saxe,2015;Wondra&Ellsworth,2015;Wuet

al,2018;VanKleefetal,2010)---wetermthi

Recommended

View more >