lecture 4 emotion theory (continued)

CSCI 534(Affective Computing) – Lecture by Jonathan Gratch

Lecture 4Emotion Theory (continued)


Affective Computing in the news

http://www.affectiva.com/

http://www.affectiva.com/


Outline

Brief review

Discuss dual-process theories of emotion

Introduce note of caution– Discuss evidence for 2 processes weaker than evidence suggest

– Studies overemphasize differences

Discuss emotion as a process

Examine appraisal and coping theory

In-class experiment on these themes


Review

Theory important– Makes predictions, proposes mechanisms, allows control

– Typically supported by substantial empirical research

But no grand unified theory of emotion– Different theories explain different aspects of emotion

But choice of theory has implications– E.g., for our labels in machine learning algorithm


Discrete or continuousRussell’s ‘80 circumplex model

Disgust Fear Surprise


Constructivist Models

Appraisal Models

Mind

Mind

Body

Body

How can we reconcile?

Is emotion a cause or consequence of thought?


Most emotion theories are dual-process theories

Emotion

Intellect

crooked lumbering animal, … the mate of

insolence and pride, shag-eared and

deaf, hardly yielding to whip and spur.”

lover of honor and modesty and

temperance, and the follower of true glory;

he needs no touch of the whip, but is

guided by word and admonition onlyThe Allegory of the Chariot


The brain makes decisions (e.g. constructs value)

by integrating signals from multiple systems

These multiple systems process information in

qualitatively different ways and in some cases

differentially weight attributes of rewards (e.g.,

time delay)

Dual Process Theories: more recent perspective


System 1 System 2Integration

Behavior

An (oversimplified) multiple systems theory


An example

Visceral

reward:

pleasure

Abstract

goal:

dietIntegration

Behavior

Would you like a piece of chocolate?


Individual differences (simple test)


Passions vs Interests (Smith)

Id vs Ego vs Superego (Freud)

Automatic vs Controlled (Schneider & Shiffrin, 1977; Benhabib & Bisin, 2004)

Hot vs Cold (Metcalfe and Mischel, 1979)

Impulsive vs Deliberative (Frederick, 2002)

Unconscious vs Conscious (Damasio, Bem)

Effortless vs Effortfull (Baumeister)

Doer vs Planner (Shefrin and Thaler, 1981)

Visceral vs Abstract (Loewenstein & O’Donoghue 2006; Bernheim & Rangel, 2003)

Mesolimbic dopamine vs PFC & parietal cortex (McClure et al, 2004)

System 1 vs System 2 (Frederick and Kahneman, 2002)

Variations on this theme


Affective system

fast

unconscious

reflexive

myopic

effortless

Analytic system

slow

conscious

reflective

forward-looking

(but still prone to error:

heuristics may be analytic)

self-regulatory

effortful and exhaustible

Commonalities between classification schemes


Mesolimbic dopamine reward system

Frontalcortex

Parietalcortex

Affective vs. Analytic Cognition

mPFC

mOFC

vmPFC


System 1 vs System 2: Kahneman

A useful fiction


A battle in your mind

Dual process theories often adopt the analogy

of two independent entities fighting for control

of your mind

Research then tries to identify which “horse” wins

• Developmental factors: children and teenagers are emotional

• Dispositional factors: some people are naturally emotional

• Cultural factors: some societies are naturally emotional

• Situational factors: some situations evoke emotional thinking

(e.g., when hungry, sleepy, rushed

But are these really independent systems?

Is there always just one winner?


Interactions

“Cognition” (System 2)

Sequential

Rule-based

Rational

“Emotion” (System 1)

Parallel

Associative

Intuitive

Some theories these systems influence each other in specific

ways?


Dual process theories


Sequential

Rule-based

Rational


Parallel

Associative

Intuitive


Dual process theories


Sequential

Rule-based

Rational


Parallel

Associative

Intuitive

Appraisal Theories


Claim

Psychological research often overemphasizes

independence of these systems

This apparent independence is an illusion of

experimental designs that try to emphasize differences

in thinking


Reductionism

Psychological studies typically try to show importance

of a single mechanism

Can lead one to overestimate importance of that factor

and underestimate contribution/interaction with other

factors

IV DV

Race

Gender

Degree

Sunny?

Hungry

Sleepy


Mechanically Separate

Ventral Medial/Orbital Prefrontal

Cortex damage

– Able to do simple laboratory cognitive

tasks

BUT show serious deficits

– Abnormalities in emotion

Severe impairments in judgment and

decision-making in real-life

– Sequential decision-making preserved

but becomes unfocused, non-goal-

directedPhineas Gage


Electrically Separate


Empirically separate Em & Cog “in the moment”(Clore, Schwarz)

Emotions inform decisions

But many experiments separate emotion from decision

– Induce an emotion: Play happy/sad/angry music

Read happy/sad/angry stories

– Make people perform an irrelevant task Buy something

Play ultimatum game

– Show logically irrelevant emotion biases decision making

But how often is emotion irrelevant to the task?


Empirically separate Em & Cog over time

Emotions unfold sequentially hand-in-hand with

cognitive processes

But many experiments break such sequences

– Explore “one-shot” decision tasks Lotteries (Reisenzein)

Ultimatum games

A common misinterpretation of this data

– Emotion is parallel and “unthinking”


One CAN separate emotion and cognition

– Its seductive Reflect longstanding theoretical and folk distinctions

Consistent with some data

– But I’ll argue this data has limited ecological validity (e.g., see also

Gigerenzer)

It is fun (and publishable) to show people are irrational

But how often does this occur in real workd

– But this leads to impoverish understanding of both Cognition w/o emotion is a broken thing

Maybe dual processes are a cognitive illusion


emotion AND cognition

The majority of everyday “thinking” involves

– Acting in a dynamic and evolving world

– Juggling multiple goals and preferences

– Confronting opportunities and threats

Emotion evolved hand and hand with cognition

– Two sides of the same system

Attempts to separate them leads to anomalous behavior

Yet that is what much of emotion psychology implicitly

or explicitly strives to do


(act on world) (act on self)

Emotion

CopingStrategy

ActionTendencies

“Affect”Physiological

Response

Problem-Focused Emotion-Focused

EnvironmentGoals/Beliefs/

Intentions

Some appraisal theories join both perspectives(Arnold, Lazarus, Frijda, Scherer, Ortony et al.)

Desirability

Expectedness

Controllability

Causal Attribution

Appraisal



Emotion

CopingStrategy

ActionTendencies


Response



Intentions


Desirability

Expectedness

Controllability

Causal Attribution

Resignation

Distancing

Wishful Thinking

Take action

Seek support

Cognition (System2)

Emotion(System 1)



Endogenous

Emotion

Incidental

Emotion

Emotional cognition: an example

Marsella and Gratch (2009), “EMA: A process model of appraisal dynamics,”

Journal of Cognitive Systems Research, 10(1), pp. 70-90.

Emotion about:

• Acting in a dynamic and evolving world

• Juggling multiple goals and preferences

• Confronting opportunities and threats

Emotion can be rational (e.g., Simon; Frank)

Emotion can be sequential (e.g., Schrer)

Emotion follows “rules” (e.g., Frijda)

../../Media/Delsarte/bird.mov

../../Media/Delsarte/bird.mov

Surprise Orient

Fear Retreat

Anger Attack

Empathy Protect

Affect (System 1) Decisions (System 2)

Marsella and Gratch (2009), “EMA: A process model of appraisal dynamics,”

Journal of Cognitive Systems Research, 10(1), pp. 70-90.

“Working Memory”Environment

Control Signals

Appraisal

Frames

Affective

State

Appraisal

Coping

Inference Action

Dynamics

Surprise

Fear

Anger

Empathy


Control Signals

Appraisal

Frames

Affective

State

Appraisal

Coping

Inference Action

Dynamics in

the world

Surprise

Fear

Anger

Empathy

Dynamics


Control Signals

Appraisal

Frames

Affective

State

Appraisal

Coping

Inference Action

Dynamics in

the world

Dynamics in perceived

world relationship

Surprise

Fear

Anger

Empathy

Dynamics


Control Signals

Appraisal

Frames

Affective

State

Appraisal

Coping

Inference Action

Dynamics in

the worldDynamics in perceived

world relationship

Dynamics

through action

Surprise

Fear

Anger

Empathy

Dynamics


Appraisal theory

For much of the rest of the class I’ll emphasize

appraisal theory (esp. work of Smith and Lazarus)– Emphasizes emotion as both an antecedent and consequence of

cognition Provides detailed description of factors that elicit emotion (appraisal)

Provides detailed description of how emotions can shape sequent cognition

Thus can unify appraisal and constructivist approaches

– Relatively easy it translate into a computer program

– Serves as the theory underlying my own work on affective computing

– Can help explain several aspects of emotion Why a given situation might produce a given emotion

Why an emotion might influence subsequent decisions

Why an expression might shape another’s decision

38

Theoretical Perspective:Appraisal Theory

Magda Arnold

– Emotion arises from an evolving subjective interpretation of

person’s relationships to their environment

– Well-suited to computational realization

• Emotion arises from series of judgments (appraisals) of how

some event impacts an agent’s goals

• Artificial intelligence good at doing this sort of thing

(Arnold, Lazarus, Frijda, Scherer, Ortony et al.)

39

EmotionAction

Tendencies“Affect”

PhysiologicalResponse


Intentions

Appraisal

Desirability

Expectedness

Controllability

Causal Attribution

Appraisal

All share the notion that events are “appraised”

Pattern of appraisal elicits affect


All disagree on specific appraisals


All agree people appraise same event differently

Different individual goals– UCLA fan vs USC fan

Different cultural norms and ideals– Importance of the individual vs. importance of the collective

Different appraisal styles


Appraisal Styles (e.g., Depressive appraisal style)


Locus of control: self vs. other

Internalizes blame, externalizes credit: “god’s plan”


Stability of control: changeable vs. constant

Views situation as unchangeable across time


Open problem in affective computing:Automatic detection of appraisal style

Lots of work in psychology doing manual coding– Predicts physical illness (Peterson & Seligman 1988)

– Predicts job productivity (Seligman & Schulman 1986)

– Depression (Girgus t al 1986)

Some work using shallow features (e.g. LIWC)– Style in political speeches (Pennebaker and Lay 2002)

– Mental health (Boals & Klein 2005)

Lots of room to do better


Coping is how individual responds to emotion

Coping shapes beliefs, desires and intentions



Emotion

CopingStrategy

ActionTendencies


Response



Intentions

Coping

Attempts to characterize How emotion shapes cognition

Play Game

Lose

Win

p=.8

p=.2

Utility= 20

Intention(play) EU=4

Rational models decouple preferences and beliefs

– Desires shouldn’t change beliefs (and vice versa)

e.g., Just wanting something shouldn’t make it true

– Preferences fixed over time

Wishful Thinking

Play Game

Lose

Win

p=.8

p=.2

Utility= 20

Intention(play) Sad

HopeJoy

Distancing

Fear

Utility= 10p=.6

p=.4

Resignation

• Coping serves to “confound” beliefs and desires

Emotion-biases on decision making (Loewenstein & Lerner, 2003)

Cognitive dissonance (Festinger57)

Motivated inference (Kunda87)

– Little attempt to computationally model (Marsella&Gratch; Dias)


People differ in “coping style”


Coping shapes beliefs, desires and intentions

Appraisal Emotion: – I’m afraid because I might lose

Emotion Coping– I don’t care about winning anyway

Coping Re-appraisal– I’m much happier now that I don’t care about wining

This is core idea behind most therapies for

depression (e.g., Cognitive Behavioral Therapy)

– Teach people better coping strategies


Application of Affective Computing

Teaching better coping strategies

University of Southern California, Information Sciences Institute

More serious example:Carmen’s Bright IDEAS (CBI)

Help mothers of pediatric cancer patients cope Teaching problem solving skills

Bright IDEAS (Identify, Develop, Evaluate, Act, See)

Reach a larger audience

Mothers learn through problem solving scenarios Story unfolds based on mother’s choices

Learning as a result of experiences with story’s characters

Agent-based interactive characters Model emotion, personality, dialogue, nonverbal behavior


CBI Goals

Teach Bright IDEAS IDEAS - Rational, Methodical technique

Bright attitude - Critical Emotional Factors

Self-efficacy, healthy coping styles

Problems appraised as challenges, not threats

Provide safe environment that facilitates exploration Low pressure

Manage stress, facilitate coping

Freedom to explore

Provide engaging, revealing experience Promote identification

Concretize and highlight pedagogy


CBI Photo Gallery

Act 2: User Interaction Act 3: Diana and Jimmy

Act 2: Gina and Carmen talkAct 1: Backstory of Carmen


CBI: Act 2 Excerpt (screen capture)


CBI Interaction: Rubber-band model

Key Pedagogical/Dramatic Junctures

Failure Success

User

DialogEmotions/

Thoughts

GinaCarmen

Story Progression

This isn’t

going

to help.

Success


Realizing the tug-of-war: Gina’s role

Gina “directs” socially aware dialog: Goal: Guide Carmen thru BI

Her dialog strategies & sub-strategies structure conversation

“Suggest BI, Guide through I-D-E-A-S”

“Identify (I step) a problem by asking a sequence of questions”

Dialog moves flexibly realize strategies

Dialog Moves: Suggest, Agree, Ask/Prompt, Re-Ask, Answer, Reassure, Agree/Sympathize, Praise, Offer-Answer, Clarify, Resign, Summarize

Moves organized in state machine

Gina’s moves sensitive to the cognitive and emotional state of

Carmen


Realizing tug-of-war: Carmen’s Role

Carmen’s emotions & coping drive her dialog

Gina asks about Diana’ uncontrollable tantrums

Threatens Carmen’s ego-ideal of being a good mother

Leads to Anxiety/Anger

Carmen copes by discounting significance of tantrums

“She is just being babyish”

May in turn lead to Guilt

Realization of Cognitve Appraisal Theory (Lazarus 91)

Same model underlies development of BI pedagogy

User influences Carmen’s emotion and coping choices

Allow pedagogically relevant exploration

Consequences played out dramatically


Dialog

Annotations

CBI Character Agent’s Brain


Experiment

Divide class in half

One half will close eyes while other does task

Take scale and pen

On next slide there is an image of a person

Use the scale to rate your immediate impressions

of this person

Other half of class will rate different picture


What was my experimental design?

IV?

DV?

lecture 4 emotion theory (continued)

Documents