lecture 3: theories of emotion - institute for creative...

CSCI 534(Affective Computing) – Lecture by Jonathan Gratch

Lecture 3: Theories of Emotion


Affective Computing in the news

Dacher Keltner

Berkeley


Outline

Review why emotion theory useful– Give some positive and negative examples

Introduce some features that distinguish different theories– Emotions as discrete or continuous

– Emotions as “atoms”, “molecules”, or “mxtures”

– Emotions as a consequence or antecedent of emotion

Review some specific influential theories

In-class “experiment” (3-unit students welcome to depart)


Why should we care about emotion theory?


What is a Theory

Theory explains how some aspect of human behavior

or performance is organized. It thus enables us to make

predictions about that behavior.– Provides a set of interrelated concepts, definitions, and propositions

that explains or predicts aspects of human behavior by specifying

relations among variables.

– Allows us to explain what we see and to figure out how to bring about

change.

– Is a tool that enables us to identify a problem and to plan a means for

altering the situation.

– Create a basis for future research. Researchers use theories to form

hypotheses that can then be tested.

– Creates a basis for building software: suggests what variables are

important to measure and how they relate to each other


Does a learned model count as a theory?

We’ll learn about machine learning approaches– Collect bunch of data

– Look at lots of features and try to predict some outcome

Allow us to make predictions?– Yes

Give insight into underlying mechanism?– Not typically (black box). But can answer what features are relevant

Indicate how to bring about change?– Not typically

Input

Features(events in a

video game)

Predicted

output


And can be misleading: Famous example

In ‘80s, the Pentagon wanted to harness computer technology to make

their tanks harder to attack.

The research team went out and took 100 photographs of tanks hiding

behind trees, and then took 100 photographs of trees - with no tanks.

They trained a neural network. It reached near-perfect accuracy

Independent testing showed all “no tank” photos taken on sunny day

and all “tank” photos taken on cloudy day

Because neural network was “black box”, this not easy to discover


Affective Computing Example

Last week, showed you system that tIn

tries to recognize nonverbal signs of depression and PTSD

We collected data from two populations

– Craigslist (and online job-recruiting service)

– US Vets: organization that provides mental-health service for former soldiers

Tried machine learning approach

Discovered vocal pitch a strong predictor of depression

– Lower pitch predicted depression severity

– Not predicted by existing theory

Turns out there was big imbalance in our data

– US Vets had highest rates of depression

– US Vets also had highest rates of Males (most soldiers are male)

– We actually “discovered” that men speak with a lower pitch


Advantages of building on theory

Theory makes explicit the mechanisms that (are

claimed to) underlie some behavior– Allows us to explain what we see and to figure out how to bring about

change

Theories (typical) have good empirical support– The theories we will discuss are supported by dozens of empirical

studies

– They may still be incorrect of insufficient but are unlikely to suffer the

sort of mistakes we just discussed


10

World Events Mental State(beliefs, goals)

Example: Appraisal Theory

Argues for importance of three

interrelated concepts

• World events

• Mental state (e.g. goals)

• Emotional Response

If we know two of these

variables, we can make

predictions about the third

Response= f(Env., Mind)

Body

Expression

Action Tendency

Physiological Response


11

EnvironmentBeliefs,

Goals

E.G.: Generating Emotional Response R=f(E,M)

COMPUTER PREDICITONS:

• Computer could predict what

emotion a person might hold

• Computer could generate a

believable emotion to user

Emotional Response

Expression

Action Tendency



12

Emotional Response

Expression

Action Tendency


EnvironmentBeliefs,

Goals

E.G.: inferring emotional antecedents M=f-1(E,R)Reverse Appraisal

COMPUTER PREDICITONS:

• Computer could predict what

goal person has (i.e., what

team are they rooting for


“Angry” “Happy”

“Sad”“Apa-

thetic”

Influential theory: Galen’s 4-process model of emotion

“Valence”

“Arousal”

Posits 4 “prototypical emotions”.

Emotions organized in 2-dimensional space (valence, arousal)

Argues emotions tend to transition along arrows.

http://en.wikipedia.org/wiki/Image:Four_temperament_b.PNG









“Angry” “Happy”

“Sad”“Apa-

thetic”

Galen’s 4-process model of emotion

“Valence”

“Arousal”

YB

BB

Bl

Ph

Prototypical emotions associated with 4 specific physiological systems










Popular 2-dimensional model of emotion

“Valence”

“Arousal”

YB

BB

Bl

Ph

Happy

Apathetic

Angry

Sad

Each prototypical state associated with a characteristic expression

C S

M P










Again, this theory affords implementation and prediction

Dimension 2

Dim

en

sio

n 1

YB

BB

Bl

Ph

Happy

Apathetic

Angry

Sad

C S

M P

Recognition “language”– 4 “prototypical” emotion labels but

– 2 dimensions

Predictions– If we recognize Anger expect YB is active

– If recognized Anger, don’t expect transition to

Apathy

– If BB active, expect sad expressions and self-

report of Sadness

Control– Can create Apathy by activating Ph system or

suppressing other systems

– Can’t control Ph by activating Apathy










Choleric Sanguine

Melan-

choly

Phleg-

matic

Theory of the Four Humoursby Galen of Pergamun (c. 180AD)

Wetness

Tem

pera

ture

Yellow

Bile

Black

Bile

Blood

Phlegm

Wet

(Water)

Dry

(Earth)

Cold

(Air)

Hot

(Fire)

A Hippocratic physician would prescribe treatment to void the

body of imbalanced humor. if it was a fever -- a hot, dry disease --

the culprit was yellow bile or blood. So, the doctor could reduce

this by, e.g., bleeding the patient, or increase its opposite,

phlegm, by prescribing cold baths.










Proof of the Theory of the Four Humoursby Galen of Pergamun (c. 180AD)

1793 an epidemic of yellow fever struck Philadelphia

Benjamin Rush (signer of Declaration of Independence)

treated by vigorous bloodletting

Each patient that recovered and survived served to prove the

theory

Each patient that died confirmed that the patient was too ill for

the treatment to work

Any issue with this?– Example of confirmation bias: a common decision-making bias

– Another example: “proof that aliens have landed on earth”

https://www.google.com/webhp?sourceid=chrome-instant&ion=1&espv=2&ie=UTF-8#q=proof+that+aliens+have+landed+on+earth


Falsifiability

A good theory is falsifiable– Falsifiability or refutability of a theory is an inherent possibility to

prove it to be false.

– Theories that are so vague they can explain anything (ex. Psychic

readings) are not falsifiable

– The more specific a theory it is, the more likely it is falsifiable

Galen’s theory is falsifiable (and has be falsified)– Even if Benjamin Rush failed to test it properly

Karl Popper: on falsifiability, testability

‘What characterises the empirical method is its manner of

exposing to falsification, in every conceivable way, the

system to be tested. Its aim is not to save the lives of

untenable systems but, on the contrary, to select the one

which is by comparison the fittest, by exposing them all to

the fiercest struggle for survival’


Why should we care about emotion theory

Provides a definition of “emotion” and other related concepts

that influence, or are influenced by emotion, and thus a

starting point for affective computing

Unfortunately, psychology hasn’t sorted it all out yet

– Different theories suggest different concepts and relationships between them

E.g., Say we want to recognize emotion

– Give labeled data to machine learning algorithm

– But what are the labels?

Joy vs. Hope vs. Fear?

Positive vs. Negative?

Affective computing researchers must make educated guess

about which theory to use

– But their success or failure can inform research in the social sciences


21

Human

Behavior

Theory

• Psychology

• Linguistics

• Neuroscience

• Economics

Data

Integrated

“Test bed”

Theory

• e.g., Rapport (positive,

contingent, nonverbal feedback)

facilitates conflict

resolution

Affective computing is interdisciplinary science

MRE SASO-ST Gunslinger DCAPSRapport

RapportEmbed capability

within interactive

virtual human

testbed


22

Human

Behavior

Integrated

“Test bed”

Human

Studies

Affective computing is interdisciplinary science

Verify Implementation

• Consistent with prior

findings?

• Treated “as if” real

Test theoretical predictions

MRE SASO-ST Gunslinger DCAPS

Inform theoretical

debate in social

science

Rapport


For us,A theory should answer “What is emotion?”

Emotion is a feeling

Emotion is a state (of physiological arousal)

A brain process that computes the value of an experience --- Le Doux

A word we assign to certain configuration of bodily states, thoughts, and

situational factors – Feldman Barrett.

God’s punishment for disobedience -- St Augustine


But also, what is emotion NOT?

From Scherer (optional reading)


So what is the accepted theory of emotion?

Unfortunately, none exists


Why no grand unified theory of emotion?


What is an emotion?

Components of emotion

Emphasizes that emotion potentially impacts several aspects

– Cognitive: influences or influenced by thinking

– Physiological: related to hormones, heart-rate, sweating…

– Expressive: relates to facial expressions, posture, vocal features

– Motivation: relates to goals and drives

– Feeling: relates to conscious awareness being in an emotional state


What is an emotion?

Phases of emotion: Emphasizes that emotions have “stages”

– Low-level: automatic cognitive processes (e.g., reflexes)

– Hi-level: deliberate, conscious cognitive processes

– Goals/need setting

– Examining action alternative: decision-making/action-selection

– Behavior preparation

– Behavior execution

– Communication with other


What is an emotion?

Different theories emphasize different aspects:

– Appraisal theories emphasize cognitive antecedents of emotion

– Discrete emotion theories emphasize physiological and expressive

consequences of emotion

Affective computing researchers tend to draw on different

theories depending on the aspects they focus on

– E.g : emotion recognition techniques often draw upon discrete

emotion theory and avoid appraisal models


What is an emotion: theoretical disagreements

Different theories can be distinguished by how they

chose to define emotion with respect to the

previously-mentioned components and phases

– Is emotion discrete or continuous?

– Is emotion an “atom” or “molecule”? (Barrett)

– Is emotion an antecedent or consequent of cognition?


Emotions as discrete categories,

biologically fixed, universal to all humans

and many animals

Basic Emotions: Anger, disgust, fear,

happiness, sadness, surprise

Rene Decartes, Silvan Tomkins, Paul

Ekman

Emotions are a combination of several

psychological dimensions

Wilhelm Wundt, James Russell, Lisa

Feldman Barrett


Some discrete emotion theories

Tomkins

– Excitement, joy, surprise, distress,

anger, fear, shame,

dissmell (reaction to bad smell),

disgust (reaction to bad taste)

Ekman

– Sadness, happiness, anger, fear,

disgust, and surprise,

sometimes includes contempt

E.g. Le Doux fear

circuit


Some Dimensional models

Russell & Mehrabian’s ‘77 PAD model (pleasure,

arousal, dominance) Russell’s ‘80 circumplex model

High self-control ↔“letting go”

Mania ↔ depression


Implications for classification / measurement

Continuous



Discrete

Disgust Fear Surprise

Continuous


Emotion components are tightly-coupled

and can be treated as a circuit linking

stimuli and response

Jaak Panksepp, Joseph LeDoux, Paul

Ekman

Emotions are defined by loose

configuration of different components

Phoebe Ellsworth, Klaus Scherer, Lisa

Feldman Barrett

Atom Molecule or Mixture



If emotion is atomic circuit, all components should be aligned

– i.e., Facial expressions, physiological response and felt emotion

should be consistently-aligned with each other

– “Emotion” can refer to the overall circuit but can be measured by any

of the components

– Measured expressions should predict physiology and felt emotion

– Multi-modal recognition should perform the best



If emotion is atomic circuit, all components should be aligned

– i.e., Facial expressions, physiological response and felt emotion

should be consistently-aligned with each other

– “Emotion” can refer to the overall circuit but can be measured by any

of the components

– Measured expressions should predict physiology and felt emotion

– Multi-modal recognition should perform the best

If emotion a molecule or mixture, components not aligned

– Allow that components influence each other but may be out of sync

– Expressions need not accurately reflect physiology and felt emotion

– Constructivist Theories (Feldman Barrett): Emotion is a label we

assign to our sensed physiological state

– Appraisal theories (Scherer & Ellsworth): Emotion is a label a scientist

might apply when different components align in a prototypical way


Atom or Mixture

Discrete: redundancy across channels– Multimodal should be strictly better than unimodal

Predicted

output


Atom or Mixture

Discrete: redundancy across channels– Multimodal should be strictly better than unimodal

– Late fusion should be great

Mixture: not so fast…– Or at least, association between modalities and predicted emotion is complex

Predicted

output


Thought precedes emotion. Emotion

precedes and motivates behavior

Walter Cannon, Phoebe Ellsworth, Klaus

Scherer

Behavioral response precedes our

labelling the situation as emotional

William James, Stanley Schachter, Lisa

Feldman Barrett

Top down(e.g. Appraisal Theory)

Behav. Drives emotion(e.g., constructivist theories)


Thought precedes emotion. Emotion

precedes and motivates behavior

Walter Cannon, Phoebe Ellsworth, Klaus

Scherer

Behavior and body response precedes

and motivates emotion and cognition

William James, Stanley Schachter, Lisa

Feldman Barrett

Top down(e.g. Appraisal Theory)

Bottom up (e.g., constructivist theories)


Appraisal Theory

“Bottom up” theories argue “seeing the bear” produces fear-like

reactions automatically

What if we knew the bear was friendly?

What if we knew the bear was chained up?Magda

Arnold


Appraisal Theory

Appraisal models emphasize the prior beliefs and goals determine

shape emotional responses

Explain this by arguing that cognitive processes ESSENTIAL in

initiating emotional responses

World events are “appraised” along a number of dimensions:

– Is the event good or bad with respect to my goals

– Did I expect the event

– Can I control the event

– Who do I blame for the event

Different patterns of appraisal will lead to different emotions

– I blame someone else for something bad Anger


Some Appraisal Theories

Ortony, Clore and Collins (OCC) Appraisal Variables

• desirability

•appealingness

•praiseworthyness

•certainty


Some Appraisal Models

Scherer sequential checking theory

Appraisal Variables

• Relevance

• Implication

• Coping potential

• Normative significance


Appraisal theory takeaway

Emotions arise from appraisal of goals and beliefs– Emphasizes centrality of beliefs, desires and intentions to emotion elicitation

Event has no meaning in of itself

Emotion arises from how event impacts goals and beliefs

Same event will have different meaning to different people


Classic example


But not always so simple

Belief: I’m standing in a room


But not always so simple

Belief: I’m standing in a room

Does this contradict appraisal theory?


Example (Constructivist Theory)

Argues first step in the experience of emotion is

physiological arousal– Seeing the bear triggers low-level automatic reactions such as arousal

and running away

We next try to find a label to explain our feelings,

usually by looking at what we are doing (behavior)

and what else is happening at the time of arousal

(environment)

Thus, we don’t just feel angry, happy, etc. We

experience general feeling and then decide what

they mean (a specific emotion)


Schachter 2-factor theory


Practical implication


Basic emotion

Theory

Constructivist

Theory

Appraisal

Theory


Lesson: Definitions matter

Geocentricity

– Placing earth at center of universe makes it difficult to predict motion

of the planets

Alchemy

– All substance can be decomposed into earth, water, air and fire

making it difficult to predict consequences of chemical reactions

Point:

– Theory important: allows us make specific predictions and explain

variance

– Important steps on way to deeper understanding

– Recognize that technological choices depend (implicitly or explicitly)

on (folk or scientific) theoretical assumptions and failure of the

technology may reflect problems with theory, not software


End of main

lecture


In-class exercise

Split class into 3 groups

• Need group of four volunteers to watch a video

• Most of class will stay put and watch this group

• Need one more group of four to watch the class

I’ll give out some handouts

• First group will mark down how they feel watching the video

• Class will guess what the first group is feeling based on their reactions

• Last group will guess what the first group is feeling based on class’s

reactions

NO TALKING


Discussion

Classification– What featured did you use to identify the felt emotion

Dimensions vs. Basic emotions– Which framework best captured the “meaning” of the interaction

Observers– Why were (or weren’t) the 3rd group able to infer what is going on

Mirroring

How do you think a computer would do?


65

lecture 3: theories of emotion - institute for creative...

Documents