issues in emotion-oriented computingreichardt/itemotion/...high-level: emotions in bdi models...
TRANSCRIPT
Issues in Emotion-oriented ComputingTowards a shared understanding
Marc Schröder & Roddy Cowie
http://emotion-research.net
humaine - emotion-research.net 2
HUman-MAchine Interaction Network on Emotions
EU network of excellence 2004-2007scientific and technological groundwork for emotion-oriented computing
interdisciplinary understandingprincipled approaches
humaine - emotion-research.net 3
Sub-areas in Emotion-oriented Computing
Empirical evidenceEmotion
detection
planning
Emotional language
evaluation
Description of
emotion
Ethicsof emotion
Emotion detection
Planning action
Theory ofemotional processes
Raw empirical
data
Emotion incomplexmedia
Usability / evaluation
Synthesis of basicsigns
Application oriented
Core technologies
Psychologically oriented
humaine - emotion-research.net 4
Sub-areas in Emotion-oriented Computing
Empirical evidenceEmotion
detection
planning
Emotional language
evaluation
Description of
emotion
Ethicsof emotion
Emotion detection
Planning action
Theory ofemotional processes
Raw empirical
data
Emotion incomplexmedia
Usability / evaluation
Synthesis of basicsigns
Application oriented
Core technologies
Psychologically oriented
Theory ofemotional processes
humaine - emotion-research.net 5
Theory of Emotional Processes
what to study: states that are relevant to emotion-oriented computingterminological clarification: clear definition of “emotions”, “moods”, …practically useful methods for describing emotional states
humaine - emotion-research.net 6
“Less than 10% of human life is truly unemotional”
Cowie et al., QUB:reality TV show “Cast away”ratings of unedited tapes
truly unemotional episodesare rareemotion in the strong senseis rare
need concepts to describe the ~80% “elements of emotion”
humaine - emotion-research.net 7
Aspects of “emotional life”
Cowie, O’Neill et al., QUB
emergent emotion (full-blown)emergent emotion (suppressed)moodpartial emotion (topic shifting)partial emotion (simmering)attitudeestablished emotioninterpersonal stancesaltered states of arousal/control/seriousness
humaine - emotion-research.net 8
Aspects of “emotional life”
Cowie, O’Neill et al., QUB
emergent emotion (full-blown)emergent emotion (suppressed)moodpartial emotion (topic shifting)partial emotion (simmering)attitudeestablished emotioninterpersonal stancesaltered states of arousal/control/seriousness
DefinitionThis refers to states between a mood and an emergent emotion, where a person is in a single kind of emotional state, but it is not fixed on a single thing or issue – instead the emotion attaches to one object, then another, then another, in quite rapid succession. Example“Angry and tired from previous day. Getting annoyed when styling hair because it wouldn’t stay. Angry at dog trying to get out, and things at work.”
humaine - emotion-research.net 9
Labels for “emotional life”
Negative & forceful Anger Annoyance Contempt Disgust Irritation Negative & not in control Anxiety Embarrassment Fear Helplessness
Powerlessness WorryNegative thoughts Doubt Envy Frustration Guilt ShameNegative & passive Boredom Despair Disappointment Hurt Sadness
Agitation Shock Stress TensionPositive & lively Amusement Delight Elation Excitement Happiness
Joy Pleasure
Caring Affection Empathy Friendliness Love Positive thoughts Courage Hope Pride Satisfaction TrustQuiet positive Calm Content Relaxed Relieved Serene
Reactive Interest Politeness Surprise
Cox et al.: List consolidated from Geneva, LIMSI, Belfast sources
humaine - emotion-research.net 10
Empirical evidenceEmotion
detection
planning
Emotional language
evaluation
Description of
emotion
Ethicsof emotion
Emotion detection
Planning action
Theory ofemotional processes
Raw empirical
data
Emotion incomplexmedia
Usability / evaluation
Synthesis of basicsigns
Application oriented
Core technologies
Psychologically oriented
Sub-areas in Emotion-oriented Computing
Raw empirical
data
humaine - emotion-research.net 11
Emotions in Empirical Data
Acted
Induced
TV interviews
In action
humaine - emotion-research.net 12
Emotions in Empirical Data
“supportive” datato fill in gaps in theory
“provocative” datato challenge preconceptions
develop suitable labelling schemes
humaine - emotion-research.net 13
Empirical evidenceEmotion
detection
planning
Emotional language
evaluation
Description of
emotion
Ethicsof emotion
Emotion detection
Planning action
Theory ofemotional processes
Raw empirical
data
Emotion incomplexmedia
Usability / evaluation
Synthesis of basicsigns
Application oriented
Core technologies
Psychologically oriented
Sub-areas in Emotion-oriented Computing
Emotion detection
humaine - emotion-research.net 14
Emotion Detection: Multimodal Features
humaine - emotion-research.net 15
Emotion Detection: Selected Challenges
Integration of modalitiesdifferent time scales for analysis
face = still picturesvoice = discrete chunks on time scalephysiology = slow, continuous variation over time
Measure of success?same errors as humans?
dialogue system
more accurate than humans?stress detection
humaine - emotion-research.net 16
Empirical evidenceEmotion
detection
planning
Emotional language
evaluation
Description of
emotion
Ethicsof emotion
Emotion detection
Planning action
Theory ofemotional processes
Raw empirical
data
Emotion incomplexmedia
Usability / evaluation
Synthesis of basicsigns
Application oriented
Core technologies
Psychologically oriented
Sub-areas in Emotion-oriented Computing
Synthesis of basic signs
humaine - emotion-research.net 17
Greta agentPelachaud et al.,Univ. Paris 8
Emofilt speech synthesis filterBurkhardt, T-Systems
Synthesis of Signs of Emotion:Parametrisable Models of Body, Face and Voice
humaine - emotion-research.net 18
Synthesis of Signs of Emotion
make embodied conversational agents (ECAs) behave emotional
required key capabilities:Coordination of signs in multiple modalities ExpressivityCognitive influences on action (attention model)Creating affective awarenessBackchannel & Feedback behaviour
specify architecture requirementstowards proof-of-concept sub-systems
humaine - emotion-research.net 19
Attention Modelling for ECAs
crucial for initiating and maintaining communicationvisual salience
contrast, color, …facesemotionalexpressions
Peters et al., Univ. Paris 8
humaine - emotion-research.net 20
Empirical evidenceEmotion
detection
planning
Emotional language
evaluation
Description of
emotion
Ethicsof emotion
Emotion detection
Planning action
Theory ofemotional processes
Raw empirical
data
Emotion incomplexmedia
Usability / evaluation
Synthesis of basicsigns
Application oriented
Core technologies
Psychologically oriented
Sub-areas in Emotion-oriented Computing
Planning action
humaine - emotion-research.net 21
Emotion in Planning and Action
conceptual and practical work on cognitive architectures with emotion
low-level / sub-symbolic / reactivehigh-level / symbolic / deliberativehybrid: bridging the gap
humaine - emotion-research.net 22
Low-level Perception-action Loop:Emergent “aggression”
robots need two resources to survivewander around avoiding objectsblue robot: bumper sensitivity reduced when deficit too largeresulting behaviour: pushing the otheraway from resourceethological analysis: “aggressive” behaviour
Cañamero et al.,Univ. Hertfordshire
humaine - emotion-research.net 23
Low-level Perception-action Loop:Imprinting a “desired perception”
Imprint visual perception of “optimal” box distanceRobot will follow box in this distance,goal = re-establish desired perceptionK. Lorenz
Blanchard & Cañamero, Univ. Hertfordshire
humaine - emotion-research.net 24
High-level: Emotions in BDI models
Reasoning about emotionsinterpret emotional impactof previous dialogue moveguess possible emotional impactof next dialogue move
Based on an appraisal model (e.g., OCC)link events to emotional meaning
humaine - emotion-research.net 25
Empirical evidenceEmotion
detection
planning
Emotional language
evaluation
Description of
emotion
Ethicsof emotion
Emotion detection
Planning action
Theory ofemotional processes
Raw empirical
data
Emotion incomplexmedia
Usability / evaluation
Synthesis of basicsigns
Application oriented
Core technologies
Psychologically oriented
Sub-areas in Emotion-oriented Computing
Emotion in
complex media
humaine - emotion-research.net 26
Emotion in Complex Media
closest to applicationsexploratory work
automatic generation of potentially funny slogansplaying with semantic ambiguities in languagehuman has to decide which slogans are funnyExample: slogan for soft drinks: “Thirst come, thirst served!” (Stock et al., itc-IRST)
persuasive ECA communicationpolitenesslying behaviour
humaine - emotion-research.net 27
Generating Lying Expressions in an ECA
GAMBLE dice gameRehm et al.,Univ. Augsburg
humaine - emotion-research.net 28
Empirical evidenceEmotion
detection
planning
Emotional language
evaluation
Description of
emotion
Ethicsof emotion
Emotion detection
Planning action
Theory ofemotional processes
Raw empirical
data
Emotion incomplexmedia
Usability / evaluation
Synthesis of basicsigns
Application oriented
Core technologies
Psychologically oriented
Sub-areas in Emotion-oriented Computing
Usability/ evaluation
humaine - emotion-research.net 29
Usability of Emotion-oriented Systems
existing usability criteria do not applye.g., control, predictability, transparency
Develop criteria and methods to evaluate affective interaction systems
“sensual” feedback methodsWizard-of-Oz studies extended think-aloud protocol
humaine - emotion-research.net 30
“Sensual” Feedback Methods for Usability Testing
SenToy puppetPaiva et al., INESC-ID
Colour and shapeHöök et al., KTH
humaine - emotion-research.net 31
Wizard of Oz Method: ECA-based Dialogue
study user behaviouruser-centered design loop
design->test->redesign
explore affective interactionsocial relationship / empathy
de Rosis et al.,DI-BARI
User interface
Wizard interface
humaine - emotion-research.net 32
Empirical evidenceEmotion
detection
planning
Emotional language
evaluation
Description of
emotion
Ethicsof emotion
Emotion detection
Planning action
Theory ofemotional processes
Raw empirical
data
Emotion incomplexmedia
Usability / evaluation
Synthesis of basicsigns
Application oriented
Core technologies
Psychologically oriented
Sub-areas in Emotion-oriented Computing
Ethics of emotion
humaine - emotion-research.net 33
Ethics in HUMAINE
Data collection: emotional data is sensitivePotential abuse of emotionally competent systems
automatic persuaderssurveillance systems…
Address ethical issues proactively
humaine - emotion-research.net 34
Ethics in HUMAINE
Ethics research is new to this communityunrealistic to formulate a closed set of criteriaapproach: “Principlism” (Beauchamps & Childress, 2001)
Four Level Modelnonmaleficence, autonomy, beneficance, justiceuniversally shared ethical standardsrelies on “seeing things right”
Install ethical committeebecause every case is different
humaine - emotion-research.net 35
Conclusions
Working on these different levelsremoving obstacles in cooperation(“make things fit together”)building basis for future worksuggesting sensible research agendas
Applications Coretechnologies
Psychologicaltheories ofemotion
humaine - emotion-research.net 36