expressive emotional eca ✔ catherine pelachaud ✔ christopher peters ✔ maurizio mancini
TRANSCRIPT
Expressive Emotional Expressive Emotional ECAECA
✔ Catherine PelachaudCatherine Pelachaud✔ Christopher PetersChristopher Peters✔ Maurizio ManciniMaurizio Mancini
Expressive Emotional Expressive Emotional ECAECA
✔ Emotion behavior modeling:Emotion behavior modeling:✔ SignalsSignals✔ Temporal course of signalsTemporal course of signals✔ Expressive qualitiesExpressive qualities
✔ Often in ECA systems: Often in ECA systems: ✔ only modeling of signals is consideredonly modeling of signals is considered✔ 6 ‘universal’ expressions6 ‘universal’ expressions
Large variety of facial Large variety of facial expressions (Kollias et expressions (Kollias et
al)al)✔ Archetypal expressions modeling using Archetypal expressions modeling using
FAPs MPEG-4FAPs MPEG-4✔ Estimation of the range of variations of Estimation of the range of variations of
FAPs in archetypal expressionsFAPs in archetypal expressions✔ Intermediate expressions: rule-based Intermediate expressions: rule-based
computational model based oncomputational model based on✔ Whissell’s representation:Whissell’s representation:
✔ Activation, evaluation, Activation, evaluation, ✔ Plutchik’s angular measurePlutchik’s angular measure
Laban Movement AnalysisLaban Movement Analysis
✔ 5 major components:5 major components:✔ BodyBody: part of the body in used: part of the body in used✔ SpaceSpace: description of the directions and : description of the directions and
paths of motionpaths of motion✔ ShapeShape: changing forms of the body: changing forms of the body✔ EffortEffort: dynamics of the body moves: dynamics of the body moves✔ RelationshipRelationship: modes of interaction with : modes of interaction with
oneself, others and the environment oneself, others and the environment (facing, contact…)(facing, contact…)
✔ Textual and symbolic language of Textual and symbolic language of movement descriptionmovement description
EMOTE (Badler EMOTE (Badler et al)et al)
✔ EMOTE model for Effort and EMOTE model for Effort and ShapeShape
✔ Based on Movement Observation Based on Movement Observation ScienceScience
✔ Laban Movement AnalysisLaban Movement Analysis✔ Computational model of Effort Computational model of Effort
and Shape componentsand Shape components
EffortEffort
✔ Four motion factors: space, Four motion factors: space, weight, time, flowweight, time, flow✔ SpaceSpace✔ WeightWeight✔ TimeTime✔ FlowFlow
✔ Each factor ranges from:Each factor ranges from:✔ IndulgingIndulging in the quality in the quality✔ FightingFighting against the quality against the quality
EffortEffort✔ SpaceSpace: indirect : indirect direct direct
✔ Waving away bugs Waving away bugs pointing to a pointing to a particular spotparticular spot
✔ WeightWeight: light : light strong strong✔ Feather movement Feather movement punching punching
✔ TimeTime: sustained : sustained sudden sudden✔ Stretching Stretching grabbing a falling object grabbing a falling object
✔ FlowFlow: free : free bound bound✔ Waving widely Waving widely moving in slow motion moving in slow motion
ShapeShape
✔ Three distinct qualities of change Three distinct qualities of change in the form of movement:in the form of movement:✔ Shape flowShape flow: mover’s attitude toward : mover’s attitude toward
the changing relationship among the changing relationship among body partsbody parts
✔ Directional movementDirectional movement: mover’s : mover’s intent to bridge the action to a point intent to bridge the action to a point in the environmentin the environment
✔ ShapingShaping: mover’s carving or molding : mover’s carving or molding attitude with the environmentattitude with the environment
ShapeShape
✔ Three dimensions:Three dimensions:✔ HorizontalHorizontal: spreading : spreading enclosing enclosing
✔ Opening arms to embrace Opening arms to embrace clasping clasping someone in a hugsomeone in a hug
✔ VerticalVertical: rising : rising sinking sinking✔ Reaching for something in a high shelf Reaching for something in a high shelf
stamping the floor with indignationstamping the floor with indignation✔ SagittalSagittal: advancing : advancing retreating retreating
✔ Reaching out to shake hand Reaching out to shake hand Avoiding a Avoiding a punchpunch
Expressive Expressive Communicative Communicative
Gestures (Hartmann Gestures (Hartmann et al)et al)
✔ Expressivity parameters:Expressivity parameters:✔ Spatial: amplitude of movementSpatial: amplitude of movement✔ Temporal: duration of movementTemporal: duration of movement✔ Power: dynamic property of movementPower: dynamic property of movement✔ Fluidity: smoothness and continuity of movementFluidity: smoothness and continuity of movement✔ Repetitiveness: tendency to rhythmic repeatsRepetitiveness: tendency to rhythmic repeats✔ Overall Activation: quantity of movement across modalitiesOverall Activation: quantity of movement across modalities
✔ Expressivity modeling: modal and intermodalExpressivity modeling: modal and intermodal
Emotional Emotional ModelingModeling
• Carmen's Bright IDEAS ( Stacy Marsella, USC)
• Interactive drama
• Agent exhibits gestures based on their emotional states and personality traits
FantasyA (Ana FantasyA (Ana Paiva, Inesc)Paiva, Inesc)
• Role-playing game
• The emotion detected from user drives the behavior of one of the agents in the video game
SenToy (Ana Paiva, SenToy (Ana Paiva, Inesc)Inesc)
• SenToy is a tangible interface device, used to influence emotional behaviour in FantasyA
• SAFIRA: EU project
Behavior ModelingBehavior Modeling
✔ Facial expression:Facial expression:✔ Use facial analysis techniques/motion Use facial analysis techniques/motion
capture to look at:capture to look at:✔ Value of facial animation parameters Value of facial animation parameters
(muscular contraction, FAP…)(muscular contraction, FAP…)✔ Pattern of movement for a given parameterPattern of movement for a given parameter✔ Relation of facial animation parameters Relation of facial animation parameters
within a same expressionwithin a same expression✔ Coarticulation of movements between Coarticulation of movements between
expressionsexpressions
Gesture Gesture
✔ Gesture specification often based on sign Gesture specification often based on sign language gesture specification:language gesture specification:✔ Hand descriptionHand description✔ Palm orientationPalm orientation✔ Wrist position Wrist position ✔ Movement: wrist, elbow, finger…Movement: wrist, elbow, finger…
✔ Parameters characterizing quality of Parameters characterizing quality of movement: Laban, sub-set…?movement: Laban, sub-set…?
✔ Parameters act on overall animation Parameters act on overall animation (Emote) or on gesture phases (Emote) or on gesture phases (preparation, stroke, …)?(preparation, stroke, …)?
Behavior Behavior
✔ Gaze: Gaze: ✔ Direction, gaze length, mutual gaze, ratio of Direction, gaze length, mutual gaze, ratio of
gaze avoidance/gaze look atgaze avoidance/gaze look at✔ Body:Body:
✔ Body posture, overall movement qualityBody posture, overall movement quality✔ Voice: Voice:
✔ Paralinguistic parametersParalinguistic parameters✔ Text Text
Signals Signals analysis/generationanalysis/generation
✔ Use analysis method to:Use analysis method to:✔ Get information on parametersGet information on parameters✔ Multimodal integrationMultimodal integration✔ Signals synchrony across modalitiesSignals synchrony across modalities✔ Build computational model for generationBuild computational model for generation
✔ Get the expressive quality of behaviorGet the expressive quality of behavior✔ Signals generation should not be done on Signals generation should not be done on
one frame only but on a temporal one frame only but on a temporal intervalinterval
Conclusion Conclusion
✔ Emotion is shown through:Emotion is shown through:✔ SignalsSignals✔ Their temporal courseTheir temporal course
✔ Which parameters to look at? Which are Which parameters to look at? Which are the perceptually relevant ones?the perceptually relevant ones?
✔ How to describe:How to describe:✔ The signals?The signals?✔ Their evolution through time?Their evolution through time?✔ Correlation between modalities?Correlation between modalities?
Conclusion Conclusion
✔ Relation between intensity of emotion Relation between intensity of emotion and behaviors?and behaviors?✔ Selection of behaviorsSelection of behaviors✔ Quality of behaviorsQuality of behaviors✔ Duration of behaviorsDuration of behaviors✔ Behaviors across modalitiesBehaviors across modalities