affective computing a seminar presentation by karthik raman, 06005003 adith swaminathan, 06005005...

44
Affective Computing Affective Computing A Seminar Presentation by A Seminar Presentation by K K arthik Raman, 06005003 arthik Raman, 06005003 A A dith Swaminathan, 06005005 dith Swaminathan, 06005005 O O mkar Wagh, 06005006 mkar Wagh, 06005006 S S amhita Kasula, 06D05014 amhita Kasula, 06D05014 “There can be no knowledge without emotion. We may be aware of a truth, yet until we have felt its force, it is not ours. To the cognition of the brain must be added the experience of the soul.” Arnold Bennett (British novelist, playwright, critic, and essayist, 1867-1931)

Upload: ashley-wright

Post on 25-Dec-2015

221 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Affective Computing A Seminar Presentation by Karthik Raman, 06005003 Adith Swaminathan, 06005005 Omkar Wagh, 06005006 Samhita Kasula, 06D05014 “There

Affective ComputingAffective Computing

A Seminar Presentation byA Seminar Presentation by

KKarthik Raman, 06005003arthik Raman, 06005003

AAdith Swaminathan, 06005005dith Swaminathan, 06005005

OOmkar Wagh, 06005006mkar Wagh, 06005006

SSamhita Kasula, 06D05014amhita Kasula, 06D05014

“There can be no knowledge without emotion. We may be aware of a truth, yet until we have felt its force, it is not ours. To the cognition of the brain must be added the experience of the soul.”  Arnold Bennett (British novelist, playwright, critic, and essayist, 1867-1931)

Page 2: Affective Computing A Seminar Presentation by Karthik Raman, 06005003 Adith Swaminathan, 06005005 Omkar Wagh, 06005006 Samhita Kasula, 06D05014 “There

AbstractAbstract

Affective Computing is a field of research in Affective Computing is a field of research in AI dealing with emotions and machines. AI dealing with emotions and machines. We address We address

• the impact of emotion on intellectual the impact of emotion on intellectual processes, processes,

• propose a basic theory for recognizing propose a basic theory for recognizing emotions, emotions,

• survey a few existing techniques applied survey a few existing techniques applied in affective computing, and in affective computing, and

• motivate the reason for controlled motivate the reason for controlled integration of these techniques in AI.integration of these techniques in AI.

Page 3: Affective Computing A Seminar Presentation by Karthik Raman, 06005003 Adith Swaminathan, 06005005 Omkar Wagh, 06005006 Samhita Kasula, 06D05014 “There

MotivationMotivation

• AI (and Cognition) is very limited in scope AI (and Cognition) is very limited in scope if we limit it to rational thought.if we limit it to rational thought.

• Can you quantify Fear? Can you tell Can you quantify Fear? Can you tell whether I am afraid?whether I am afraid?

• If I had a computer that could read your If I had a computer that could read your facial expressions, the tone of your voice, facial expressions, the tone of your voice, and “barked” accordingly, will you accept and “barked” accordingly, will you accept it as having a puppy-like “intelligence”?it as having a puppy-like “intelligence”?

• How often have you used Emoticons in How often have you used Emoticons in chat messages? Did you feel hampered chat messages? Did you feel hampered without them?without them?

• If we pursued this to the end, could we If we pursued this to the end, could we have an AI based NAZI propaganda?have an AI based NAZI propaganda?

Page 4: Affective Computing A Seminar Presentation by Karthik Raman, 06005003 Adith Swaminathan, 06005005 Omkar Wagh, 06005006 Samhita Kasula, 06D05014 “There

Understanding Emotion:Hints from Understanding Emotion:Hints from PsychologyPsychology

• Psychology focuses on three broad divisions : Psychology focuses on three broad divisions : Affect, Behaviour and Cognition (ABC)Affect, Behaviour and Cognition (ABC)

• Affect is the ability to Affect is the ability to feelfeel• Some contrasting theories of emotion –Some contrasting theories of emotion –

– James-Lange theory : We act therefore we feel.James-Lange theory : We act therefore we feel.– Neurological Theory : Emotion is a mental Neurological Theory : Emotion is a mental

state due to influence of certain state due to influence of certain neurochemicals (think hormones) on the limbic neurochemicals (think hormones) on the limbic brain brain •The Limbic part of brain is theorised to The Limbic part of brain is theorised to

control emotion, behaviour, long-term control emotion, behaviour, long-term memory and smell.memory and smell.

•Recent findings show that the limbic system Recent findings show that the limbic system is not central to emotion.is not central to emotion.

Page 5: Affective Computing A Seminar Presentation by Karthik Raman, 06005003 Adith Swaminathan, 06005005 Omkar Wagh, 06005006 Samhita Kasula, 06D05014 “There

Theories of EmotionTheories of Emotion– Cognitive Theories : Emotions are a heuristic to Cognitive Theories : Emotions are a heuristic to

process information in the cognitive domain.process information in the cognitive domain.

– Two Factor theory : Appraisal of the situation, and the Two Factor theory : Appraisal of the situation, and the physiological state of the body creates the emotional physiological state of the body creates the emotional response. Emotion, hence, has two factors.response. Emotion, hence, has two factors.

What’s the take-away from all this? No one has a clear theory formulating Emotions!!

Emotion vs Emotion Display : Such widely differing theories for Emotion need not handicap our studies, since all of them are agreed on the various observable properties of Emotions – Emotion Display (or Affect Display).Typical Human Affect Display occurs through –

•Voice•Face•Gestures

Page 6: Affective Computing A Seminar Presentation by Karthik Raman, 06005003 Adith Swaminathan, 06005005 Omkar Wagh, 06005006 Samhita Kasula, 06D05014 “There

Role of Emotion in IntellectRole of Emotion in Intellect

Three major areas of Three major areas of Intelligent activity are Intelligent activity are influenced by influenced by emotions –emotions –

• LearningLearning• Long-term MemoryLong-term Memory• ReasoningReasoning

Popular (exaggerated) Popular (exaggerated) examples of highly examples of highly intelligent, but intelligent, but emotionally emotionally challenged characters challenged characters have been shown have been shown here.here.

Images courtesy Google Images

Page 7: Affective Computing A Seminar Presentation by Karthik Raman, 06005003 Adith Swaminathan, 06005005 Omkar Wagh, 06005006 Samhita Kasula, 06D05014 “There

Modelling LearningModelling Learning

• Learning by ExampleLearning by Example– Nearest analogy in AI is PAC learnabilityNearest analogy in AI is PAC learnability– Parrot repeating English words, Infant learning Parrot repeating English words, Infant learning

languagelanguage• Learning by GuidanceLearning by Guidance

– Nearest analogy in AI would be A* search (the Nearest analogy in AI would be A* search (the heuristic is a guide)heuristic is a guide)

– Our Educational System is based on this methodOur Educational System is based on this method• Learning by FeedbackLearning by Feedback

– Nearest analogy is Neural Network/Expectation Nearest analogy is Neural Network/Expectation Maximisation (where the output is used to tweak Maximisation (where the output is used to tweak parameters of the system)parameters of the system)

– Dog learning new commands, typical carrot-and-stick Dog learning new commands, typical carrot-and-stick scenariosscenarios

Page 8: Affective Computing A Seminar Presentation by Karthik Raman, 06005003 Adith Swaminathan, 06005005 Omkar Wagh, 06005006 Samhita Kasula, 06D05014 “There

Emotion and IntelligenceEmotion and Intelligence

• Somatic Marker HypothesisSomatic Marker Hypothesis– Real-life decision making situations may have many Real-life decision making situations may have many

complex and conflicting alternatives : the cognitive complex and conflicting alternatives : the cognitive processes would be unable to provide an informed processes would be unable to provide an informed optionoption

– Emotion (by way of somatic markers) aid us Emotion (by way of somatic markers) aid us (visualisable as a heuristic)(visualisable as a heuristic)• Reinforcing stimulus induces a physiological state, and this Reinforcing stimulus induces a physiological state, and this

association gets stored (and later bias cognitive processing)association gets stored (and later bias cognitive processing)

• Iowa Gambling ExperimentIowa Gambling Experiment– Designed to demonstrate Emotion-based LearningDesigned to demonstrate Emotion-based Learning– People with damaged Prefrontal Cortex (where the People with damaged Prefrontal Cortex (where the

semantic markers are stored) did poorly.semantic markers are stored) did poorly.

Page 9: Affective Computing A Seminar Presentation by Karthik Raman, 06005003 Adith Swaminathan, 06005005 Omkar Wagh, 06005006 Samhita Kasula, 06D05014 “There

Emotion in ReasoningEmotion in Reasoning

• Minsky’s Ideas : An intelligent system should be Minsky’s Ideas : An intelligent system should be able to describe the same situation in multiple able to describe the same situation in multiple ways (resourcefulness) – such a meta-ways (resourcefulness) – such a meta-description is “Panalogy”description is “Panalogy”

• We now need meta-knowledge to decide which We now need meta-knowledge to decide which description is “fruitful” for our current situation description is “fruitful” for our current situation and reasoningand reasoning– Emotion is the tool in people that switches these Emotion is the tool in people that switches these

descriptions “without thinking”.descriptions “without thinking”.– A machine equipped with such meta-knowledge will A machine equipped with such meta-knowledge will

be more versatile when faced with a new situation.be more versatile when faced with a new situation.

Page 10: Affective Computing A Seminar Presentation by Karthik Raman, 06005003 Adith Swaminathan, 06005005 Omkar Wagh, 06005006 Samhita Kasula, 06D05014 “There

Emotional ComputersEmotional Computers

[xkcd] a webcomicwww.xkcd.com

Page 11: Affective Computing A Seminar Presentation by Karthik Raman, 06005003 Adith Swaminathan, 06005005 Omkar Wagh, 06005006 Samhita Kasula, 06D05014 “There

Use of emotional computersUse of emotional computers

• Musical Tutor for piano lessonsMusical Tutor for piano lessons– Is it maintaining interest?Is it maintaining interest?– Is the student making mistakes?Is the student making mistakes?– Is the lesson tough or the piano key Is the lesson tough or the piano key

stuck?stuck?– Should it just make the user happy?Should it just make the user happy?

• Human teachers use affective cuesHuman teachers use affective cues

• Imagine an emotionless tutor.Imagine an emotionless tutor.

Page 12: Affective Computing A Seminar Presentation by Karthik Raman, 06005003 Adith Swaminathan, 06005005 Omkar Wagh, 06005006 Samhita Kasula, 06D05014 “There

So how do we go about it?So how do we go about it?

• Answer=Affective Theory of Answer=Affective Theory of ComputationComputation

• What are emotions? We don’t really What are emotions? We don’t really know!know!

• AvenuesAvenues– Express EmotionsExpress Emotions– Influence EmotionsInfluence Emotions– Act on EmotionsAct on Emotions– Percieve EmotionsPercieve Emotions

Page 13: Affective Computing A Seminar Presentation by Karthik Raman, 06005003 Adith Swaminathan, 06005005 Omkar Wagh, 06005006 Samhita Kasula, 06D05014 “There

Express EmotionsExpress Emotions

• Display EmotionsDisplay Emotions– Computer voices with natural intonationComputer voices with natural intonation– Computer FacesComputer Faces– ““How” to show I'm happy.How” to show I'm happy.– Example:- AnimationExample:- Animation

• Model EmotionsModel Emotions– React to eventsReact to events– Internal Representation of EmotionInternal Representation of Emotion– Example:-KismetExample:-Kismet

Page 14: Affective Computing A Seminar Presentation by Karthik Raman, 06005003 Adith Swaminathan, 06005005 Omkar Wagh, 06005006 Samhita Kasula, 06D05014 “There

KISMETKISMET

• Recognise stimuliRecognise stimuli

• Intelligently display Intelligently display emotionemotion

• Efficient model for Efficient model for emotions(more on emotions(more on this later)this later)

• Realistic(don't you Realistic(don't you get that puppy dog get that puppy dog feeling?)feeling?)

Page 15: Affective Computing A Seminar Presentation by Karthik Raman, 06005003 Adith Swaminathan, 06005005 Omkar Wagh, 06005006 Samhita Kasula, 06D05014 “There

[A,V,S] Emotion Model[A,V,S] Emotion Model

• [Arousal , Valence , Stance] :- A 3-[Arousal , Valence , Stance] :- A 3-tuple models an “emotion”.tuple models an “emotion”.

• Arousal:- Surprise at high arousal, Arousal:- Surprise at high arousal, fatigue at low arousalfatigue at low arousal

• Valence:- Content at high valence, Valence:- Content at high valence, Unhappiness at low valenceUnhappiness at low valence

• Stance:- Stern at closed stance, Stance:- Stern at closed stance, accepting at open stanceaccepting at open stance

Page 16: Affective Computing A Seminar Presentation by Karthik Raman, 06005003 Adith Swaminathan, 06005005 Omkar Wagh, 06005006 Samhita Kasula, 06D05014 “There

Kismet's Emotive Response TableKismet's Emotive Response Table

Page 17: Affective Computing A Seminar Presentation by Karthik Raman, 06005003 Adith Swaminathan, 06005005 Omkar Wagh, 06005006 Samhita Kasula, 06D05014 “There

Influence EmotionsInfluence Emotions

• Computers(in fact all media) already Computers(in fact all media) already do this!!do this!!

• E.g., a computer game makes one E.g., a computer game makes one happyhappy

• Targeted marketingTargeted marketing– Frequency and types of AdsFrequency and types of Ads– User profilingUser profiling

Page 18: Affective Computing A Seminar Presentation by Karthik Raman, 06005003 Adith Swaminathan, 06005005 Omkar Wagh, 06005006 Samhita Kasula, 06D05014 “There

Emotional ActionsEmotional Actions

• Which action suits which emotion?Which action suits which emotion?

• A decision must be madeA decision must be made– Too many or too little parameters to Too many or too little parameters to

evaluate rationallyevaluate rationally– Intimately related to human psyche(e.g., Intimately related to human psyche(e.g.,

choosing a gift for a loved one)choosing a gift for a loved one)

• Humans abilityHumans ability– Represent the same thing in many waysRepresent the same thing in many ways– Representation depends on current Representation depends on current

emotionemotion

Page 19: Affective Computing A Seminar Presentation by Karthik Raman, 06005003 Adith Swaminathan, 06005005 Omkar Wagh, 06005006 Samhita Kasula, 06D05014 “There

Percieve EmotionsPercieve Emotions

• Observe a human and infer his/her Observe a human and infer his/her emotionemotion

• Approaches:-Approaches:-– Speech Tone RecognitionSpeech Tone Recognition– Facial Expression RecognitionFacial Expression Recognition– Galvanic Skin Resistance(GSR), Electro-Galvanic Skin Resistance(GSR), Electro-

myograms(EMG) etc. myograms(EMG) etc.

• We'll talk about the first two (Speech We'll talk about the first two (Speech and Facial Expression). and Facial Expression).

Page 20: Affective Computing A Seminar Presentation by Karthik Raman, 06005003 Adith Swaminathan, 06005005 Omkar Wagh, 06005006 Samhita Kasula, 06D05014 “There

Facial Expression Facial Expression Recognition: Learning by Recognition: Learning by FeedbackFeedback•Classical Example of Classical Example of

Learning By Feedback.Learning By Feedback.

•Young children look at Young children look at

their parents, and their parents, and

“learn” from their facial “learn” from their facial

expressions what is expressions what is

right and what is notright and what is not Image courtesy Google Images

Page 21: Affective Computing A Seminar Presentation by Karthik Raman, 06005003 Adith Swaminathan, 06005005 Omkar Wagh, 06005006 Samhita Kasula, 06D05014 “There

Expressions & EmotionsExpressions & Emotions• Although human beings can volunarily adopt Although human beings can volunarily adopt

a facial expression, most of our expressions a facial expression, most of our expressions

are involuntary in natureare involuntary in nature

• Especially true for our immediate/reflex Especially true for our immediate/reflex

emotions. In such cases almost impossible emotions. In such cases almost impossible

to curtail our expression.to curtail our expression.

• The close link, between the two sometimes The close link, between the two sometimes

leads to the reverse too, where assuming an leads to the reverse too, where assuming an

expression leads to the emotion.expression leads to the emotion.

Page 22: Affective Computing A Seminar Presentation by Karthik Raman, 06005003 Adith Swaminathan, 06005005 Omkar Wagh, 06005006 Samhita Kasula, 06D05014 “There

Significance of Facial Significance of Facial

ExpressionsExpressions• The expression on a faces, The expression on a faces,

is the most basic form of is the most basic form of

non-verbal communication.non-verbal communication.

• Our impression of other Our impression of other

people, is highly people, is highly

dependant on their dependant on their

expression.expression.

Page 23: Affective Computing A Seminar Presentation by Karthik Raman, 06005003 Adith Swaminathan, 06005005 Omkar Wagh, 06005006 Samhita Kasula, 06D05014 “There

Classes of ExpressionsClasses of Expressions

• Broadly classified into Broadly classified into

happy,sad, disgust, fear, happy,sad, disgust, fear,

anger, surpise and anger, surpise and

neutral.neutral.

• Goal is to classify an Goal is to classify an

unknown expression unknown expression

into one of these classes into one of these classes

Courtesy :Google Images

Page 24: Affective Computing A Seminar Presentation by Karthik Raman, 06005003 Adith Swaminathan, 06005005 Omkar Wagh, 06005006 Samhita Kasula, 06D05014 “There

AI and Facial Expression AI and Facial Expression

RecognitionRecognition

• A base of affective computing is recognition of A base of affective computing is recognition of

human expression.human expression.

• Purpose is to introduce natural ways of Purpose is to introduce natural ways of

communication in person-to-machine communication in person-to-machine

interaction.interaction.

• As in children, a robot, can learn better, when As in children, a robot, can learn better, when

it looks for feedback from a “non-expert” , in it looks for feedback from a “non-expert” , in

the form of facial expressions.the form of facial expressions.

• More natural to us than “pushing buttons”. More natural to us than “pushing buttons”.

Page 25: Affective Computing A Seminar Presentation by Karthik Raman, 06005003 Adith Swaminathan, 06005005 Omkar Wagh, 06005006 Samhita Kasula, 06D05014 “There

General Machine VisionGeneral Machine Vision

• First step in the process is “vision”.First step in the process is “vision”.

• After the image is acquired, some After the image is acquired, some

preprocessing is done such as to reduce preprocessing is done such as to reduce

noise, improve contrast.noise, improve contrast.

• Next features are extracted and areas of Next features are extracted and areas of

interest are “detected”interest are “detected”

• Finally some high-level processing occurs.Finally some high-level processing occurs.

Page 26: Affective Computing A Seminar Presentation by Karthik Raman, 06005003 Adith Swaminathan, 06005005 Omkar Wagh, 06005006 Samhita Kasula, 06D05014 “There

Optical FlowOptical Flow

• Used to capture motion of Used to capture motion of

objects due to relative motion objects due to relative motion

between object and observer.between object and observer.

• Also used to derive “structure” Also used to derive “structure”

of objects.of objects.

• Looks at intensity of “voxels” Looks at intensity of “voxels”

and tries to solve a set of and tries to solve a set of

differential equations.differential equations.

• Voxels = Volume Pixels = Think Voxels = Volume Pixels = Think

Pixels in 3dPixels in 3d

Page 27: Affective Computing A Seminar Presentation by Karthik Raman, 06005003 Adith Swaminathan, 06005005 Omkar Wagh, 06005006 Samhita Kasula, 06D05014 “There

Methods of Facial Methods of Facial

ReocognitionReocognition• Early methods used optical flow to capture Early methods used optical flow to capture

movement of features.(Such as facial muscles)movement of features.(Such as facial muscles)

• Broadly methods are Model-Based, Feature-Broadly methods are Model-Based, Feature-

Based or Holistic Spatial Based.Based or Holistic Spatial Based.

• Model & Feature-Based Methods have a set of Model & Feature-Based Methods have a set of

predefined features which are further used.predefined features which are further used.

• Though this is simple and reduces complexity, Though this is simple and reduces complexity,

there is a loss of information.there is a loss of information.

Page 28: Affective Computing A Seminar Presentation by Karthik Raman, 06005003 Adith Swaminathan, 06005005 Omkar Wagh, 06005006 Samhita Kasula, 06D05014 “There

Holistic Spatial AnalysisHolistic Spatial Analysis• Whole image is taken not just specific features.Whole image is taken not just specific features.

• No pre-defined features. Rather try to discover No pre-defined features. Rather try to discover

intrinsic structural information. These are then intrinsic structural information. These are then

used to recognise the class of expression.used to recognise the class of expression.

• Further divided into unsupervised (examples PCA, Further divided into unsupervised (examples PCA,

ICA) and supervised (example FDA). In supervised ICA) and supervised (example FDA). In supervised

training is done on class-specified samples.training is done on class-specified samples.

• Math behind this is quite complex, based on Math behind this is quite complex, based on

feature subspaces.feature subspaces.

Page 29: Affective Computing A Seminar Presentation by Karthik Raman, 06005003 Adith Swaminathan, 06005005 Omkar Wagh, 06005006 Samhita Kasula, 06D05014 “There

Feature SelectionFeature Selection

• Selecting some features, assists in reducing Selecting some features, assists in reducing

complexity of process.complexity of process.

• Would want to select features that can Would want to select features that can

“identify” the class.“identify” the class.

• Hence the difference in the value of the Hence the difference in the value of the

feature between samples of the class should feature between samples of the class should

be small compared to those across classes.be small compared to those across classes.

• Thus identify clasification ability of feature.Thus identify clasification ability of feature.

Page 30: Affective Computing A Seminar Presentation by Karthik Raman, 06005003 Adith Swaminathan, 06005005 Omkar Wagh, 06005006 Samhita Kasula, 06D05014 “There

Weighted Saliency MapsWeighted Saliency Maps

• Simple example of such a method. Uses pixel Simple example of such a method. Uses pixel

intensities of grayscale images.intensities of grayscale images.

• Calculates ratio of variance between classes Calculates ratio of variance between classes

and within a class.and within a class.

• σk = VarB/VarW , k = 1,..., n.σk = VarB/VarW , k = 1,..., n.

• VarB=Sum of (ClassMean - OverallMean)VarB=Sum of (ClassMean - OverallMean)2,2, for for

all classes and VarW=Sum of (f -all classes and VarW=Sum of (f -

MeanofClassof(f))MeanofClassof(f))2,2, for all f. Here n is number of for all f. Here n is number of

sample points.sample points.

Page 31: Affective Computing A Seminar Presentation by Karthik Raman, 06005003 Adith Swaminathan, 06005005 Omkar Wagh, 06005006 Samhita Kasula, 06D05014 “There

Weighted Saliency Weighted Saliency

Maps(Contd.)Maps(Contd.)

• These ratios are then sorted in descending These ratios are then sorted in descending

order .order .

• Above is an example for the top 500 features Above is an example for the top 500 features

of each class for a particular sampleof each class for a particular sample

Courtesy [6]

Page 32: Affective Computing A Seminar Presentation by Karthik Raman, 06005003 Adith Swaminathan, 06005005 Omkar Wagh, 06005006 Samhita Kasula, 06D05014 “There

Speech Tone RecognitionSpeech Tone Recognition•Why have humanoid robots ?Why have humanoid robots ?

– Enjoyable interactionEnjoyable interaction– Doesn't require training on humans partDoesn't require training on humans part– Easier to teach then bot new tasksEasier to teach then bot new tasks

•Acoustic patterns contain :Acoustic patterns contain :– Who the speaker is?Who the speaker is?– What the speaker saidWhat the speaker said– How it was saidHow it was said

•The third piece of information is a strong The third piece of information is a strong indicator of the underlying intent.indicator of the underlying intent.

Page 33: Affective Computing A Seminar Presentation by Karthik Raman, 06005003 Adith Swaminathan, 06005005 Omkar Wagh, 06005006 Samhita Kasula, 06D05014 “There

Abstraction of the problemAbstraction of the problem•Classify a given sentence to convey one of:Classify a given sentence to convey one of:

– Approval : Good boy!Approval : Good boy!– Prohibition : Don't do that.Prohibition : Don't do that.– Attention bidding : Hey Kismet, look here.Attention bidding : Hey Kismet, look here.– Soothing : It's okay, don't worry.Soothing : It's okay, don't worry.– Neutral : This is a booNeutral : This is a boo

•Fernald's Prosodic Contours Fernald's Prosodic Contours

Courtesy [7]

Page 34: Affective Computing A Seminar Presentation by Karthik Raman, 06005003 Adith Swaminathan, 06005005 Omkar Wagh, 06005006 Samhita Kasula, 06D05014 “There

Robot specifications:Robot specifications:

•Aesthetics : Appearance should affect Aesthetics : Appearance should affect nature of human communication with it.nature of human communication with it.

•Real Time Perfomance : Long delays are Real Time Perfomance : Long delays are not acceptable.not acceptable.

•Voice : Humans should be able to use their Voice : Humans should be able to use their natural voice for training. It should be able natural voice for training. It should be able to recognize a vocalization as having to recognize a vocalization as having affective content when the intent of the affective content when the intent of the sentence is to approve/prohibit, etc.sentence is to approve/prohibit, etc.

Page 35: Affective Computing A Seminar Presentation by Karthik Raman, 06005003 Adith Swaminathan, 06005005 Omkar Wagh, 06005006 Samhita Kasula, 06D05014 “There

Specifications, Contd.Specifications, Contd.•Unacceptable vs Acceptable Unacceptable vs Acceptable

misclassification: Shouldn't judge misclassification: Shouldn't judge prohibition to be approval, but to judge it prohibition to be approval, but to judge it as neutral is an acceptable error.as neutral is an acceptable error.

•Expressive Feedback : Respond to Expressive Feedback : Respond to emotion to let the person know it has emotion to let the person know it has understood.understood.

•Speaker Dependence vs Independence: Speaker Dependence vs Independence: Former for personalized bots, latter for Former for personalized bots, latter for those that need to interact with many those that need to interact with many people.people.

Page 36: Affective Computing A Seminar Presentation by Karthik Raman, 06005003 Adith Swaminathan, 06005005 Omkar Wagh, 06005006 Samhita Kasula, 06D05014 “There

Algorithm : Classify Algorithm : Classify emotional content in speechemotional content in speech

• Processing : tag sample with pitch, energy, Processing : tag sample with pitch, energy, percentage periodicity.percentage periodicity.

• Filter out noise : very high pitches (non-uniform), Filter out noise : very high pitches (non-uniform), very low pitches.very low pitches.

• Calculate features (mean,variance of Calculate features (mean,variance of pitch,energy, pitch range )pitch,energy, pitch range )

• Pass to classifier for result.Pass to classifier for result.

Courtesy [7]

Page 37: Affective Computing A Seminar Presentation by Karthik Raman, 06005003 Adith Swaminathan, 06005005 Omkar Wagh, 06005006 Samhita Kasula, 06D05014 “There

5-way classification in KISMET5-way classification in KISMET

• Stage 1 : Energy parameters are used to Stage 1 : Energy parameters are used to differentiate. (soothing, low-intensity neutral differentiate. (soothing, low-intensity neutral have low mean energy).have low mean energy).

• Stage 2: Stage 2: – Using Fernald's prosodic contours, soothing Using Fernald's prosodic contours, soothing

shows a smooth contour, frequency shows a smooth contour, frequency downsweep. Neutral is coarser and flatter.downsweep. Neutral is coarser and flatter.

Courtesy [7]

Page 38: Affective Computing A Seminar Presentation by Karthik Raman, 06005003 Adith Swaminathan, 06005005 Omkar Wagh, 06005006 Samhita Kasula, 06D05014 “There

Classification : Contd.Classification : Contd.

– Approval &Attention shows high mean Approval &Attention shows high mean pitch, high pitch and energy variance; pitch, high pitch and energy variance; Prohibition has low mean pitch but high Prohibition has low mean pitch but high enery variation. Neutral shows low energy enery variation. Neutral shows low energy and pitch variation.and pitch variation.

• Stage 3 : Approval vs Attention. Both have Stage 3 : Approval vs Attention. Both have high energy, and high pitch variation. But in high energy, and high pitch variation. But in approval, there is an exaggerated rise-fall approval, there is an exaggerated rise-fall pitch contour. Yet, this differentiation is pitch contour. Yet, this differentiation is difficult, and often the content is required to difficult, and often the content is required to disambiguate.disambiguate.

Page 39: Affective Computing A Seminar Presentation by Karthik Raman, 06005003 Adith Swaminathan, 06005005 Omkar Wagh, 06005006 Samhita Kasula, 06D05014 “There

KISMET's response to emotionKISMET's response to emotion• Has a synthetic nervous system (SNS) to help Has a synthetic nervous system (SNS) to help

react to external stimulus.react to external stimulus.• The 'somatic marker' process to tag incoming The 'somatic marker' process to tag incoming

information with affective content.information with affective content.– Arousal : Level of emotional responseArousal : Level of emotional response– Valence : Is the stimulus+ve or -veValence : Is the stimulus+ve or -ve– Stance : How approachable is the percept?Stance : How approachable is the percept?

• This information is passed to the 'emotion This information is passed to the 'emotion elicitor'.elicitor'.

• Emotional Elicitor : Each [A,V,S] input Emotional Elicitor : Each [A,V,S] input contributes to some emotion process. Eg, A contributes to some emotion process. Eg, A large -ve valence might contribute to sad, large -ve valence might contribute to sad, anger, fear, distress emotions.anger, fear, distress emotions.

Page 40: Affective Computing A Seminar Presentation by Karthik Raman, 06005003 Adith Swaminathan, 06005005 Omkar Wagh, 06005006 Samhita Kasula, 06D05014 “There

Response Contd.Response Contd.

• The winning emotion process affects the The winning emotion process affects the response if its value is above some threshold.response if its value is above some threshold.

• Two thresholds, one for behavioural response, Two thresholds, one for behavioural response, the other for response through expression (the the other for response through expression (the latter is lower). This indicates that expression latter is lower). This indicates that expression leads behavioural response. leads behavioural response. – On praise, first comes interest, and then On praise, first comes interest, and then

physical alignment.physical alignment.

| Arousal | Valence | Stance | Expression------------------------------------------------------------------------------------------Approval Med. high High +ve Approach PleasedProhibition Low High -ve Withdraw SadComfort Low Medium +ve Neutral ContentAttention High Neutral Approach InterestNeutral Neutral Neutral Neutral Calm

Page 41: Affective Computing A Seminar Presentation by Karthik Raman, 06005003 Adith Swaminathan, 06005005 Omkar Wagh, 06005006 Samhita Kasula, 06D05014 “There

Do we want ‘Emotional’ Do we want ‘Emotional’ Machines?Machines?

• Nazi Propoganda Machine?Nazi Propoganda Machine?– A computer that knows how to influence emotionsA computer that knows how to influence emotions– The perfect politicianThe perfect politician

• Computers with the ability to killComputers with the ability to kill– Not a distant dream. Civilian aircraft is an Not a distant dream. Civilian aircraft is an

example.example.– Choosing a sub-optimal (emotional) path.Choosing a sub-optimal (emotional) path.– Will an angry/insulted computer behave Will an angry/insulted computer behave

dangerously?dangerously?– Popular Example:- M5 of Star Trek, HAL 9000 of Popular Example:- M5 of Star Trek, HAL 9000 of

“2001-A Space Odyssey”“2001-A Space Odyssey”– The Example:- Marvin of “The Hitch-Hiker’s Guide”The Example:- Marvin of “The Hitch-Hiker’s Guide”

Page 42: Affective Computing A Seminar Presentation by Karthik Raman, 06005003 Adith Swaminathan, 06005005 Omkar Wagh, 06005006 Samhita Kasula, 06D05014 “There

Main DilemnaMain Dilemna

• Computers without emotions – not creative Computers without emotions – not creative or intelligent.or intelligent.

• Computers acting on emotions may Computers acting on emotions may someday wipe out their creators.someday wipe out their creators.

• Possible solution : Give computers ability to Possible solution : Give computers ability to perceive, express and heuristically act on perceive, express and heuristically act on emotions, but ensure that the emotions are emotions, but ensure that the emotions are always visiblealways visible

Page 43: Affective Computing A Seminar Presentation by Karthik Raman, 06005003 Adith Swaminathan, 06005005 Omkar Wagh, 06005006 Samhita Kasula, 06D05014 “There

ConclusionConclusion

• Affective Computing is a young field of researchAffective Computing is a young field of research

• For interactive systems, something far better For interactive systems, something far better than the current crop of “intelligent” systems is than the current crop of “intelligent” systems is needed.needed.

• Affective Computing has applications in Affective Computing has applications in improving the quality of life in impaired people improving the quality of life in impaired people (successfully demonstrated for Autism)(successfully demonstrated for Autism)

• Ethical compromises need to be done to Ethical compromises need to be done to inculcate affective computersinculcate affective computers

• This field can really benefit from research into This field can really benefit from research into the human brain/mind.the human brain/mind.

Page 44: Affective Computing A Seminar Presentation by Karthik Raman, 06005003 Adith Swaminathan, 06005005 Omkar Wagh, 06005006 Samhita Kasula, 06D05014 “There

ReferencesReferences

1.1. R.W. Picard (1995), "Affective Computing“,R.W. Picard (1995), "Affective Computing“,MIT Media LabMIT Media Lab2.2. R.W. Picard (1998) , “Towards Agents that recognize emotions”, R.W. Picard (1998) , “Towards Agents that recognize emotions”,

Actes Proceedings, IMAGINAActes Proceedings, IMAGINA3.3. http://www.ai.mit.edu/projects/humanoid-robotics-group/http://www.ai.mit.edu/projects/humanoid-robotics-group/

kismet/kismet.htmlkismet/kismet.html4.4. Descarte’s Error : Emotion, Reason and the Human Brain, Descarte’s Error : Emotion, Reason and the Human Brain,

Damasio Damasio (1994 Edition)(1994 Edition)5.5. Automatic Facial Expression Recognition using L inear and Non-Automatic Facial Expression Recognition using L inear and Non-

Linear Holistic Spatial Analysis, Ma and Wang (2005) Linear Holistic Spatial Analysis, Ma and Wang (2005) Lecture Lecture Notes in CSNotes in CS

6.6. Emotion and Reinforcement : Affective Facial Expressions Emotion and Reinforcement : Affective Facial Expressions facilitate Robot Learning, Joost Brokens (2007) facilitate Robot Learning, Joost Brokens (2007) Lecture Notes in Lecture Notes in CSCS

7.7. Recognition of Affective Communicative Intent in Robot-Directed Recognition of Affective Communicative Intent in Robot-Directed Speech, Breazal and Aryananda, Speech, Breazal and Aryananda, MIT Media LabMIT Media Lab

8.8. en.wikipedia.org : Emotion, Somatic Marker Hypothesis, Vision, en.wikipedia.org : Emotion, Somatic Marker Hypothesis, Vision, Optic Flow.Optic Flow.