semantic-based emotional inference and agent interaction applied in education
DESCRIPTION
Semantic-Based Emotional Inference and Agent Interaction Applied in Education. AUTHORS: I-HEN TSAI 1 , RUI-TING SUN 1 , REN-YING FANG 1 , KOONG H.-C. LIN 1 , MIN-CHAI SHIEH 1 , JIUN-SHENG LI 2 , CHU-CHUAN HUANG 2 , JHING-FA WANG 3 1 NATIONAL UNIVERSITY OF TAINAN - PowerPoint PPT PresentationTRANSCRIPT
AUTHORS: I-HEN TSAI1, RUI-TING SUN1 , REN-YING FANG1, KOONG H.-C. LIN1, MIN-CHAI SHIEH1, JIUN-SHENG LI2, CHU-CHUAN HUANG2, JHING-FA WANG3
1NATIONAL UNIVERSITY OF TAINAN 2HCI TECHNOLOGY CENTER, ITRI 3NATIONAL CHENG KUNG UNIVERSITYPRESENTER:I-HEN TSAI
Semantic-Based Emotional Inference and Agent Interaction Applied in
Education
Overview
Abstract: In a nutshellIntroductionSystem DesignPutting together the piecesExperimentationResultsConclusionFuture Works
Abstract
A system for student interaction in education environment
Text inputInference emotion from textText to agent for visual interaction
Introduction
System to interact with students Improve concentration Bring student attention back to class
Based on analyzing text Emotion What they think of the class
Semantic Analysis
Semantic information extraction Ontology approach Connection between concepts
OMCSnet Text string input Output term relation values with target concepts Common sense inference
Dogs <bark> not <meow> Mapping rules: attributes + operations Predicates database: 250000 items
Simplified Algorithm: Sentence Parsing
Data structure requirement: 2 dequeues, digestedSymbol & digestedToken2 stacks, symbolDequeue & tokenDequeue
For each token from argv[1] to argv[n]On [* : push * into symbolDequeue, push an empty string into tokenDequeue.
If currentToken is [NP, skip to the corresponding NP]On *] : push symbolDequeue.top() into “digestedSymbol,” push tokenDequeue.top() into “digestedToken.”On * : Append currentString in tokenDequeue.top()
Algorithm: Emotional Inference
Define I[] = set of tokens translated into English.
Define E[]= { concentrate, happy, relax, easy },
emotions[sizeof(I)][4];
pathSum = 0;
for each I[i]
for each E[j]
Let D[i][j] = distance( I[i], E[j] )
pathSum += D[i][j]
end of for
for each E[j]
emotions[i][j] = (pathSum - D[i][j])/pathSum
end of for
pathSum=0
end of for
Process Flow of Semantic Analysis
Agent
Visual avatar 偽春菜 or “ukagaka” ( 伺か ) C-based system Varied interaction capabilities Can be user defined to suit need Can script wanted dialogue Interchangeable skin
System Flow
Text inputTranslationParse text to pick out significant termsMatch sig. terms with target concepts for
term relation valueDetermine concentration valuePass result to agentAgent picks dialogue based on received value
System Process Flow
Experiment Setup
Educational domain Educational institutional background and direction The speech occurrences of student chatter in class
TranslationBalanced sets
S6我吃飽了I am full
S7真想睡覺
Really want to go to bed
S8要聊天的話,請出去
To chat, please go
S9這堂課真是無聊透頂
This class is really boring the extreme
S10好無聊喔
Oh well bored
S1這次考試真容易
The examination was really easy
S2我下課後一定要讀書
I have to study after school
S3這堂課真是有趣
This class is really interesting
S4上課請專心
Please concentrate on school
S5知識就是力量
Knowledge is power
S11 我討厭上數學I hate math
S12同學們考試的時候不要東張西
望the students do not look around
during the examination
S13舉頭望老師,低頭吃午餐
raise my eyes to the teacher, bow their heads to eat lunch
S14學生上課應該要專心聽講class, the students should
concentrate on listening and speaking
S15 學習是無止盡的learning is endless
S16 學物理對我沒有用physics right I did not use
S17老師的課讓人想睡覺
the teacher's class people want to sleep
S18上課做筆記會協助記憶
class and taking notes will assist the memory
S19教室有冷氣好睡覺
Classrooms are air-conditioned sleeping well
S20同學不要趴在桌上睡覺
Students do not sleep lying on the table
Results
Main concept: concentrationIndicators: happy, easy, relaxMain concept triggers interactionIndicators allow the viewer to have some
insight of what students think of the class
Table 2. Inference Results (Concentration)
Sample Sentence
LabelInference Statistics
Concluding State
S1
concentrate:0.000000happy:0.428571relax:0.000000easy:0.571429
Not concentrating
S2
concentrate:0.733333happy:1.369697relax:1.436364easy:1.460606
Concentrating
S3
concentrate:0.733333happy:0.800000relax:0.733333easy:0.733333
Concentrating
S4
concentrate:1.000000happy:0.636364relax:0.636364easy:0.727273
Concentrating
S5
concentrate:1.451128happy:1.575188relax:1.451128easy:1.522556.
Concentrating
S6
concentrate:0.000000happy:0.642857relax:0.642857easy:0.714286
Not Concentrating
S7
concentrate:0.000000happy:0.666667relax:0.666667easy:0.666667
Not concentrating
S8
concentrate:0.000000happy:0.000000relax:0.000000easy:0.000000
Not concentrating
S9
concentrate:0.733333happy:0.800000relax:0.733333easy:0.733333
Concentrating
S10
concentrate:0.000000happy:0.000000relax:0.000000easy:0.000000
Not concentrating
S11
concentrate:0.714286happy:0.761905relax:0.761905 easy:0.761905
Concentrating
S12
concentrate:0.000000happy:0.428571relax:0.000000easy:1.571429
Not Concentrating
S13
concentrate:0.000000happy:2.036364relax:1.836364easy:2.127273
Not Concentrating
S14
concentrate:1.0000happy:0.000000relax:0.000000easy:0.000000
Concentrating
S15
concentrate:0.000000happy:0.000000relax:0.000000easy:0.000000
Not Concentrating
S16
concentrate:0.00000happy:0.000000relax:0.000000easy:0.000000
Not Concentrating
S17
concentrate:2.125641happy:3.005594relax:2.929837easy:2.938928
Concentrating
S18
concentrate:0.733333happy:0.80000relax:0.733333easy:0.733333
Concentrating
S19
concentrate:0. 714286
happy:0. 785714
relax:0. 0.785714
easy:0. 714286
Concentrating
S20
concentrate:0.00000
happy: 1.303030
relax: 1.393939
easy: 1.303030
Not concentrating
Table 3. Inference Results (Mood)
Sample Sentence Label
Inference Ration Comparison
Inferred Mood State
S1 1.0:0.0:1.3 Easy
S2 1.0:1.05:1.07 Easy
S3 1.1:1.0:1.0 Happy
S4 1.0:1.0:1.1 Easy
S5 1.1:1.0:1.1 Happy
S6 1.0:1.0:1.1 Easy
S7 1.0:1.0:1.0 Happy
S8 0.0:0.0:0.0 Null
S9 1.1:1.0:1.0 Happy
S10 0.0:0.0:0.0 Null
S11 1.0:1.0:1.0 AllS12 1.0:0.0:3.7 EasyS13 1.1:1.0:1.2 EasyS14 0.0:0.0:0.0 NullS15 0.0:0.0:0.0 NullS16 0.0:0.0:0.0 NullS17 1.02:1.0:1.003 HappyS18 1.1:1.0:1.0 HappyS19 1.1:1.1:1.0 Happy, RelaxS20 1.0:1.1:1.0 Relax
Conclusions
Main issues: Translation issues OMCSnet
Concept search must be perfect match, eg. <concentrate> does not equal <concentrated>
Lack of desired link between concept Shortest path first
Agent Needs feedback mechanism
Future works
Better translation alternative(?)OMCSnet
Weighting system(?), in attempt Alternate ontology mapping structure Add more items
Agent Feedback mechanism
Interaction affects future inference Other multimedia outputs
Thank You for your attention!