Forgetful and Emotional: Recent Progress in Development of

Dynamic Memory Management System for Conversational Agents

Michal Ptaszynski, Pawel Dybala,Rafal Rzepka, Kenji Araki

Introduction

• Who Are We?KenjiAraki

RafalRzepka

PawelDybala

MichalPtaszynski

…

Hokkaido UniversitySapporo, Japan

Language Media Laboratory,Graduate School of Information Science and Technology

Introduction

• What Do We Do?Genetic

Algorithms

Machine Ethics

Humor Processing

Affective Computing

…

Introduction

• What Do We Do?Genetic

Algorithms

Machine Ethics

Humor Processing

Affective Computing

…

Problem Description

Problem Description

Problem Description

• Expanding Database• of Dialogue Agent• Becomes a Problem:

– Large space– Processing time– Which information is

good…?– …At the certain time?

Problem Description

• How do humans do it?– Agent Database =

Human Memory

• Memory:– Process of Forgetting

(and Recalling)

Forgetting

• Definition: “Forgetting is a process in which parts of knowledge become rearranged, inaccessible or inactive.”[1,2]

• Usual attitude toward forgetting: BAD• New findings:

– “forgetting is not a defect, but helps organize memory and remember about important things.” [8]

[1] J. R. Anderson, The Architecture of Cognition, Harvard University Press (1983).[2] S. Markovitch and P. D. Scott. The Role of Forgetting in Learning, In Proceedings of The Fifth International Conference on Machine Learning. Ann Arbor, MI: Morgan Kaufmann, (1988).[8] I. Kahn, N. M. Dudukovic, B. A. Kuhl and A. D. Wagner. Decreased demands on cognitive control reveal the neural processing benefits of forgetting, Nature Neuroscience, 10, pp. 908-914 (2007).

Forgetting

• Definition: “Forgetting is a process in which parts of knowledge become rearranged, inaccessible or inactive.”[1,2]

• Usual attitude toward forgetting: BAD• New findings:

– “forgetting is not a defect, but helps organize memory and remember about important things.” [8]

[1] J. R. Anderson, The Architecture of Cognition, Harvard University Press (1983).[2] S. Markovitch and P. D. Scott. The Role of Forgetting in Learning, In Proceedings of The Fifth International Conference on Machine Learning. Ann Arbor, MI: Morgan Kaufmann, (1988).[8] I. Kahn, N. M. Dudukovic, B. A. Kuhl and A. D. Wagner. Decreased demands on cognitive control reveal the neural processing benefits of forgetting, Nature Neuroscience, 10, pp. 908-914 (2007).

Forgetting

• Forgetting is dependant on: – Time (chronological fading of memories) [4]– Emotions (Emotionally stronger memories fade

slower) [5,6,7]

• Recalling is indivisible from Forgetting

[4] H. Ebbinghaus. Memory A Contribution to Experimental Psychology (1885). (translated by: T. Utsugi. Kioku nitsuite: jikken shinrigaku he no koken, Tokyo, Seishinsho Shobo (1978).[5] F. G. Zitman. Emotion and memory in mood-anxiety disorders, Schreuder, BJN (2001).[6] N. Luk. The Role of Emotions in Language Teaching, In The Journal of the Imagination in Language and Teaching, 7 (2002).[7] P. Wolfe. The role of meaning and emotion in learning, In New Directions for Adult and Continuing Education, 2006(110), pp. 35-41 (2006).

Forgetting

• Forgetting is dependant on: – Time (chronological fading of memories) [4]– Emotions (Emotionally stronger memories fade

slower) [5,6,7]

• Recalling is indivisible from Forgetting

[4] H. Ebbinghaus. Memory A Contribution to Experimental Psychology (1885). (translated by: T. Utsugi. Kioku nitsuite: jikken shinrigaku he no koken, Tokyo, Seishinsho Shobo (1978).[5] F. G. Zitman. Emotion and memory in mood-anxiety disorders, Schreuder, BJN (2001).[6] N. Luk. The Role of Emotions in Language Teaching, In The Journal of the Imagination in Language and Teaching, 7 (2002).[7] P. Wolfe. The role of meaning and emotion in learning, In New Directions for Adult and Continuing Education, 2006(110), pp. 35-41 (2006).[2] S. Markovitch and P. D. Scott. The Role of Forgetting in Learning, In Proceedings of The Fifth International Conference on Machine Learn-ng. Ann Arbor, MI: Morgan Kaufmann, (1988).[10] M. Ishikawa. A Structural Connectionist Learning Algorithm with Forgetting, J. of Japanese Society for Artificial Intelligence 5(5) (1990).

The idea of chronological forgetting has been applied in

AI and related fields [2,10] (and at present is widely used:

e.g., access history in your web browser).

• Forgetting is dependant on: – Time (chronological fading of memories) [4]– Emotions (Emotionally stronger memories fade

slower) [5,6,7]

• Recalling is indivisible from Forgetting

The idea of chronological forgetting has been applied in

AI and related fields [2,10] (and at present is widely used:

e.g., access history in your web browser).

Forgetting

[4] H. Ebbinghaus. Memory A Contribution to Experimental Psychology (1885). (translated by: T. Utsugi. Kioku nitsuite: jikken shinrigaku he no koken, Tokyo, Seishinsho Shobo (1978).[5] F. G. Zitman. Emotion and memory in mood-anxiety disorders, Schreuder, BJN (2001).[6] N. Luk. The Role of Emotions in Language Teaching, In The Journal of the Imagination in Language and Teaching, 7 (2002).[7] P. Wolfe. The role of meaning and emotion in learning, In New Directions for Adult and Continuing Education, 2006(110), pp. 35-41 (2006).[2] S. Markovitch and P. D. Scott. The Role of Forgetting in Learning, In Proceedings of The Fifth International Conference on Machine Learn-ng. Ann Arbor, MI: Morgan Kaufmann, (1988).[10] M. Ishikawa. A Structural Connectionist Learning Algorithm with Forgetting, J. of Japanese Society for Artificial Intelligence 5(5) (1990).

However the ideas of applying emotional weights in forgetting algorithms and adding the recalling ability

has not been studied sufficiently yet. *) Although this might change in the near future (see RWWA Symposium in the next room)

System Description

• Dialogue Agent is trained on conversation sets

System Description

• Dialogue Agent is trained on conversation sets• Each conversation set is one “context unit” (CU)

(different topics different words used)

A：帰還しますたとりあえず着替えてきます(・∀・)

B：電車ｷﾀ━━━(ﾟ∀ﾟ)━━━!!!!!

A：今戻りました。

結果から報告すると成功でうぇｄｒｆｔｇｙふじこｌ

思い出しながら経過など報告させて頂きます。

それとみんな有難う。

B：性交ｷﾀｷﾀｷﾀｷﾀ━━━(ﾟ∀ﾟ≡(ﾟ∀ﾟ≡ﾟ∀ﾟ)≡ﾟ∀ﾟ)━━━━!!!!!!!!!! A：こんにちは

B：さよなら

A：すまん。俺も裏ぐった。文才が無いか

ら、過程は書けないけど。このスレまじで魔力ありすぎ… おまいらにも光あれ…

B:なんだとこあんちくしょうぁあああああ

A：きになる

B：>>731 彼女が出来たのか？

A：>>734 違うけど。でも大チャンス。こういう

こと続くとネタにしか聞こえないよな。とにかくおまいら外に出てみろ

CU_01 CU_02 CU_03

System Description

• Dialogue Agent is trained on conversation sets• Each conversation set is one “context unit” (CU)• Agent database contains many CUs

System Description

• Dialogue Agent is trained on conversation sets• Each conversation set is one “context unit” (CU)• Agent database contains many CUs• But not all of them have to be processed at all

times Some of them could be deactivated when not needed (forgetting)

System Description

• Dialogue Agent is trained on conversation sets• Each conversation set is one “context unit” (CU)• Agent database contains many CUs• But not all of them have to be processed at all

times Some of them could be deactivated when not needed (forgetting) and reactivated again when needed (recalling)

System Description

Need to answer these questions:1. On what basis agent should forget a CU?2. What information could be extracted from a

CU to fulfill this task?3. How to recall a deactivated CU?

System Description

1. On what basis agent should forget a CU?

Forgetting is a function of Time (T) and Emotional value (E): V = f(T, E)

System Description

2. What information could be extracted from a CU to fulfill this task?Every CU has:

- a certain time stamp

- a certain emotional level

attached when created and renewed when information from a CU is used in other

conversation

need to measure it

System Description

Measuring emotional level of a CU:

- Perform affect analysis [16] of all utterances in the training set to obtain emotive values.

- Approximation of all emotive values = emotional level of the conversation (CU).

- (perform continuously also on new CUs)

[16] M. Ptaszynski, P. Dybala, R. Rzepka and K. Araki. Affecting Corpora: Experiments with Automatic Affect Annotation System - A Case Study of the 2channel Forum -, In Proceedings of The Conference of the Pacific Association for Computational Linguistics 2009 (PACLING-09), pp. 223-228 (2009).

System Description

• Forgetting process: TimeEmotions

• Recalling

“Forget” by zipping parts of database and storing them using less space

System Description

• Recalling DBDBを会話で利用

CU

CU

インターネット

ユーザと会話

CUをDBに追加

Use DB in conversation

User input

Internet

Reactivate similar CU

Extract association lists from Internet [24]

Calculate similarity with deactivated CUs

[24] S. Higuchi, R. Rzepka and K. Araki. A Casual Conversation System Using Modality and Word Associations Retrieved from the Web, In Proceedings of the EMNLP-2008, pp. 382-390 (2008).

Unzip and reactivate the relevant CUs

System Description

Forgetting process: TimeEmotions

Recalling

Problem Description

• Expanding Database• of Dialogue Agent• Becomes a Problem:

– Large space– Processing time– Which information is

good…?– …At the certain time?

Problem Description

• Expanding Database• of Dialogue Agent• Becomes a Problem:

Large spaceProcessing time– Which information is

good…?– …At the certain time?

Problem Description

• Expanding Database• of Dialogue Agent• Becomes a Problem:

Large spaceProcessing time– Which information is

good…?– …At the certain time?

System Description

• Construction of a CU

System Description

• Construction of a CU

① - T. Kudo. MeCab: Yet Another Part-of-Speech and Morphological Analyzer, 2001. http://mecab.sourceforge.net/② - T. Kudo, Y. Matsumoto. Japanese Dependency Analyisis using Cascaded Chunking, CONLL 2002, Taipei (2002).③ - M. Ptaszynski, P. Dybala, R. Rzepka and K. Araki. Affecting Corpora: Experiments with Automatic Affect Annotation System - A Case Study of the 2channel

Forum -, In Proceedings of The Conference of the Pacific Association for Computational Linguistics 2009 (PACLING-09), pp. 223-228 (2009).- Michal Ptaszynski, Jacek Maciejewski, Pawel Dybala, Rafal Rzepka, Kenji Araki, "CAO: Fully Automatic Emoticon Analysis System", In Proceedings of The

Twenty-Fourth AAAI Conference on Artificial Intelligence (AAAI-10), Atlanta, USA, to appear in July, 2010④ - M. Ptaszynski, P. Dybala, W. Shi, R. Rzepka and K. Araki. Towards Context Aware Emotional Intelligence in Machines: Computing Contextual

Appropriateness of Affective States, In Proceedings of Twenty-first International Joint Conference on Artificial Intelligence (IJCAI-09), Pasadena, California, USA, pp. 1469-1474 (2009).

Conclusions

• We presented the description of a dynamic database management system for dialogue agents.

• The system borrows from memory processes in humans: forgetting and recalling, based on:– Chronological fading of memories (CUs)– Emotional values attached to memories (CUs)

• We have developed sub-systems to implement the method

Further Work

• Implementation• Evaluation

Thank you for your attention!