deep learning for dialogue modeling - nthu

Applied Deep LearningYun-Nung (Vivian) Chen

Deep Learning for Dialogue ModelingYUN-NUNG (VIVIAN) CHEN 陳縕儂VIVIANCHEN.IDV.TW

Mar 29th, 2017

Outline

IntroductionFramework

Natural/Spoken Language Understanding

Dialogue Management

Output Generation

ChallengeRecent TrendConclusion

Outline

Dialogue Management

Output Generation

Apple Siri (2011)

Google Now (2012)

Facebook M & Bot (2015)

Intelligent Assistants

Google Home (2016)

Microsoft Cortana (2014)

Amazon Alexa/Echo (2014)

Why We Need?

Daily Life Usage◦ Weather◦ Schedule◦ Transportation◦ Restaurant Seeking

Why We Need?– Get things done

• E.g. set up alarm/reminder, take note

– Easy access to structured data, services and apps• E.g. find docs/photos/restaurants

– Assist your daily schedule and routine• E.g. commute alerts to/from work

– Be more productive in managing your work and personal life

Why Natural Language?• Global Digital Statistics (2015 January)

Global Population

Active Internet Users

Active Social Media Accounts

Active Unique Mobile Users

The more natural and convenient input of devices evolves towards speech.

Intelligent Assistant Architecture

Reactive Assistance

ASR, LU, Dialog, LG, TTS

Proactive Assistance

Inferences, User Modeling, Suggestions

Data Back-end Data

Bases, Services and Client Signals

Device/Service End-points(Phone, PC, Xbox, Web Browser, Messaging Apps)

User Experience“restaurant suggestions”“call taxi”

• Spoken dialogue systems are intelligent agents that are able to help users finish tasks more efficiently via spoken interactions.

• Spoken dialogue systems are being incorporated into various devices (smart-phones, smart TVs, in-car navigating system, etc).

Spoken Dialogue System (SDS)

JARVIS – Iron Man’s Personal Assistant Baymax – Personal Healthcare Companion

Good dialogue systems assist users to access information conveniently and finish tasks efficiently.

APP BOT

Seamless and automatic information transferring across domains reduce duplicate information and interaction

• A bot is responsible for a “single” domain, similar to an app

愛食記地圖LINE

Goal: Schedule a lunch with Vivian

Outline

Dialogue Management

Output Generation

System Framework

Speech Recognition

Language Understanding (LU)• Domain Identification• User Intent Detection• Slot Filling

Dialogue Management (DM)• Dialogue State Tracking• System Action/Policy

Decision

Output Generation

Hypothesisare there any action movies to see this weekend

Semantic Framerequest_moviegenre=action, date=this weekend

System Action/Policyrequest_location

Text responseWhere are you located?

Screen Displaylocation?

Text InputAre there any action movies to see this weekend?

Speech Signal current bottleneck error propagation

Interaction Example

Intelligent Agent Q: How does a dialogue system process this request?

Good Taiwanese eating places include Din Tai Fung, Boiling Point, etc. What do you want to choose? I can help you go there.

find a good eating place for taiwanese food

System Framework

Speech Recognition

Decision

Output Generation

Speech Signal

1. Domain IdentificationRequires Predefined Domain Ontology

find a good eating place for taiwanese foodUser

Organized Domain Knowledge (Database)Intelligent Agent

Restaurant DB Taxi DB Movie DB

Classification!

2. Intent DetectionRequires Predefined Schema

Intelligent Agent

Restaurant DB

FIND_RESTAURANTFIND_PRICEFIND_TYPE

Classification!

3. Slot FillingRequires Predefined Schema

Intelligent Agent

Restaurant DB

Restaurant Rating TypeRest 1 good TaiwaneseRest 2 bad Thai

FIND_RESTAURANTrating=“good”type=“taiwanese”

SELECT restaurant { rest.rating=“good” rest.type=“taiwanese”}Semantic Frame Sequence Labeling

O O B-rating O O O B-type O

System Framework

Speech Recognition

Decision

Output Generation

Speech Signal

State TrackingRequires Hand-Crafted States

Intelligent Agent

location rating type

loc, rating rating, type loc, type

i want it near to my office

State TrackingRequires Hand-Crafted States

Intelligent Agent

i want it near to my office

State TrackingHandling Errors and Confidence

Intelligent Agent

find a good eating place for taixxxx food

FIND_RESTAURANTrating=“good”type=“taiwanese”

FIND_RESTAURANTrating=“good”type=“thai”

FIND_RESTAURANTrating=“good”

Policy for Agent Action• Inform

– “The nearest one is at Taipei 101”

• Request– “Where is your home?”

• Confirm– “Did you want Taiwanese food?”

• Database Search

• Task Completion / Information Display– ticket booked, weather information

Din Tai Fung::

System Framework

Speech Recognition

Decision

Speech Signal

Output Generation

Output / NL Generation• Inform

– “The nearest one is at Taipei 101” v.s.

• Request– “Where is your home?” v.s.

• Confirm– “Did you want Taiwanese food?”

Outline

Dialogue Management

Output Generation

Challenge• Predefined semantic schemaChen et al., “Matrix Factorization with Knowledge Graph Propagation for Unsupervised Spoken Language Understanding,” in ACL-IJCNLP, 2015.

• Data without annotationsChen et al., “Zero-Shot Learning of Intent Embeddings for Expansion by Convolutional Deep Structured Semantic Models,” in ICASSP, 2016.

• Semantic concept interpretationChen et al., “Deriving Local Relational Surface Forms from Dependency-Based Entity Embeddings for Unsupervised Spoken Language Understanding,” in SLT, 2014.

• Predefined dialogue statesChen, et al., “End-to-End Memory Networks with Knowledge Carryover for Multi-Turn Spoken Language Understanding,” in Interspeech, 2016.

• Error propagationHakkani-Tur et al., “Multi-Domain Joint Semantic Frame Parsing using Bi-directional RNN-LSTM,” in Interspeech, 2016.

• Cross-domain intention/bot hierarchySun et al., “An Intelligent Assistant for High-Level Task Understanding,” in IUI, 2016.Sun et al., “AppDialogue: Multi-App Dialogues for Intelligent Assistants,” in LREC, 2016.Chen et al., “Leveraging Behavioral Patterns of Mobile Applications for Personalized Spoken Language Understanding,” in ICMI, 2016.

• Cross-domain information transferringKim et al., “New Transfer Learning Techniques For Disparate Label Sets,” in ACL-IJCNLP, 2015.

FIND_RESTAURANTrating=“good” rating=5? 4?

HotelRest Flight

Travel

Trip Planning

Outline

Dialogue Management

Output Generation

A Single Neuron

Sigmoid function

Activation function

w, b are the parameters of this neuron28

A Single Neuron

5.0"2" 5.0"2"

ynotyis

A single neuron can only handle binary classification29

MN RRf :

A Layer of Neurons

• Handwriting digit classification MN RRf :

A layer of neurons can handle multiple possible output,and the result depends on the max one

……“1” or not

“2” or not

“3” or not

10 neurons/10 classes

Which one is max?

Deep Neural Network (DNN)

• Fully connected feedforward network

……

Layer 1

……

Layer 2…

…Layer L

……

Input Output

vector x

vector y

Deep NN: multiple hidden layers

MN RRf :

RNN for SLU• IOB Sequence Labeling for Slot Filling

• Intent Classification

𝑤0 𝑤1 𝑤2 𝑤𝑛

h0𝑓 h1

𝑓 h2𝑓 h𝑛

h0𝑏 h1

𝑏 h2𝑏 h𝑛

𝑦 0 𝑦 1 𝑦 2 𝑦 𝑛

(a) LSTM (b) LSTM-LA (c) bLSTM-LA

(d) Intent LSTM

intent

h0 h1 h2 h𝑛

𝑦 0 𝑦 1 𝑦 2 𝑦 𝑛

h0 h1 h2 h𝑛

𝑦 0 𝑦 1 𝑦 2 𝑦 𝑛

h0 h1 h2 h𝑛

RNN for SLU• Joint Multi-Domain Intent Prediction and Slot Filling

– Information can mutually enhanced

semantic frame sequence

ht-1 ht+1htW W W W

taiwanese

B-type

please

FIND_RESTV

Slot Tagging Intent Prediction

Hakkani-Tur, et al., “Multi-Domain Joint Semantic Frame Parsing using Bi-directional RNN-LSTM,” in Interspeech, 2016.

just sent email to bob about fishing this weekend

O O O OB-contact_name

OB-subject I-subject I-subject

I send_emailD communication

send_email(contact_name=“bob”, subject=“fishing this weekend”)

are we going to fish this weekend

S2 send_email(message=“are we going to fish this weekend”)

send email to bob

send_email(contact_name=“bob”)

B-messageI-message

I-message I-message I-messageI-message I-message

B-contact_nameS1

Single Turn

Multi-Turn

Domain Identification Intent Prediction Slot Filling

Contextual SLU (Chen et al., 2016)

Knowledge Attention Distributionpi

Memory Representation

Weighted Sum h

∑ Wkg

oKnowledge Encoding

Representationhistory utterances {xi}

current utterance

Inner Product

Sentence EncoderRNNin

x1 x2 xi…

Contextual Sentence Encoder

x1 x2 xi…

RNNmem

slot tagging sequence y

ht-1 ht

wt-1 wt

yt-1 yt

RNN Tagger

Chen, et al., “End-to-End Memory Networks with Knowledge Carryover for Multi-Turn Spoken Language Understanding,” in Interspeech, 2016.

1. Sentence Encoding 2. Knowledge Attention 3. Knowledge Encoding

Contextual SLU (Chen et al., 2016)

Idea: additionally incorporating contextual knowledge during slot tagging track dialogue states in a latent way

E2E Supervised Dialogue System

Wen, et al., “A Network-based End-to-End Trainable Task-Oriented Dialogue System,” arXiv.:1604.04562v2.

0 0 0 … 0 1

Database Operator

Copy field

Database

Seven daysCurry PrinceN

Royal StandardLittle Seuol

DB pointer

Can I have korean

Korean 0.7British 0.2French 0.1

Belief Tracker

Intent Network

Can I have <v.food>

Generation Network <v.name> serves great <v.food> .

Policy Network

MySQL query:“Select * where food=Korean”

InfoBot: E2E Dialogue System with Supervised & Reinforcement Learning

Dhingra, et al., “End-to-End Reinforcement Learning of Dialogue Agents for Information Access,” arXiv.:1609.00777v2.

Movie=?; Actor=Bill Murray; Release Year=1993

Find me the Bill Murray’s movie.

I think it came out in 1993.

When was it released?

Groundhog Day is a Bill Murray movie which came out in 1993.

KB-InfoBotUser

(Groundhog Day, actor, Bill Murray)(Groundhog Day, release year, 1993)(Australia, actor, Nicole Kidman)(Mad Max: Fury Road, release year, 2015)

Knowledge Base (head, relation, tail)

Idea: differentiable database for propagating the gradients

TC-Bot: E2End Task-Completion Bot

B-type

wi+1 wi+2

B-type

wi+1 wi+2

System Action / Policyrequest_location

User Dialogue ActionInform(location=San Francisco)

Time t-1

<slot>

wi+1 wi+2

Language Understanding (LU)

Time t-2

Time t

Dialogue Management

w0 w1 w2

Natural Language Generation (NLG)EOSUser

User Agenda Modeling

User SimulatorEnd-to-End Neural Dialogue System

Idea: supervised learning for each component and reinforcement learning for end-to-end training the neural dialogue system

Li, et al., “End-to-End Task-Completion Neural Dialogue Systems,” arXiv.:1703.01008.

RL TC-BotUser goal: Two tickets for “the witch” tomorrow 9:30 PM at regal meridian 16, Seattle.

RULE BASED SYSTEMUser: Can I get 2 tickets for the witch?Agent: Which theater do you want?User: Regal meridian 16Agent: What time would you like to see it?User: 9:30 pmAgent: What date would you like to see it?User: Tomorrow!Agent: How many people are you?User: Just two adults.Agent: What is your location?User: Seattle!Agent: What kind of video format do you like?User: Oh, any kind is okay. Agent: Do you have any price constraint?User: No, I don’t care.Agent: Great - I was able to purchase 2 tickets for you to see the witch tomorrow at regal meridian 16 theater in Seattle at 9:30 pm.User: Thank you.Agent: Thank you!

REINFORCEMENT LEARNING SYSTEMUser: Can I get 2 tickets for the witch?Agent: What date would you like to see it?User: TomorrowAgent: Which theater do you prefer?User: Regal meridian 16Agent: What is your location?User: SeattleAgent: What time would you like to see it?User: 9:30 pm.Agent: Great - I was able to purchase 2 tickets for you to see the witch tomorrow at regal meridian 16 theater in Seattle at 9:30 pm.User: Thanks.Agent: Thanks!

The system can learn how to efficiently interact with users for task completion

Outline

Dialogue Management

Output Generation

Conclusion• The conversational systems can manage information access

via spoken interactions• A domain is usually constrained by the backend service

– Semantic schema should be predefined– Cross-domain knowledge and intention is difficult to handled

• NN-Based Dialogue System– Pipeline outputs are represented as vectors distributional

• Semantic frames as vectors to encode confidence

• Implicitly represent dialogue states in hidden vectors

– The execution is constrained by backend services symbolic

Q & AT H A N K S F O R Y O U R AT T E N T I O N !

deep learning for dialogue modeling - nthu

Technology

nthu hss culture study introduction

nthu hope laboratory

prosodic correspondence in tone sandhi* - nthu

deep reinforcement learning for dialogue generation

sensors and actuators - nthu pme micro device labatory

deep learning and animated learners research project ·...

generative deep neural networks for...

pengaruh strategi...

nthu research methodology 3:17:17

introducing the nthu-eztable kaggle contest (predicting...

nthu pme edl, chih-wei lin biomimetic micro sensors:...

hiroshi ishikawa - nthu

(pengaruh strategi pembelajaran deep dialogue dan

deep reinforcement learning for dialogue . · pdf...

social computing, spring 2014 @ nthu version jan 13,...

deep learning for dialogue systems - gtc on demand · 2018....

image comm. lab ee/nthu digital image/video processing

introduction and installation cs340100, nthu yoshi

chih-wei lin (d937708) nthu pme engineering design lab

inheritance and the modifiers cs340100, nthu yoshi