Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2190–2199Copenhagen, Denmark, September 7–11, 2017. c©2017 Association for Computational Linguistics

Towards Implicit Content-Introducing for Generative Short-TextConversation Systems

Lili Yao1, Yaoyuan Zhang1, Yansong Feng1, Dongyan Zhao1,2 and Rui Yan1,2 ∗1Institute of Computer Science and Technology, Peking University, Beijing, China

2Beijing Institute of Big Data Research, Beijing, China{yaolili,zhang yaoyuan,fengyansong,zhaody,ruiyan}@pku.edu.cn

Abstract

The study on human-computer conversa-tion systems is a hot research topic nowa-days. One of the prevailing method-s to build the system is using the gen-erative Sequence-to-Sequence (Seq2Seq)model through neural networks. Howev-er, the standard Seq2Seq model is proneto generate trivial responses. In this pa-per, we aim to generate a more meaning-ful and informative reply when answeringa given question. We propose an implicitcontent-introducing method which incor-porates additional information into the Se-q2Seq model in a flexible way. Specifical-ly, we fuse the general decoding and theauxiliary cue word information throughour proposed hierarchical gated fusion u-nit. Experiments on real-life data demon-strate that our model consistently outper-forms a set of competitive baselines interms of BLEU scores and human evalu-ation.

1 Introduction

To establish a conversation system with adequateartificial intelligence is a long-cherished goal forresearchers and practitioners. In particular, auto-matic conversation systems in open domains areattracting increasing attention due to its wide ap-plications, such as virtual assistants and chatbot-s. In open domains, researchers mainly focus ondata-driven approaches, since the diversity and un-certainty make it impossible to prepare the inter-action logic and domain knowledge. Basically,there are two mainstream ways to build an open-domain conversation system: 1) to search pre-established database for candidate responses by

∗Corresponding author: ruiyan@pku.edu.cn

query retrieval (Isbell et al., 2000; Wang et al.,2013; Yan et al., 2016; Song et al., 2016), and 2) togenerate a new, tailored utterance given the user-issued query (Shang et al., 2015; Vinyals and Le,2015; Serban et al., 2016; Mou et al., 2016; Songet al., 2016). In these studies, generation-basedconversation systems have shown impressive po-tential. Especially, the Sequence-to-Sequence (Se-q2Seq) model (Sutskever et al., 2014) based onneural networks has been extensively used in prac-tice; the idea is to encode a query as a vector and todecode the vector into a reply. Inspired by (Mouet al., 2016), we mainly focus on the generativeshort-text conversation without context informa-tion.

Despite this, the performance of Seq2Seqgeneration-based conversation systems is far fromsatisfactory because its generation process is notcontrollable; it responses to a query according tothe pattern learned from the training corpus. Asa result, the system is likely to generate an un-expected reply even with little semantics, e.g, “Idon’t know” and “Okay” due to the high frequencyof these patterns in training data (Li et al., 2016a;Mou et al., 2016). To address this issue, Li et al.(2016a) proposed to increase diversity in the Se-q2Seq model so that more informative utteranceshave a chance to stand out. Mou et al. (2016)provided a content-introducing approach that gen-erates a reply based on a predicted word. Theword is usually enlightening and drives the gen-erated response to be more meaningful. However,this method is to some extent rigid; it requires thepredicted word to explicitly occur in the generat-ed utterance. As shown in Table 1, sometimes, itis better to generate a semantic related sentencebased on the cue word rather than including it inthe reply directly.

As for such content-introducing method, thereare two aspects that need to be taken into consid-

2190

Query 你不觉得好丑吗(Don’t you think it is ugly?)Cue Word 审美(Aesthetics)Reply 好恶心啊! (It’s disgusting!)Query 先放个大招(Let me use my ultimate power.)Cue Word 技能(Skill)Reply 新技能？(New skill?)

Table 1: The content-introducing conversation ex-amples.

eration. 1) How to add the additional cue wordsduring the generation process? One of the pre-vailing methods is modifying the neural cell withvarious gating mechanisms (Wen et al., 2015a,b;Xu et al., 2016). However, we need careful oper-ation to ensure the neuron works as expected. 2)How to display the cue words in replies? As men-tioned above, the explicit content-introducing ap-proach in (Mou et al., 2016) does not fit well withall situations.

In this paper, we present an implicit content-introducing method for generative conversationsystems, which incorporates cue words using ourproposed hierarchical gated fusion unit (HGFU) ina flexible way. Our main contributions are as fol-lows:

• We propose the cue word GRU, another neu-ral cell, to deal with the auxiliary informa-tion. Compared with other gating methods,our cue word GRU is more flexible.

• We focus on the implicit content-introducingmethod during generation: the informationof the cue word will be fused into the gen-eration process but not necessarily occur ex-plicitly. In this way, we change the “hard”content-introducing method into a new “soft”schema.

The rest of paper is organized as follows. We s-tart by introducing the technical background. InSection 3, we describe our proposed method. InSection 4, we illustrate the experimental setup andevaluations against a variety of baselines. Section5 briefly reviews related work. Finally, we con-clude our paper in Section 6.

2 Technical Background

2.1 Seq2Seq Model and AttentionMechanism

Seq2Seq model was first introduced in statisticalmachine translation; the idea is to encode a source

sentence as a vector by a recurrent neural network(RNN) and to decode the vector to a target sen-tence by another RNN. Now, the conversationalgeneration is treated as a monolingual translationtask (Ritter et al., 2011; Shang et al., 2015). Givena queryQ = (x1, ..., xn), the encoder represents itas a context vector C and then the decoder gener-ates a response R = (y1, ..., ym) word by word bymaximizing the generation probability of R con-ditioned on Q. The objective function of Seq2Seqcan be written as:

p(y1, ..., ym|x1, ..., xn)

=p(y1|C)T∏

t=2

p(yt|C, y1, ..., yt−1)(1)

To be specific, the encoder RNN calculates thecontext vector by:

ht = f(xt, ht−1);C = hT (2)

where ht is the hidden state of encoder RNNat time t and f is a non-linear transformationwhich can be a long-short term memory unit (L-STM) (Hochreiter and Schmidhuber, 1997) or agated recurrent unit (GRU) (Cho et al., 2014). Inthis work, we implement f using GRU.

The decoder RNN generates each reply wordconditioned on the context vector C. The prob-ability distribution pt of candidate words at everytime step t is calculated as:

st = f(yt−1, st−1, C); pt = softmax(st, yt−1)(3)

where st is the hidden state of decoder RNN attime t and yt−1 is the generated word in the replyat time t− 1.

Attention mechanisms (Bahdanau et al., 2014)have been proved effective to improve the gener-ation quality. In Seq2Seq with attention, each yi

corresponds to a context vector Ci; it is weightedaverage of all hidden states of the encoder. For-mally, Ci is defined as Ci =

∑Tj=1 αijhj , where

αij is given by:

αij =exp(eij)∑T

k=1 exp(eik); eij = η(si−1, hj) (4)

where η is usually implemented as a multi-layerperceptron (MLP) with tanh as an activation func-tion.

2191

𝑪𝒕(𝛼𝑡)𝒉𝟏

𝒚𝟏

𝒉𝒎

𝒚𝒎…

…

𝒔𝒏

𝒙𝒏…

…𝒔𝟏

𝒙𝟏

𝑪𝒕(𝛼𝑡 )𝒔𝟏

𝒙𝟏

𝒔𝒏

𝒙𝒏…

…

𝒉𝟏

𝒚𝟏

𝒉𝒎

𝒚𝒎…

…

Online process

𝒊𝒏𝒇𝒐: 𝑪𝒘

Pre-process Trained model

User’s query

Cue word prediction

Implicit content introducing

Figure 1: The architecture of our system. Basedon the constructed corpus, we train our implicitcontent-introducing conversation system. Given auser-issued query, we first predict the cue word.Then, we incorporate the cue word into decodingprocess to generate a meaningful response.

2.2 Pointwise Mutual Information

Pointwise mutual information (PMI) (Church andHanks, 1990) is a measure of association ratiobased on the information theoretic concept of mu-tual information. Given a pair of outcomes x andy belonging to discrete random variables X andY , the PMI quantifies the discrepancy betweenthe probability of their coincidence based on theirjoint distribution and their individual distributions.Mathematically:

PMI(x, y) = logp(x, y)p(x)p(y)

= logp(x|y)p(x)

(5)

This quantity is zero if x and y are independent,positive if they are positively correlated, and neg-ative if they are negatively correlated.

3 Implicit Content-IntroducingConversation System

Figure 1 provides an overview of our system archi-tecture. We crawl conversational data from socialmedia which are publicly available. After filteringand cleaning procedures, we establish the conver-sational parallel dataset, which consists of a largenumber of aligned 〈query − reply〉 pairs. Basedon the entire set, we first predict the cue word forthe given query in Subsection 3.1. Next, we pro-pose the new implicit content-introducing process,which explores when to incorporate the predictedcue word in Subsection 3.2 and how to apply suchinformation in Subsection 3.3.

Input: 𝑦𝑡−1

ℎ𝑡

Output: 𝑦𝑡 Cue

word

=

𝑦0

ℎ1

𝑦1

𝑦1

ℎ2

𝑦2

𝑦2

ℎ3

𝑦3

𝑦𝑡−1

ℎ𝑡

𝑦𝑡

…

Cue word

Figure 2: The information fusion patterns. Thelocal information initialization is presented by theblue arrowhead and the global information incep-tion includes both the blue arrowhead and thegreen arrowhead.

3.1 Cue Word PredictionIn computational linguistics, PMI has been usedfor finding collocations and associations betweenwords. As mentioned in Mou et al. (2016), it isan appropriate statistic for cue words prediction,which is also adopted in this paper to predict a cueword Cw for the given query. Formally, given aquery word wq and a reply word wr, the PMI iscomputed as:

PMI(wq, wr) = logp(wq|wr)p(wq)

(6)

Then, we choose the cue word Cw withhighest PMI score against the query wordswq1, ..., wqn during the prediction, i.e., Cw =argmaxwr

PMI(wq1, ..., wqn, wr), where

PMI(wq1, ..., wqn, wr) ≈ log∏

i p(wqi|wr)∏i p(wqi)

=∑

i

logp(wqi|wr)p(wqi)

=∑

i

PMI(wqi, wr)

(7)The approximation is based on the indepen-

dence assumptions of both the prior distribu-tion p(wqi) and posterior distributions p(wqi|wr).Even the two assumptions may not be true, we usethem in a pragmatic way so that the word-level P-MI is additive for a whole utterance. PMI penal-izes a common word by dividing its prior proba-bility; hence, it prefers a word which is most “mu-tually informative” with the query.

3.2 Information Fusion PatternsTo implant the specific information in conversa-tion system, we consider two types of informationfusion patterns, namely 1) Local information ini-tialization 2) Global information inception.

Local information initialization. In the localpattern, we fuse the cue word Cw as the auxiliary

2192

𝒉𝒕

1 −

+

𝑡𝑎𝑛ℎ

𝜎

𝑪𝒘𝑪𝒕𝒚𝒕−𝟏

𝒉𝒘 𝒉𝒚

𝑡𝑎𝑛ℎ

𝒉t−1𝒉t−1

𝒉𝒚 𝒉𝒘

𝒉𝒚′ 𝒉𝒘

′𝑘

Standard GRU

Cue Word GRU

Fusion Unit

𝑟𝑦 𝑟𝑤

𝑧𝑦 𝑧𝑤

Figure 3: The structure of a HGFU. The bottom oftwo GRUs deal with corresponding input source,i.e., the last generated word yt−1 and the cue wordCw. After that, fusion unit combines the output oftwo GRUs to compute current hidden state ht.

.

information only in the beginning of decoding. Wedescribe this kind of pattern by the blue arrowheadin Figure 2. Recurrent neural networks(RNNs)such as gated recurrent units (GRUs) have the a-bility to keep the information from the beginningto the end to some extent. Therefore, the cue wordadded on the first step of the neural networks canstill influence the generation of the later steps.

Global information inception. However, weobserve that, although the network is capable ofdeciding what to keep in the cell state to affect thelater generation, the influence of the added infor-mation in the beginning of decoding is becomingweaker and weaker over time. Therefore, to pro-vide the model a broader and more flexible spacefor learning, we propose a global information in-ception pattern, which fuses the cue word Cw asthe auxiliary information at every step of decod-ing. This process is presented by both the bluearrowhead and the green arrowheads in Figure 2.

3.3 Hierarchical Gated Fusion Unit

In this subsection, we propose our HierarchicalGated Fusion Unit (HGFU), which incorporatescue words into the generation process and relaxesthe constraint from the “hard” content-introducingmethod into a new “soft” schema. Figure 3 pro-vides an overview of the structure of a HGFU. As

seen, the framework consists of three components:the standard GRU, the cue word GRU, and the fu-sion unit. Among them, standard GRU and cueword GRU take the last generated word yt−1 andcue word Cw respectively as the decoder GRU’sinput; the fusion unit combines the hidden statesof both GRUs to predict the next word yt. In thefollowing, we will illustrate these components indetail.

3.3.1 Standard GRU

We adopt the standard gated recurrent unit (GRU)with the attention mechanism at the decoder part.Let ht−1 be the last hidden state, yt−1 be the em-bedding of the last generated word, and Ct be thecurrent attention-based context. The current hid-den state of the general decoding, hy, is definedas:

ry = σ(Wryt−1 + Urht−1 + UcrCt + br)zy = σ(Wzyt−1 + Uzht−1 + UczCt + bz)

hy = tanh(Whyt−1 + Uh(ry ◦ ht−1) + UchCt + bh)

hy = (1− zy) ◦ ht−1 + zy ◦ hy

(8)where W ’s ∈ Rdim×E and U ’s ∈ Rdim×dim areweight matrices; b’s ∈ Rdim are bias terms; Edenotes the word embedding dimensionality anddim denotes the number of hidden state units.This general decoding process is presented by the“Standard GRU” in Figure 3.

3.3.2 Cue word GRU

To generate more meaningful and informativereplies, we introduce cue words as the additionalinformation during generation. Naturally, the keypoint lies in how to incorporate such information.One of the prevailing methods is modifying theneural cell by various gating mechanisms. How-ever, these approaches are designed specially fora specific scenario, and not effective as expectedwhen they are employed to other tasks. To tacklethis issue, we propose the cue word GRU, anotherindependent neural cell, to deal with the auxiliaryinformation. Since this neural cell can be replacedeasily by other units, it greatly improves the flexi-bility and reusability.

Given the last hidden state ht−1 , the additionalcue word Cw and the current attention-based con-text Ct, the new hidden state of the auxiliary de-

2193

coding hw is computed by following equations:

rw = σ(WrCw + Urht−1 + UcrCt + br)zw = σ(WzCw + Uzht−1 + UczCt + bz)

hw = tanh(WhCw + Uh(rw ◦ ht−1) + UchCt + bh)

hw = (1− zw) ◦ ht−1 + zw ◦ hw

(9)where W ’s and U ’s are weights and b’s are biasterms like those in the standard GRU. Note that thestandard GRU does not share parameter matrixeswith the cue word GRU. The “Cue word GRU” inFigure 3 describes the auxiliary decoding process.

3.3.3 Fusion unit

To combine both the general decoding informationand the auxiliary decoding information, we applythe fusion unit (Arevalo et al., 2017) integratingthe hidden states of both standard GRU, i.e., hy,and the cue word GRU, i.e., hw, to compute thecurrent hidden state ht. The equations are as fol-lows:

h′y = tanh(W1hy)

h′w = tanh(W2hw)

k = σ(Wk[h′y, h

′w])

ht = k ◦ hy + (1− k) ◦ hw

θ = {W1,W2,Wk}

(10)

with θ the parameters to be learned. From the e-quations above we can see that, the gate neuron kcontrols the contribution of the information calcu-lated from hy and hw to the overall output of theunit.

3.4 Model Training

When training on the aligned corpus, we random-ly sample a noun in the reply as the cue word. Theobjective function was the cross entropy error be-tween the generated word distribution pt and theactual word distribution yt in the training corpus.

4 Experiments

In this section, we compare our method with the-state-of-art response generation models based on ahuge conversation resource. The objectives of ourexperiments are to 1) evaluate the effectiveness ofour proposed HGFU model, and 2) explore howcue words affect the process of reply generation.

谢 谢 夸 奖 ！ 么 么 哒 ！

内 心 是 崩 溃 的 吧

递 纸 巾 ！

说 过 吗 ？ 好 像 没 有 说 过 啊 ！

内心

夸奖

纸巾

说过

Co

rrel

ati

on

𝒌

op

enn

ess

Figure 4: Heat map and the k gate openness. Bot-tom: The correlation between the generated replywords and the cue word. Top: The openness of kgate in fusion unit.

4.1 Experimental setup

We evaluated our model on a massive Chi-nese dataset of human conversation crawled fromthe Baidu Tieba1 forum. There are 500,000〈query − reply〉 pairs for training, 2,000 for val-idation, and another unseen 27,871 samples fortesting. In total, we kept about 63,000 distinctwords.

In our experiments, the encoder, the standarddecoder and the cue word decoder have 1,000 hid-den units; the word embedding dimensionality is610 which were initialized randomly and learnedduring training. We applied AdaDelta with a mini-batch size of 80 for optimization. These valueswere mostly chosen empirically. In order to pre-vent overfitting, early stopping was implementedusing a held-out validation set.

4.2 Comparison Methods

In this paper, we conduct extensive experimentsto compare our proposed method against sever-al representative baselines. All the methods ac-tually are implemented in two ways to utilize thecue word, which are local information initializa-tion and global information inception.

rGRU: Through a specially designed Recal-l gate (Xu et al., 2016), domain knowledge wastransformed into the extra global memory of adeep neural network.

SCGRU: In SCGRU (Wen et al., 2015b), an ad-ditional control cell was introduced to gate the dia-

1http://tieba.baidu.com

2194

Query 班主任还拍了我超级丑的照片已被笑死.(上上上镜镜镜)Related Criterion Labels(Cue word) The teacher took a photo of me; it was really ugly

and people laughed at me. (Photogenic)Reply1 谁的照片？Whose photo? Logic Consistency UnsuitableReply2 什么时候拍的？When did he took the photo? Implicit Relevance NeutralReply3 抱抱。Give you a hug. Implicit Relevance NeutralReply4 我拍照也都是巨丑的！My photos are also ugly! —— Suitable

Table 2: An example query, corresponding cue word in bold and its candidate replies with human anno-tation. The query states that people laughed at the author’s photo, it is unsuitable to ask the ownershipof this photo in Reply1. Generally, Reply2 and Reply3 apply to this scenario, but they do not reflec-t semantic relevance with the cue word. Reply4 talks about the respondent’s situation and related to“Photogenic”, thus it is a suitable response.

logue act (DA) features during the generation pro-cess.

SLGD: We implemented the StochasticLanguage Generation in Dialogue (SLGD)method (Wen et al., 2015a), which added ad-ditional features in each gate of the neuralcell.

FGRU: To explore more fusion strategies, intu-itively, we fused the cue word and hidden states byvector concatenation during the decoding process.

Note that rGRU and SCGRU incorporate addi-tional information by gating mechanisms, whileSLGD and FGRU fuse the information into eachgate of the neural cell directly.

4.3 Experiment Evaluation

Objective metrics. To evaluate the performanceof different methods for the conversation gener-ation task, we leverage BLEU (Papineni et al.,2002) as the automatic evaluation metric, whichis originally designed for machine translation andevaluates the output by using n-gram matchingbetween the output and the reference. Here, weuse BLEU-1, BLEU-2 and BLEU-3 in our experi-ments.

Subjective metrics. Since automatic metric-s may not consistently agree with human percep-tion (Stent et al., 2005), human testing is essentialto assess subjective quality. Hence, we randomlysampled 150 queries in the test set, then we invitedfive annotators to offer a judgment. For fairness,all of our human evaluation was conducted in arandom, blind fashion, i.e., replies obtained fromthe five evaluated models are pooled and random-ly permuted for each annotator. Three levels areassigned to a reply with scores from 0 to 2: 0 =

Method BLEU-1 BLEU-2 BLEU-3 Human score

Local

rGRU 1.087 0.419 0.249SCGRU 2.135 0.622 0.255SLGD 1.678 0.508 0.209FGRU 2.262 0.598 0.208HGFU 1.861 0.545 0.209

Global

rGRU 1.793 0.676 0.277 0.542SCGRU 3.637 0.981 0.369 0.73SLGD 4.146 1.059 0.367 0.71FGRU 4.197 1.013 0.282 0.677HGFU 4.893 1.225 0.393 0.942

Table 4: Performance of evaluated methods.

Unsuitable reply, 2 = Suitable reply, and 1 = Neu-tral reply.

To make the annotation task operable, the suit-ability of the generated reply is judged not onlybased on Grammar and Fluency, Logic Consis-tency and Semantic Relevance following (Shanget al., 2015), but also Implicit Relevance, i.e., thegenerated reply should be semantically relevant tothe predicted cue word, no matter the cue wordexplicitly appears in the reply or not. If any of thefirst three criteria is contradicted, the reply shouldbe labeled as “Unsuitable”. Only the replies con-forming to all requirements are labeled as “Suit-able”. Table 2 shows an example of the annotationresults of a query and its replies. The first reply islabeled as “Unsuitable” because of the logic con-sistency. Reply2 and Reply3 are not semanticallyrelated to the cue word, and is therefore annotatedas “Neutral”.

4.4 Overall Performance

The overall results against all baseline methodsare listed in Table 4. Our proposed HGFU mod-el in global schema obviously shows better per-formance than the baseline methods; it obtains the

2195

Chinese Sentence English TranlationQuery 写的真心棒！(夸夸夸奖奖奖) What a nice written! (Appreciation)Reply 谢谢夸奖！么么哒！ Thanks for your appreciation! Love you!Query 还是无法淡定。(内内内心心心) Still cannot calm down. (Heart)Reply 内心是崩溃的吧。 Your heart must be broken.Query 我先去哭一会。(纸纸纸巾巾巾) I am going to cry for a while. (Tissue)Reply 递纸巾！ Offer you a tissue!Query 当初你们不是说过他是诺维斯基吗？(说说说过过过) Didn’t you say that he was Nowitzki†? (Say)Reply 说过吗？好像没有说过啊！？ Did I say it? I don’t seem to say it!?

Table 3: The explicit introducing-content cases of our HGFU model. The predicted cue word in boldexplicitly occurs in the generated reply. Nowitzki† is a NBA basketball player.

highest BLEU scores as well as the highest humanscore.

In terms of automatic evaluations, the global-based methods perform much better than a set oflocal-based methods, which demonstrates the ef-fectiveness of global information inception. Asmentioned above, the global schema provides themodel a broader and more flexible space for learn-ing, which is benefit for information fusion. Whenit comes to human scores (For the sake of con-venience, we only conducted human evaluation inglobal schema), there are similar conclusions toBLEU results.

From Table 4, we can see that the performanceof rGRU is not as good as the other systems, whileSCGRU outperforms the others in the local patternand shows comparative performance in the glob-al schema. These two methods both augment thestandard neural network with specially designedgate to control the cue word, but the results varygreatly. It is the limitation of gating mechanism-s that is lacking in adaptiveness. Besides, SLGDadding cue word term in each gate of the neuralcell has the similar result as FGRU method, whichconcatenates cue word with hidden state. Basical-ly, our proposed HGFU has a significant improve-ment against the baseline systems. The most prob-able credits come from the cue word GRU: we ap-ply the extra GRU unit to control the auxiliary in-formation instead of fusion in the standard GRU,which is more flexible.

Till now, we have elaborated the overall per-formance of all methods. Next we will come toa closer look at some representative cases of ourHGFU model for further analysis and discussions.

4.5 Analysis and Case Studies

Given a query and the cue word, our HGFU modelgenerates a meaningful and informative response.In Table 3, the predicted cue word occurs in thegenerated response and we treat this kind of gen-eration as the explicit introducing-content. How-ever, we do not strictly restrict tothis. As shown inTable 5,our HGFUmodel also generates the replieswithout containing the cue word, while the re-sponsesare still somehow related to the cue wordand the query. This reflects our expectation: theinformation of the cue word will be fused into thegeneration process but not necessarily occur ex-plicitly. It provesthe characteristics of our pro-posed new “soft” schema, whichare more flexible,extensible, and controllable.

We further analyze these explicit cases using aheat map as shown in Figure 4. We use variousshades of blue to present the extent of correla-tion between the cue word and the generated re-ply. The darker the blue is, the higher correlationthey have. For the added information in the reply(Here is exactly the cue word in darkblue), its po-sition and occurrence times are not fixed, whichare autonomously controlled by our model.

Besides, the rectangular pulse is also a signif-icant presentation of this correlation, which indi-cates how the k gate in fusion unit balance the in-fluence of hy and hw. When in the high level ofthe rectangular pulse, k “opens” the switch of hw

to generate the current word; when in the low lev-el, the fusion unit mainly takes hy for generation.We observe that the switch corresponds with theheat map: the generated word is more correlatedwith the cue word when the switch is open.

2196

Chinese Sentence English TranlationQuery 在微博看到这样一个评论真

的是完全无法反驳。(观观观点点点)I saw such a comment in the microblog which can-not be refused completely. (View)

Reply 路人表示不服啊！ As an outsider, I am not convinced!Query 怎么突然就下雨了？(委委委屈屈屈) Why is it raining suddenly? (Grievance)Reply 好伤心啊。 So sad.Query 泰民这张也是做了很久桌

面。(屏屏屏保保保)This photo of Taemin† was also taken as a desktopfor a long while. (Screenshot)

Reply 锁屏吗？ As the lockscreen?Query 混脸熟求勾搭！(小小小新新新) Make acquaintance and seek chances for further re-

lations! (Freshman)Reply 同新人！求认识。 I am also the new! Nice to meet you.

Table 5: The implicit introducing-content cases of our HGFU model. The cue word in bold is notcontained in the reply, while the response is still related to the cue word. Taemin† is a Korean singer.

5 Related work

5.1 Conversation Systems

Automatic human-computer conversation has at-tractedincreasing attention over the past few years.At the very beginning, people start the researchusing hand-crafted rules and templates (Walkeret al., 2001; Misu and Kawahara, 2007; Williamset al., 2013). These approaches require no da-ta or little data for trainingbuthuge manual ef-fort to build the model, which is very time-consuming. For now, buildinga conversation sys-temmainly falls into two categories: retrieval-based and generation-based. As information re-trieval techniques are developing fast, Leuski et al.(2009) build systems to select the most suit-able response from the query-reply pairs usinga statistical language model in cross-lingual in-formation retrieval. Yan et al. (2016) proposea retrieval-based conversation system with thedeep learning-to-respond schema through a deepneural network framework driven by web data.Recently, generation-based conversation system-s have shownimpressive potential. Shang et al.(2015) generate replies for short-text conversationby Seq2Seq-basedneural networks with local andglobal attentions.

5.2 Content Introducing

In vertical domains, Wen et al. (2015b) apply anadditional control cell to gate the dialogue ac-t (DA) features during the generation process toensure the generated repliesexpressthe intendedmeaning. Also, the Stochastic Language Gener-ation in Dialogue method (Wen et al., 2015a) addsadditional features in each gate of the neural cel-

l. Xu et al. (2016) introduce a new trainable gateto recall the global domain memory to enhance theability of modeling the sequence semantics. Dif-ferent from the above work, our paper addressesthe problem of content introducing in the open-domain generative conversation systems.

In open domains, Xing et al. (2016) incorpo-rate topic information into Seq2Seq framework togenerate informative and interesting responses. Toprovide informative clues for content introducing,Li et al. (2016b) detect entities from previous ut-terances and search for more related entities ina large knowledge graph. A very recent studysimilar to ours is Mou et al. (2016), where thepredicted word explicitly occurs in the generatedutterance. Unlike the existing work, we explorean implicit content-introducing method for neuralconversation systems, which utilizes the addition-al cue word in a “soft” manner to generate a moremeaningful response given a user-issued query.

6 Conclusion

In this paper, we explore an implicit content-introducing method for generative short-text con-versation system. Given a user-issued query, ourproposed HGFU incorporates an additional cueword in a “soft” manner to generate a more mean-ingful response. The HGFU model consists ofthree components: the standard GRU, the cueword GRU and the fusion unit. The standard GRUoperates a general decoding process, and the cueword GRU imitates this process but treats the pre-dicted cue word as the current input. As for the fu-sion unit, it combines both the hidden states of thestandard GRU and the cue word GRU to generate

2197

the current output word. The experimental resultsdemonstrate the effectiveness of our approach.

Acknowledgments

This work was supported by the National Hi-TechR&D Program of China No. 2015AA015403;the National Science Foundation of China No.61672058.

ReferencesJohn Arevalo, Thamar Solorio, Manuel Montes-y

Gomez, and Fabio A Gonzalez. 2017. Gated mul-timodal units for information fusion. arXiv preprintarXiv:1702.01992.

Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Ben-gio. 2014. Neural machine translation by joint-ly learning to align and translate. CoRR, ab-s/1409.0473.

Kyunghyun Cho, Bart Van Merrienboer, Dzmitry Bah-danau, and Yoshua Bengio. 2014. On the propertiesof neural machine translation: Encoder-decoder ap-proaches. Eighth Workshop on Syntax, Semanticsand Structure in Statistical Translation.

Kenneth Ward Church and Patrick Hanks. 1990. Wordassociation norms, mutual information, and lexicog-raphy. Computational linguistics, 16(1):22–29.

Sepp Hochreiter and Jurgen Schmidhuber. 1997.Long short-term memory. Neural computation,9(8):1735–1780.

Charles Lee Isbell, Michael Kearns, Dave Kormann,Satinder Singh, and Peter Stone. 2000. Cobot inlambdamoo: A social statistics agent. In Proceed-ings of the Seventeenth National Conference on Ar-tificial Intelligence, pages 36–41. AAAI Press.

Anton Leuski, Ronakkumar Patel, David Traum, andBrandon Kennedy. 2009. Building effective ques-tion answering characters. In Proceedings of the7th SIGdial Workshop on Discourse and Dialogue,pages 18–27. Association for Computational Lin-guistics.

Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao,and Bill Dolan. 2016a. A diversity-promoting ob-jective function for neural conversation models. InProceedings of the 2016 Conference of the NorthAmerican Chapter of the Association for Computa-tional Linguistics: Human Language Technologies,pages 110–119. Association for Computational Lin-guistics.

Xiang Li, Lili Mou, Rui Yan, and Ming Zhang. 2016b.Stalematebreaker: A proactive content-introducingapproach to automatic human-computer conversa-tion. In Proceedings of the Twenty-Fifth Internation-al Joint Conference on Artificial Intelligence, IJCAI

2016, New York, NY, USA, 9-15 July 2016, pages2845–2851.

Teruhisa Misu and Tatsuya Kawahara. 2007. Speech-based interactive information guidance system usingquestion-answering technique. In Acoustics, Speechand Signal Processing, 2007. ICASSP 2007. IEEEInternational Conference on, volume 4, pages IV–145. IEEE.

Lili Mou, Yiping Song, Rui Yan, Ge Li, Lu Zhang,and Zhi Jin. 2016. Sequence to backward and for-ward sequences: A content-introducing approachto generative short-text conversation. In COLING2016, 26th International Conference on Computa-tional Linguistics, Proceedings of the Conference:Technical Papers, December 11-16, 2016, Osaka,Japan, pages 3349–3358.

Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic e-valuation of machine translation. In Proceedings ofthe 40th annual meeting on association for compu-tational linguistics, pages 311–318. Association forComputational Linguistics.

Alan Ritter, Colin Cherry, and William B Dolan. 2011.Data-driven response generation in social media. InProceedings of the conference on empirical methodsin natural language processing, pages 583–593. As-sociation for Computational Linguistics.

Iulian V. Serban, Alessandro Sordoni, Yoshua Bengio,Aaron Courville, and Joelle Pineau. 2016. Buildingend-to-end dialogue systems using generative hier-archical neural network models. In Proceedings ofthe Thirtieth AAAI Conference on Artificial Intelli-gence, pages 3776–3783. AAAI Press.

Lifeng Shang, Zhengdong Lu, and Hang Li. 2015.Neural responding machine for short-text conver-sation. In Proceedings of the 53rd Annual Meet-ing of the Association for Computational Linguisticsand the 7th International Joint Conference on Natu-ral Language Processing of the Asian Federation ofNatural Language Processing, ACL 2015, July 26-31, 2015, Beijing, China, Volume 1: Long Papers,pages 1577–1586.

Yiping Song, Rui Yan, Xiang Li, Dongyan Zhao, andMing Zhang. 2016. Two are better than one: An en-semble of retrieval-and generation-based dialog sys-tems. arXiv preprint arXiv:1610.07149.

Amanda Stent, Matthew Marge, and Mohit Singhai.2005. Evaluating evaluation methods for generationin the presence of variation. In International Con-ference on Intelligent Text Processing and Compu-tational Linguistics, pages 341–351. Springer.

Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014.Sequence to sequence learning with neural network-s. In Advances in neural information processing sys-tems, pages 3104–3112.

2198

Oriol Vinyals and Quoc Le. 2015. A neural conversa-tional model. Deep Learning Workshop of the 32ndInternational Conference on Machine Learning.

Marilyn A Walker, Rebecca Passonneau, and Julie EBoland. 2001. Quantitative and qualitative evalu-ation of darpa communicator spoken dialogue sys-tems. In Proceedings of the 39th Annual Meetingon Association for Computational Linguistics, pages515–522.

Hao Wang, Zhengdong Lu, Hang Li, and EnhongChen. 2013. A dataset for research on short-textconversations. In Proceedings of the 2013 Confer-ence on Empirical Methods in Natural LanguageProcessing, pages 935–945. Association for Com-putational Linguistics.

Tsung-Hsien Wen, Milica Gasic, Dongho Kim, NikolaMrksic, Pei-Hao Su, David Vandyke, and Steve Y-oung. 2015a. Stochastic language generation in di-alogue using recurrent neural networks with convo-lutional sentence reranking. In Proceedings of the16th Annual Meeting of the Special Interest Groupon Discourse and Dialogue, pages 275–284.

Tsung-Hsien Wen, Milica Gasic, Nikola Mrksic, Pei-Hao Su, David Vandyke, and Steve Young. 2015b.Semantically conditioned lstm-based natural lan-guage generation for spoken dialogue systems. InProceedings of the 2015 Conference on EmpiricalMethods in Natural Language Processing, pages1711–1721.

Jason Williams, Antoine Raux, Deepak Ramachan-dran, and Alan Black. 2013. The dialog state track-ing challenge. In Proceedings of the SIGDIAL 2013Conference, pages 404–413.

Chen Xing, Wei Wu, Yu Wu, Jie Liu, Yalou Huang,Ming Zhou, and Wei-Ying Ma. 2016. Top-ic augmented neural response generation with ajoint attention mechanism. arXiv preprint arX-iv:1606.08340.

Zhen Xu, Bingquan Liu, Baoxun Wang, Chengjie Sun,and Xiaolong Wang. 2016. Incorporating loose-structured knowledge into lstm with recall gate forconversation modeling. CoRR, abs/1605.05110.

Rui Yan, Yiping Song, and Hua Wu. 2016. Learningto respond with deep neural networks for retrieval-based human-computer conversation system. InProceedings of the 39th International ACM SIGIRconference on Research and Development in Infor-mation Retrieval, pages 55–64. ACM.

2199