Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processingand the 9th International Joint Conference on Natural Language Processing, pages 4959–4968,Hong Kong, China, November 3–7, 2019. c©2019 Association for Computational Linguistics

4959

Stick to Facts: Towards Fidelity-oriented Product Description Generation

Zhangming Chan1,2,∗,†, Xiuying Chen1,2,∗, Yongliang Wang3, Juntao Li1,2,Zhiqiang Zhang3, Kun Gai3, Dongyan Zhao1,2 and Rui Yan1,2,‡

1 Center for Data Science, AAIS, Peking University2 Wangxuan Institute of Computer Technology, Peking University

3 Alibaba Group, Beijing{zhangming.chan,xy-chen,ljt,zhaody,ruiyan}@pku.edu.cn

{yongliang.wyl,zhang.zhiqiang,jingshi.gk}@alibaba-inc.com

Abstract

Different from other text generation tasks, inproduct description generation, it is of vitalimportance to generate faithful descriptionsthat stick to the product attribute informa-tion. However, little attention has been paidto this problem. To bridge this gap we pro-pose a model named Fidelity-oriented ProductDescription Generator (FPDG). FPDG takesthe entity label of each word into account,since the product attribute information is al-ways conveyed by entity words. Specifically,we first propose a Recurrent Neural Network(RNN) decoder based on the Entity-label-guided Long Short-Term Memory (ELSTM)cell, taking both the embedding and the en-tity label of each word as input. Second, weestablish a keyword memory that stores theentity labels as keys and keywords as values,and FPDG will attend to keywords throughattending to their entity labels. Experimentsconducted a large-scale real-world product de-scription dataset show that our model achievesthe state-of-the-art performance in terms ofboth traditional generation metrics as well ashuman evaluations. Specifically, FPDG in-creases the fidelity of the generated descrip-tions by 25%.

1 Introduction

The effectiveness of automatic text generation hasbeen proved in various natural language process-ing applications, such as neural machine transla-tion (Luong et al., 2015; Zhou et al., 2017), di-alogue generation (Tao et al., 2018a; Hu et al.,2019), and abstractive text summarization (Chenet al.; Gao et al., 2019). One such task, productdescription generation has also attracted consid-erable attention (Lipton et al., 2015; Wang et al.,

∗Equal contribution. Ordering is decided by a coin flip.†Contribution during internship at Alibaba Group.‡Corresponding author.

Input high-waist; straight; jeans; blue; ZARA

Bad

output

This UNIQLO low-waist jeans are suitable for thecurvature of the waistline and will not slip whenworn for a long time. The straight version and blackcolor together makes you even slimmer.

Good

outputThis blue jeans look personalized and chic, makingyou more distinctive. With a high-waist design, thelegs look even more slender, and the whole personlooks taller. The straight design hides the fat in thelegs and is visually slimmer.

Table 1: Example of product description generation.The text with underline demonstrates faithful descrip-tion, and text with deleteline demonstrates wrong de-scription.

2017; Zhang et al., 2019a). An accurate and attrac-tive description not only helps customers make aninformed decision but also improves the likelihoodof purchase. Concretely, the product descriptiongeneration task takes several keywords describingthe attributes of a product as input, and then out-puts fluent and attractive sentences that highlightthe feature of this product.

There is one intrinsic difference between prod-uct description generation and other generationtasks, such as story or poem generation (Li et al.,2018; Yao et al., 2019); the generated descriptionhas to be faithful to the product attributes. An ex-ample case is shown in Table 1, where the goodproduct description matches the input information,while the bad description mistakes the brand andthe style of the jeans. Our preliminary study re-veals that 48% of the outputs from a state-of-the-art sequence-to-sequence system suffer fromthis problem. In the real-world e-commerce prod-uct description generation system, generating textwith unfaithful information is unacceptable. Awrong brand name will damage the interests of ad-vertisers, while a wrong product category mightbreak the law by misleading consumers. Any kindof unfaithful description will bring huge economic

4960

losses to online platforms.Though of great importance, no attention has

been paid to this problem. Existing prod-uct description generators include (Chen et al.,2019; Zhang et al., 2019b), where they focus ongenerating personalized descriptions and pattern-controlled descriptions, respectively.

In this paper, we address the fidelity problemin generation tasks by developing a model namedFidelity-oriented Product Description Generator(FPDG), which takes keywords about the productattributes as inputs. Since product attribute infor-mation is always conveyed by entity words (up to89.72% in input keywords), FPDG generates faith-ful product descriptions by taking into account theentity label of each word. Specifically, first, wepropose an Entity-label-guided Long Short-TermMemory (ELSTM) as the cell in the decoder RNN,which takes the entity label of an input word, aswell as the word itself as input. Then, we estab-lish a keyword memory storing the input keywordswith their entity labels. In each decoding step,the current hidden state of the ELSTM focuses onproper words in this keyword memory in regardto their entity categories. We also collect a large-scale real-world product description dataset fromone of the largest e-commerce platforms in China.Extensive experiments conducted on this datasetshow that FPDG outperforms the state-of-the-artbaselines in terms of traditional generation met-rics and human evaluations. Specifically, FPDGgreatly improves the fidelity of generated descrip-tion by 24.61%.

To our best knowledge, we are the first to ex-plore the fidelity problem of product descriptiongeneration. Besides, to tackle this problem, wepropose an ELSTM and a keyword memory to in-corporate entity label information, so as to gener-ate more accurate descriptions.

2 Related Work

We detail related work on text generation, entity-related generation, and product description.

Text generation. Recently, sequence-to-sequence (Seq2Seq) neural network models havebeen widely used in NLG approaches. Their ef-fectiveness has been demonstrated in a varietyof text generation tasks, such as neural machinetranslation (Luong et al., 2015; Bahdanau et al.,2014; Wu et al., 2016), abstractive text summa-rization (See et al., 2017a; Hsu et al., 2018; Chen

and Bansal, 2018), dialogue generation (Tao et al.,2018a; Xing et al., 2017), etc. Along another line,there are also works based on an attention mech-anism. Vaswani et al. (2017) proposed a Trans-former architecture that utilizes the self-attentionmechanism and has achieved state-of-the-art re-sults in neural machine translation. Since then, theattention mechanism has been used in a variety oftasks (Devlin et al., 2018; Fan et al., 2018; Zhouet al., 2018).

Entity-related generation. Named entityrecognition (NER) is a fundamental componentin language understanding and reasoning (Green-berg et al., 2018; Katiyar and Cardie, 2018). In(Ji et al., 2017), they proved that adding entity re-lated information can improve the performance oflanguage modeling. Building upon this work, in(Clark et al., 2018), they combined entity contextwith previous-sentence context, and demonstratedthe importance of the latter in coherence test. An-other line of related work generates recipes usingneural networks to track and update entity repre-sentations (Bosselut et al., 2018). Different fromthe above works, we utilize entity labels as supple-mentary information to assist decoding in the textgeneration task.

Product descriptions. Quality product descrip-tions are critical for providing a competitive cus-tomer experience in an e-commerce platform. Dueto its importance, automatically generating theproduct description has attracted considerable in-terests. Initial works include (Wang et al., 2017),which incorporates statistical methods with thetemplate to generate product descriptions. Withthe development of neural networks, (Chen et al.,2019) explored a new way to generate personal-ized product descriptions by combining the powerof neural networks and a knowledge base. (Zhanget al., 2019b) proposed a pointer-generator neu-ral network to generate product description whosepatterns are controlled.

In real-world product description generation ap-plication, however, the most important prerequi-site is the fidelity of generated text, and to the bestof our knowledge, no research has been conductedon this so far.

3 Problem Formulation

FPDG takes a list of keywords X = {x1, ..., xTX}

as inputs, where TX is the number of keywords.These keywords are all about the important at-

4961

Input Keywords

Entity-label-SAM

Brand name Brand category ... ColorAdidas Sports shoes ... Black

... ... ... ...

Keyword Memory

|----------------Keyword Encoder--------------| |----------------------------------Entity-based Generator------------------------------|

ELSTM ELSTM ELSTM ELSTM

This black Adidas is

Adidas Black

Brand Color

...

...

Word-SAMword

entity-label <SOS> Regular black Color Adidas BrandThis<SOS>

Figure 1: Overview of FPDG. Green denotes an entity label and purple denotes a word. We divide our modelinto two components: (1) The Keyword Encoder stores the word and its entity label in the token memory, anduses Self-Attention Modules (SAMs) to encode words and entity labels; (2) The Entity-based Generator generatesproduct description based on the token memory and SAM encoders.

tributes of the product. The goal of FPDG is togenerate a product description Y = {y1, ..., yTY

}that is not only grammatically correct but alsoconsistent with the input information, such as thebrand name and the fashion style. Essentially,FPDG tries to optimize the parameters to maxi-mize the probability P (Y |X) =

∏TYt=1 P (yt|X),

where Y = {y1, ..., yTY} is the ground truth an-

swer.

4 Model

In this section, we introduce our Fidelity-orientedProduct Description Generator (FPDG) model indetail. An overview of FPDG is shown in Figure 1and can be split into two modules:

(1) Keyword Encoder (See § 4.1): We first usea key-value memory to store the entity-label askey and the corresponding word as value. To bet-ter learn the interaction between words, we em-ploy two self-attention modules from Transformer(Vaswani et al., 2017) to model the input keywordsand their entity-labels separately.

(2) Entity-based Generator (See § 4.2): We pro-pose a recurrent decoder based on Entity-label-guided LSTM (ELSTM) to generate the productdescription.

4.1 Keyword Encoder

In the product description dataset, most of the in-put keywords are entity words (up to 89.72%), andthe description text should be faithful to these en-tity words. Hence, we incorporate entity label in-formation to improve the accuracy of the gener-ated text. In this section, we introduce how to em-bed input keywords with entity label information.

To begin with, we use an embedding matrix eto map a one-hot representation of each word in

xi into a high-dimensional vector space. Sinceour input keywords have no order information,we use the Self-Attention Module (SAM) fromTransformer (Vaswani et al., 2017) to model thetemporal interactions between the words, insteadof RNN-based encoder. We use three fully-connected layers to project e(xi) into three spaces,i.e., the query qi = Fq(e(xi)), the key ki =Fk(e(xi)) and the value vi = Fv(e(xi)). The at-tention module then takes qi to attend to each k·,and uses these attention distribution results αi,· ∈RTX as weights to obtain the weighted sum of vi,as shown in Equation 2. Next, we add the originalword representation e(xi) on βi as the residentialconnection layer, as shown in Equation 3:

αi,j =exp (qikj)∑TX

n=1 exp (qikn), (1)

βi =∑TX

j=1 αi,jvj , (2)

hi = e(xi) + βi, (3)

where αi,j denotes the attention weight of the i-thword on the j-th word. Finally, we apply a feed-forward layer on hi to obtain the final word repre-sentation hi:

hi = max(0, hi ·W1 + b1) ·W2 + b2, (4)

where W1,W2, b1, b2 are all trainable parameters.We refer to the above process as:

hi = SAM (Fq(xi), Fk(x·), Fv(x·)) . (5)

In the meantime, we leverage the e-commerce-adapted AliNER1 to label each word in the inputsuch as “brand name” and “color”. Non-entitywords are labeled as “normal word”. We denote ci

1https://ai.aliyun.com/nlp/ner

4962

Entity-Label-

LSTM

Figure 2: The structure of ELSTM, which is a hybridof three LSTMs.

as the entity label for the i-th word. To better learnthe interaction between these entity labels, we usea second SAM, taking c· as input, and obtain thefinal representation for ci as mi:

mi = SAM (Fq(ci), Fk(c·), Fv(c·)) . (6)

We also propose a key-value keyword memorythat stores the label results as shown in Figure 1.The keys in the keyword memory are TC repre-sentations of different entity-labels, where TC isthe number of entity categories. The values inthe memory are self-attention representations ofwords that belong to each category. We denote thek-th entity-label in the keyword memory as ck, andthe i-th word belonging to this category as e(xki ).This keyword memory will be incorporated intothe generation process in §4.2.3.

4.2 Entity-based Generator

To incorporate entity label information into gen-eration process, we propose a modified versionof LSTM, named Entity-label-guided Long Short-Term Memory (ELSTM). We first introduce EL-STM and then introduce the RNN decoder basedon ELSTM.

4.2.1 ELSTMAs shown in Figure 3, ELSTM consists of threeLSTMs, two Word-LSTMs and one Entity-label-LSTM. The hidden states of the two Word-LSTMsare integrated together, forming wh

t , while the hid-den state of Entity-Label-LSTM is lht . The wordhidden state wh

t attends to the SAM outputs topredict the next word, while the entity label hid-den state lht is used to attend to the keyword mem-ory and predict entity label of next word. In other

words, ELSTM is a hybrid of three LSTMs witha deeper interaction between these cells. Overall,ELSTM takes two variables as input, the embed-ding of an input word e(yt) and its entity labelmt.

The structure of each LSTM in ELSTM is thesame as original LSTM, thus, is omitted here dueto space limitations. Next, we introduce the inter-action in ELSTM in detail. As shown in Figure 2,Word-LSTM0 takes the e(yt) as input, and outputsthe initial word hidden state wh′

t+1:

wh′t+1 = Word-LSTM0(w

ht , e(yt)). (7)

Next we calculate the entity label hidden state byEntity-Label-LSTM, taking both mt and wh′

t+1 asinput:

lh′

t+1 = Entity-Label-LSTM(lht ,mt + wh′t+1). (8)

To further improve the accuracy of entity label pre-diction, we also apply another multi-layer projec-tion, incorporating entity label context vector cmt(introduced in §16) to obtain polished entity hid-den state lht+1:

lht+1 = FC([lh′

t+1, cmt ]), (9)

where [, ] denotes the concatenation operation.Finally, Word-LSTM1 takes the entity-label

hidden state lht+1 and word embedding e(yt) as in-put, and outputs predicted word hidden state thatcontains entity label information:

wh′1t+1 = Word-LSTM1(w

ht , e(yt) + lht+1). (10)

Note that it is possible that the predicted entity-label hidden state is not accurate enough. Hence,we apply a gate fusion combiningwh′1

t+1 with initialword hidden state wh′

t+1 to ensure the word hiddenstate quality:

γ = σ(FC([wh′1t+1, w

h′t+1])), (11)

wht+1 = γw

h′1t+1 + (1− γ)wh′

t+1. (12)

The fusion result is wht+1, i.e., the updated word

hidden state.In this way, the entity label information and

word information are fully interacted in ELSTMcell, while the hidden states are updated.

4963

Initial State

Brand name Brand category ... Color

Adidas Sports shoes ... Black

... ... ... ...

ELSTM

Entity-Label-SAM

Word-SAM Word-Attention

Label-Attention

ELSTM

Figure 3: An overview of the description generator.

4.2.2 ELSTM-based Attention Mechanism

Established on ELSTMs, we now have a newRNN decoder to generate descriptions, incorporat-ing the two SAM encoders and the keyword mem-ory. First, we apply an Bi-RNN to encode inputkeywords, and use its last hidden state as decoderinitial hidden states, i.e., wh

0 and lh0 . The t-th de-coding step is calculated as:[wht+1

lht+1

]= ELSTM

([wht

lht

], (e(yt),mt)

). (13)

Next, similar to the traditional attention mecha-nism from (Bahdanau et al., 2015), we summarizethe input word representation e(y.) and entity la-bel representation m· into the word context vectorcwt and entity label vector cmt , respectively. As forhow to obtain cwt and cmt , we use the two hiddenstates in ELSTM, i.e., wh

t and lht to attend to h.and m., respectively. Specifically, the entity labelcontext vector cmt is calculated as:

γt,i =Wa tanh(Wbl

ht +Whhi

), (14)

γt,i = exp (γt,i) /∑TX

j=1 exp ( ˆγt,j) , (15)

cmt =∑TX

i=1 γt,imi. (16)

The decoder state lht is used to attend to each entitylabel representation mi, resulting in the attentiondistribution γt ∈ RTX , as shown in Equation 15.Then we use the attention distribution γt to obtaina weighted sum of the entity label representationsm· as the word context vector cmt , shown in Equa-tion 16. cwt is obtained in a similarity by usinghidden state wh

t attending to e(y.), and we omitthe details for brevity. This two context vectorsplay different parts, and is introduced in §4.2.4.

4.2.3 Incorporating Keyword MemorySo far, we have finished calculating the contextvector. Next, we describe how to incorporate theguidance from the keyword memory. We first usethe entity label hidden state to attend to the en-tity label keys in the keyword memory by gumbelsoftmax (Jang et al., 2016) (Equation 17), and thenuse the attention weights to obtain the weightedsum of self-attention representation of values inthe memory (Equation 19):

πk′

t =exp((lhtWec

k + pk)/τ)∑Tcj=1 exp((w

htWec

j + pk)/τ), (17)

vk′

i = SAM(Fq(e(x

ki ), Fk(e(x

k· ), Fv(e(x

k· )),

(18)

o′t+1 =

∑TCk=1

(πkt∑T

ck

i=1(vk′i )), (19)

where pk = − log(− log(gk)), gk ∼ U(0, 1), τ ∈[0, 1] is the softmax temperature. Tck denotes thenumber of words in the input keywords that belongto the k-th entity category. We choose gumbel-softmax instead of regular softmax because a gen-erated word can only belong to one entity cate-gory. In this way, FPDG first predicts the entitylabel of the predicted word and then uses o

′t+1 to

store the information of words that belong to thiscategory.

4.2.4 Projection LayersNext, using a fusion gate gt, o

′t+1 is combined with

word hidden state cwt+1 in a similar way in Equa-tion 12 to obtain ot+1. Finally, we obtain the finalgeneration distribution Pv over vocabulary:

P vt+1 = softmax

(Wv[ot+1, w

ht+1] + bv

). (20)

We concatenate the memory vector ot+1, the wordcontext vector cwt+1, and the output of the decoderELSTM wh

t+1 as the input of the output projectionlayer.

Apart from predicting the next word, we alsouse the entity label hidden state lht to predict theentity label of the next word as an auxiliary task.The distribution P e

t+1 over entity categories is cal-culated as:

P et+1 = softmax(We[l

h′t+1, c

mt+1] + be) (21)

We use negative log-likelihood as loss function:

L = −(∑TY

t=1 logPv(yt) + λ ·∑TY

t=1 logPe(mt)),(22)

4964

where λ is the weight of entity label predictionloss. The gradient descent method is employed toupdate all parameters and minimize this loss func-tion.

5 Experimental Setup

5.1 Research QuestionsWe list four research questions that guide the ex-periments: RQ1 (See § 6.1): What is the over-all performance of FPDG? Are generated descrip-tions faithful? RQ2 (See § 6.2): What is the effectof each module in FPDG? RQ3 (See § 6.3): Howdoes the keyword memory work in each decodingstep? RQ4 (See § 6.4): Can ELSTM successfullycapture entity label information?

5.2 DatasetWe collect a large-scale real-world product de-scription dataset from one of the largest e-commerce platforms in China2. The inputs arekeywords about the product attributes selected bysellers on the platform, and the product descrip-tions are written by professional experts that aregood at marketing. Overall, there are 404,000training samples, and 5,000 validation and testsamples. On average, there are 10 keywords inthe input, and 63 words in the product description.89.72% inputs are entity words.

5.3 Evaluation MetricsFollowing (Fu et al., 2017), we use the evalua-tion package of (Chen et al., 2015), which includesBLEU-1, BLEU-2, BLEU-3, BLEU-4, METEORand ROUGE-L. BLEU is a popular machine trans-lation metric that analyzes the co-occurrences ofn-grams between the candidate and reference sen-tences. METEOR is calculated by generating analignment between the words in the candidate andreference sentences, with an aim of 1:1 correspon-dence. ROUGE-L is a metric designed to evaluatetext summarization algorithms.

(Tao et al., 2018b) notes that only using theBLEU metric to evaluate text quality can be mis-leading. Therefore, we also evaluate our model byhuman evaluation. Three highly educated partic-ipants are asked to score 100 randomly sampledsummaries generated by Pointer-Gen and FPDG.We chose Pointer-Gen since its performance is rel-atively higher than other baselines. The statisticalsignificance of observed differences between the

2https://www.taobao.com/

performance of two runs are tested using a two-tailed paired t-test and is denoted using N(or H) forstrong significance for α = 0.01.

5.4 Comparison Methods

We first conduct an ablation study to prove the ef-fectiveness of each module in FPDG. Model w/oELSTM is implemented with only keys in thememory because there are no entity representa-tions as values. Then, to evaluate the performanceof our proposed dataset and model, we compare itwith the following baselines:(1) Seq2Seq: The sequence-to-sequence frame-work (Sutskever et al., 2014) is one of the initialworks proposed for the language generation task.(2) Pointer-Gen: A sequence-to-sequence frame-work with pointer and coverage mechanism pro-posed in (See et al., 2017b).(3) Conv-Seq2seq: A model combining the con-volutional neural networks and the sequence-to-sequence network (Gehring et al., 2017).(4) FTSum: A faithful summarization model pro-posed in (Cao et al., 2018), which leverages openinformation extraction and dependency parse tech-nologies to extract actual fact descriptions fromthe source text.(5) Transformer: A network architecture thatsolely based on attention mechanisms, dis-pensing with recurrence and convolutions en-tirely (Vaswani et al., 2017).(6) PCPG: A pattern-controlled product descrip-tion generation model proposed in (Zhang et al.,2019b). We adapt the model for our scenario.

To verify whether the performance improve-ment is obtained by adding additional entity labelinputs, we directly concatenate the word embed-ding with the entity embedding as input for thesebaselines, denoted as “with Entity Embedding” inTable 2.

5.5 Implementation Details

We implement our experiments in Pytorch3 onTesla V100 GPUs4. The word and entity-label em-bedding dimensions are set to 256. The number ofhidden units and the entity-label hidden size arealso set to 256. All inputs were padded with ze-ros to a maximum keyword number of the batch.There are 36 categories of entity labels together.

3https://pytorch.org/4Our model will be incorporated into Chuangyi-

Taobao (https://chuangyi.taobao.com/pages/smartPhrase).

4965

Models BLEU Metric METEOR ROUGE-LBLEU-1 BLEU-2 BLEU-3 BLEU-4

Seq2seq 30.10 13.61 7.564 4.824 14.08 25.04with Entity Embedding 30.46 13.77 7.722 5.012 14.09 24.96

Pointer-Gen 30.66 13.84 7.933 5.278 14.10 24.93with Entity Embedding 30.92 13.92 7.988 5.219 14.14 24.85

Conv-Seq2seq 28.22 12.15 6.393 4.012 13.72 23.92with Entity Embedding 28.23 12.22 6.401 4.012 13.75 24.03

FTSum 30.33 13.57 7.563 4.689 13.86 24.63with Entity Embedding 30.47 13.61 7.611 4.937 13.97 24.82

Transformer 29.11 12.77 7.116 4.620 13.95 23.36with Entity Embedding 29.37 12.64 6.852 4.315 13.88 23.73

PCPG(including Entity) 29.46 13.02 7.259 4.620 13.72 24.50

Our FPDG 32.26 14.79 8.472 5.629 15.17 25.27

w/o KW-MEM 31.85 14.36 8.134 5.351 15.01 25.09w/o ELSTM 31.41 14.57 8.430 5.630 14.98 25.25

Table 2: RQ1: Comparison between baselines.

λ is set to 0 in the first 500 steps, and 0.6 in therest of the training process. We performed mini-batch cross-entropy training with a batch size of256 documents for 15 training epochs. we set theminimum encoding step to 15 and maximum de-coding step size to 70. During decoding, we em-ploy beam search with a beam size of 4 to gener-ate a more fluent sentence. It took around 6 hrson GPUs for training. After each epoch, we evalu-ated our model on the validation set and chose thebest performing model for the test set. We use theAdam optimizer (Duchi et al., 2010) as our opti-mizing algorithm and the learning rate is 1e-3.

6 Experimental Results

6.1 Overall Performance

For research question RQ1, we examine the per-formance of our model and baselines in termsof BLEU, as shown in Table 2. Firstly, amongall baselines, Pointer-Gen obtains the best perfor-mance, outperforming the worst baseline Conv-Seq2Seq by 2.44 in BLEU-1. Secondly, directlyconcatenating the entity label embedding with theword embedding does not bring much help, onlyleading to an improvement of 0.26 in BLEU-1 forthe Pointer-Gen model. Finally, our model out-performs all baselines for all metrics, outperform-

Fluency Informativity Fidelity

Pointer-Gen 2.23 1.84 1.91FPDG 2.46N 2.19N 2.38N

Table 3: RQ1: Human evaluation comparison withPointer-Gen baseline.

ing the strongest baseline Pointer-Gen, by 5.22%,6.86%, 6.79%, and 6.65% in terms of BLEU-1,BLEU-2, BLEU-3, and BLEU-4, respectively.

As for human evaluation, we ask three highlyeducated participants to rank generated summariesin terms of fluency, informativity, and fidelity. Therating score ranges from 1 to 3 with 3 being thebest. The results are shown in Table 3, whereFPDG outperforms Pointer-Gen by 10.31% and19.02% in terms of fluency and informativity, and,specifically, FPDG greatly improves the fidelityvalue by 24.61%. We also conduct the paired stu-dent t-test between our model and Pointer-Gen,and the result demonstrates the significance of theabove results. The kappa statistics are 0.35 and0.49, respectively, which indicates fair and moder-ate agreement between annotators5.

5(Landis and Koch, 1977) characterize kappa values < 0as no agreement, 0-0.20 as slight, 0.21-0.40 as fair, 0.41-0.60as moderate, 0.61-0.80 as substantial, and 0.81-1 as almost

4966

Figure 4: RQ3: Visualizations of entity-label-attentionwhen generating the word on the left.

6.2 Ablation Study

Next, we turn to research question RQ2. We con-duct ablation tests on the usage of the keywordmemory and ELSTM, corresponding to FPDGw/o KW-MEM and ELSTM, respectively. TheROUGE score result is shown at the bottom ofTable 2. Performances of all ablation models areworse than that of FPDG in terms of almost allmetrics, which demonstrates the necessity of eachmodule in FPDG. Specifically, ELSTM makesthe greatest contribution to FPDG, improving theBLEU-1, BLEU-2 scores by 2.71% and 1.51%.

6.3 Analysis of Keyword Memory

We then address RQ3 by analyzing the entity-label-level attention on the keyword memory. Tworepresentative cases are shown in Figure 4. Thefigure in the above is the attention map when gen-erating the word “toryburch”, and the bottom fig-ure is when generating the word “printflower”.The darker the color, the higher the attention.Due to limited space, we omit irrelevant entitycategories. When generating “toryburch”, whichis a brand name, the entity-label-level attentionpays most attention to the “Brand” entity label,and when generating “flower”, which is a styleelement, it mostly pays attention to “Element”.This example demonstrates the effectiveness of theentity-label-level attention.

6.4 Analysis of ELSTM

We now turn to RQ4; whether or not the ELSTMcan capture entity label information. We exam-ine this question by verifying whether ELSTM canpredict the entity label of the generated word. Theaccuracy of the predicted entity label is calculatedin a teacher-forcing style, i.e., each ELSTM takes

perfect agreement.

Input:正品; Prada;/;普拉达; 男包; 尼龙; 双肩背包; 男士; 休闲; 旅行包; 电脑包; 登山包(authentic; Prada;man bag; leisure; travel bag; laptop bag; backpack)

Reference

这款来自prada的拉链双肩包。采用纯色设计，包面上的字母印花图案十分显眼，令人眼前一亮，瞬间脱颖而出。开合双拉链处理，使用顺滑，方便拿取物品。大容量的包包，为物品的放置提供足够的空间。(This zippered shoulderbag from Prada. With a solid color design, the letterprint on the surface of the bag is very conspicuous,making it instantly stand out. Open and close dou-ble zipper makes it easy to use take items smoothly.Large-capacity bag provides enough space for theplacement of items.)

Pointer-Gen

这款来自prada的翻盖双肩包。采用了翻盖的设计，具有很好的防盗效果，让你的出行更加安全可靠。大容量的包型设计，满足日常的生活所需，让你的出行更加方便快捷。(This flipoverbag from Prada. The clamshell design has a goodanti-theft effect, making your trip safer and morereliable. The large-capacity package design meetsthe needs of everyday life, making your travel moreconvenient and quick.

FPDG

这款来自prada的纯色双肩包。纯色的设计，给人一种大气又干练的感觉，打造时下流行的简约风格。拉链的开合设计，划拉顺畅，增加物品安全性。大容量的包型，既能容纳生活物品，还让携带更加的便捷。(This solidcolor backpack from Prada. The solid color de-sign gives people an atmosphere and a sense ofexquisiteness, creating a simple style that is popularnowadays. The opening and closing design of thezipper smoothes the stroke and increases the safetyof the item. Large-capacity package can accommo-date living items and make it easier to carry.

Table 4: Examples of the generated answers by Pointer-Gen and FPDG. The text with underline demonstratesfaithful description, and text with deleteline demon-strates wrong description.

the ground truth entity label and word as input, andoutputs the entity label of the next word. We em-ploy recall at position k in n candidates (Rn@k)as evaluation metrics. Over the whole test dataset,R1@36 is 64.12%, R2@36 is 80.86%, and R3@36is 94.02%, which means ELSTM can capture theentity label information to a great extent and guidethe word generation.

We also show a case study in Table 4. The de-scription generated by Pointer-Gen introduces thePrada bag as a “clamshell bag has a good anti-theft effect”, which is contrary to the fact. Whileour model generates the faithful description: “Theopening and closing design of the zipper smoothesthe stroke”.

7 Conclusion and Future Work

In this paper, we explore the fidelity problem inproduct description generation. To tackle this

4967

challenge, based on the consideration that prod-uct attribute information is typically conveyed byentity words, we incorporate the entity label infor-mation of each word to enable the model to havea better understanding of the words and better fo-cus on key information. Specifically, we proposean Entity-label-guided Long Short-Term Memory(ELSTM) and a token memory to store and cap-ture the entity label information of each word.Our model outperforms state-of-the-art methods interms of BLEU and human evaluations by a largemargin. In the near future, we aim to fully preventthe generation of unfaithful descriptions and bringFPDG online.

Acknowledgments

We would like to thank the reviewers for their con-structive comments. This work was supported bythe National Key Research and Development Pro-gram of China (No. 2017YFC0804001), the Na-tional Science Foundation of China (NSFC No.61876196 and NSFC No. 61672058). Rui Yanwas sponsored by Alibaba Innovative Research(AIR) Grant. Rui Yan is the corresponding author.

ReferencesDzmitry Bahdanau, Kyunghyun Cho, and Yoshua Ben-

gio. 2014. Neural machine translation by jointlylearning to align and translate. arXiv preprintarXiv:1409.0473.

Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Ben-gio. 2015. Neural machine translation by jointlylearning to align and translate. In ICLR.

Antoine Bosselut, Omer Levy, Ari Holtzman, CorinEnnis, Dieter Fox, and Yejin Choi. 2018. Simulat-ing action dynamics with neural process networks.ICLR.

Ziqiang Cao, Furu Wei, Wenjie Li, and Sujian Li. 2018.Faithful to the original: Fact aware neural abstrac-tive summarization. In AAAI.

Qibin Chen, Junyang Lin, Yichang Zhang, HongxiaYang, Jingren Zhou, and Jie Tang. 2019. To-wards knowledge-based personalized product de-scription generation in e-commerce. arXiv preprintarXiv:1903.12457.

Xinlei Chen, Hao Fang, Tsung-Yi Lin, Ramakr-ishna Vedantam, Saurabh Gupta, Piotr Dollar, andC Lawrence Zitnick. 2015. Microsoft coco captions:Data collection and evaluation server. arXiv preprintarXiv:1504.00325.

Xiuying Chen, Zhangming Chan, Shen Gao, Meng-Hsuan Yu, Dongyan Zhao, and Rui Yan. Learn-ing towards abstractive timeline summarization. InIJCAI.

Yen-Chun Chen and Mohit Bansal. 2018. Fast abstrac-tive summarization with reinforce-selected sentencerewriting. In ACL, pages 675–686.

Elizabeth Clark, Yangfeng Ji, and Noah A Smith. 2018.Neural text generation in stories using entity repre-sentations as context. In NAACL, pages 2250–2260.

Jacob Devlin, Ming-Wei Chang, Kenton Lee, andKristina Toutanova. 2018. Bert: Pre-training of deepbidirectional transformers for language understand-ing. arXiv preprint arXiv:1810.04805.

John C. Duchi, Elad Hazan, and Yoram Singer. 2010.Adaptive subgradient methods for online learningand stochastic optimization. JMLR, 12:2121–2159.

Angela Fan, Mike Lewis, and Yann Dauphin. 2018. Hi-erarchical neural story generation. In ACL, pages889–898.

Kun Fu, Junqi Jin, Runpeng Cui, Fei Sha, and Chang-shui Zhang. 2017. Aligning where to see and whatto tell: image captioning with region-based attentionand scene-specific contexts. PAMI, 39(12):2321–2334.

Shen Gao, Xiuying Chen, Piji Li, Zhaochun Ren, Li-dong Bing, Dongyan Zhao, and Rui Yan. 2019. Ab-stractive text summarization by incorporating readercomments. In AAAI.

Jonas Gehring, Michael Auli, David Grangier, DenisYarats, and Yann N Dauphin. 2017. Convolutionalsequence to sequence learning. In ICML, pages1243–1252. JMLR. org.

Nathan Greenberg, Trapit Bansal, Patrick Verga, andAndrew McCallum. 2018. Marginal likelihoodtraining of bilstm-crf for biomedical named entityrecognition from disjoint label sets. In EMNLP,pages 2824–2829.

Wan-Ting Hsu, Chieh-Kai Lin, Ming-Ying Lee, KeruiMin, Jing Tang, and Min Sun. 2018. A unifiedmodel for extractive and abstractive summarizationusing inconsistency loss. In ACL, pages 132–141.

Wenpeng Hu, Zhangming Chan, Bing Liu, DongyanZhao, Jinwen Ma, and Rui Yan. 2019. Gsn: Agraph-structured network for multi-party dialogues.arXiv preprint arXiv:1905.13637.

Eric Jang, Shixiang Gu, and Ben Poole. 2016. Cat-egorical reparameterization with gumbel-softmax.ICLR.

Yangfeng Ji, Chenhao Tan, Sebastian Martschat, YejinChoi, and Noah A Smith. 2017. Dynamic entity rep-resentations in neural language models. In EMNLP,pages 1830–1839.

4968

Arzoo Katiyar and Claire Cardie. 2018. Nested namedentity recognition revisited. In ACL, pages 861–871.

J Richard Landis and Gary G Koch. 1977. The mea-surement of observer agreement for categorical data.biometrics, pages 159–174.

Juntao Li, Yan Song, Haisong Zhang, Dongmin Chen,Shuming Shi, Dongyan Zhao, and Rui Yan. 2018.Generating classical chinese poems via conditionalvariational autoencoder and adversarial training. InProceedings of the 2018 Conference on EmpiricalMethods in Natural Language Processing, pages3890–3900.

Zachary C Lipton, Sharad Vikram, and JulianMcAuley. 2015. Capturing meaning in product re-views with character-level generative text models.arXiv preprint arXiv:1511.03683.

Thang Luong, Ilya Sutskever, Quoc Le, Oriol Vinyals,and Wojciech Zaremba. 2015. Addressing the rareword problem in neural machine translation. InACL, volume 1, pages 11–19.

Abigail See, Peter J Liu, and Christopher D Man-ning. 2017a. Get to the point: Summarization withpointer-generator networks. In ACL, pages 1073–1083.

Abigail See, Peter J. Liu, and Christopher D. Man-ning. 2017b. Get to the point: Summarizationwith pointer-generator networks. pages 1073–1083.ACL.

Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014.Sequence to sequence learning with neural net-works. In NIPS.

Chongyang Tao, Shen Gao, Mingyue Shang, Wei Wu,Dongyan Zhao, and Rui Yan. 2018a. Get the pointof my utterance! learning towards effective re-sponses with multi-head attention mechanism. InIJCAI, pages 4418–4424.

Chongyang Tao, Lili Mou, Dongyan Zhao, and RuiYan. 2018b. Ruber: An unsupervised method forautomatic evaluation of open-domain dialog sys-tems. In AAAI.

Ashish Vaswani, Noam Shazeer, Niki Parmar, JakobUszkoreit, Llion Jones, Aidan N Gomez, ŁukaszKaiser, and Illia Polosukhin. 2017. Attention is allyou need. In NIPS, pages 5998–6008.

Jinpeng Wang, Yutai Hou, Jing Liu, Yunbo Cao, andChin-Yew Lin. 2017. A statistical framework forproduct description generation. In IJCAI, pages187–192.

Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc VLe, Mohammad Norouzi, Wolfgang Macherey,Maxim Krikun, Yuan Cao, Qin Gao, KlausMacherey, et al. 2016. Google’s neural ma-chine translation system: Bridging the gap between

human and machine translation. arXiv preprintarXiv:1609.08144.

Chen Xing, Wei Wu, Yu Wu, Jie Liu, Yalou Huang,Ming Zhou, and Wei-Ying Ma. 2017. Topic awareneural response generation. In AAAI.

Lili Yao, Nanyun Peng, Ralph Weischedel, KevinKnight, Dongyan Zhao, and Rui Yan. 2019. Plan-and-write: Towards better automatic storytelling. InProceedings of the AAAI Conference on ArtificialIntelligence, volume 33, pages 7378–7385.

Tao Zhang, Jin Zhang, Chengfu Huo, and Weijun Ren.2019a. Automatic generation of pattern-controlledproduct description in e-commerce. In The WorldWide Web Conference, pages 2355–2365. ACM.

Tao Zhang, Jin Zhang, Chengfu Huo, and Weijun Ren.2019b. Automatic generation of pattern-controlledproduct description in e-commerce. In WWW,pages 2355–2365.

Long Zhou, Wenpeng Hu, Jiajun Zhang, andChengqing Zong. 2017. Neural system combi-nation for machine translation. arXiv preprintarXiv:1704.06393.

Xiangyang Zhou, Lu Li, Daxiang Dong, Yi Liu, YingChen, Wayne Xin Zhao, Dianhai Yu, and Hua Wu.2018. Multi-turn response selection for chatbotswith deep attention matching network. In ACL,pages 1118–1127.