Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 3960–3969Brussels, Belgium, October 31 - November 4, 2018. c©2018 Association for Computational Linguistics

3960

Stylistic Chinese Poetry Generation via Unsupervised StyleDisentanglement

Cheng Yang1,2,3, Maosong Sun1,2,4∗, Xiaoyuan Yi1,2,3, Wenhao Li1,2,31Department of Computer Science and Technology, Tsinghua University

2Institute for Artificial Intelligence, Tsinghua University3State Key Lab on Intelligent Technology and Systems, Tsinghua University

4Jiangsu Collaborative Innovation Center for Language Ability, Jiangsu Normal University{cheng-ya14,yi-xy16,liwh16}@mails.tsinghua.edu.cn

sms@tsinghua.edu.cn

Abstract

The ability to write diverse poems in differ-ent styles under the same poetic imagery is animportant characteristic of human poetry writ-ing. Most previous works on automatic Chi-nese poetry generation focused on improvingthe coherency among lines. Some work ex-plored style transfer but suffered from expen-sive expert labeling of poem styles. In this pa-per, we target on stylistic poetry generation ina fully unsupervised manner for the first time.We propose a novel model which requires nosupervised style labeling by incorporating mu-tual information, a concept in information the-ory, into modeling. Experimental results showthat our model is able to generate stylistic po-ems without losing fluency and coherency.

1 Introduction

Classical Chinese poetries are great heritages ofthe history of Chinese culture. One of the mostpopular genres of classical Chinese poems, i.e.quatrains, contains four lines with five or sevencharacters each and additional rhythm and tune re-strictions. During 1,000 years history of quatrains,various styles, e.g. pastoral, descriptive and ro-mantic, have been developed to express differentfeelings of poets. In human poetry writing, poetsare able to write completely different poems in di-verse styles even given the same keyword or firstsentence. For example, as shown in Fig. 1, whena poet mentioned “ 月” (the moon), she/he maywrite about the Great Wall in the battlefield styleor the sleepless feeling in the romantic style. Suchability to write stylistic poems under the same po-etic imagery is an important characteristic of hu-man poetries.

Automatic poetry generation is one of the firstattempts towards computer writing. Chinese qua-train generation has also attracted much atten-

∗corresponding author: sms@tsinghua.edu.cn

Figure 1: An example of poems in diverse styles underthe same keyword.

tion in recent years. Early works inspired bystatistical machine translation explored rule-basedand template-based methods (He et al., 2012;Yan et al., 2013), while recent works (Zhangand Lapata, 2014; Wang et al., 2016; Yan, 2016;Zhang et al., 2017; Yang et al., 2017; Yi et al.,2017) employed neural network based sequence-to-sequence approaches which have shown theireffectiveness in neural machine translation forpoem generation. Most works target on improvingthe coherency among all lines and the conformitybetween the theme and subsequent lines by plan-ning (Wang et al., 2016), polishing schema (Yan,2016), poem block (Yi et al., 2017) and condi-tional variational autoencoder (Yang et al., 2017).Different from these previous works, we aim tolearn the ability of diverse stylistic poetry genera-tion which can generate multiple outputs (poems)in various styles under the same input (keywordsor the first sentence). This ability enables a poetrygeneration system to be closer to a real poet andallows the model to generate more expressive andcreative poems.

However, there is no explicit label about whatstyle or category a poem or a sentence is forthousands of poems in the database. Therefore,traditional supervised sequence-to-sequence mod-

3961

els (Zhang et al., 2017) are not capable to gener-ate stylistic poems without expert labeling. To thebest of our knowledge, we are the first effort atstylistic poetry generation in a fully unsupervisedmanner.

In this paper, we propose a novel poetry gen-eration model which can disentangle the poemsin different styles and generate style-specific out-puts conditioned on the manually selected styleinput. We employ sequence-to-sequence modelwith attention mechanism (Bahdanau et al., 2014)as our basis and maximize the mutual informationwhich measures the dependency between two ran-dom variables in information theory to strengthenthe relationship between manually selected styleinputs and generated style-specific outputs. Exper-imental results show that our model is able to gen-erate poems in various styles without losing flu-ency and coherency.

To summarize, our effort provides the followingthree contributions:• To the best of our knowledge, we are the first

effort at stylistic poetry generation which is an im-portant characteristic of human poetries in a fullyunsupervised manner.• We innovatively incorporate mutual infor-

mation, a measurement in information theory,for unsupervised style disentanglement and style-specific generation.• Experimental results show that our model is

able to generate diverse poems in various styleswithout losing fluency and coherency.

2 Related Works and Motivations

Poetry generation is a classic task in computerwriting (Gervas, 2001; Levy, 2001; Netzer et al.,2009; Oliveira, 2012). Chinese poetry generationhas also attracted much attention during the lastdecade. Early works are mostly rule-based andtemplate-based (Wu et al., 2009). (He et al., 2012)employed statistical machine translation and (Yanet al., 2013) adopted automatic summarizationtechniques for classical poem generation.

As neural network based approaches have beensuccessfully applied in various applications suchas machine translation (Bahdanau et al., 2014),people came up with the idea to apply sequence-to-sequence models for poem generation. (Zhangand Lapata, 2014) employed the Recurrent NeuralNetwork (RNN) as their basis and further consid-ered the global context using Convolutional Neu-

ral Network (CNN). They also incorporated otherhand-crafted features into the model to improvethe coherency. (Yan, 2016) proposed an itera-tive polishing schema based on two RNNs, whichcan refine the generated poems for several times.(Wang et al., 2016) generated poetries in a two-stage process: planning the keywords of each linefirst and then generating each line sequentially.(Yi et al., 2017) proposed poem blocks to learnsemantic meaning within a single line and seman-tic relevance among lines in a poem. (Yang et al.,2017) employed a conditional variational autoen-coder with augmented word2vec architecture toenhance the conformity between the theme andgenerated poems. All previous works above tar-get on improving the coherency and conformityof poetry generation while a recent work (Zhanget al., 2017) was proposed to improve the nov-elty of generated poems using external memories.Their model can also transfer the style of a poeminto three predefined topics. However, they needto manually label many poems in these three pre-defined topics to learn the patterns.

Formally, a traditional sequence-to-sequencemodel actually learns a conditional probabilitydistribution Pr(soutput|sinput) where soutput is thegenerated poems and sinput is the input keywordor the first sentence. Different from machinetranslation tasks where the output sentence soutputis rather certain and compact given input sen-tence sinput, the conditional probability distribu-tion Pr(soutput|sinput) has strong uncertainty inliterary creation scenarios such as poem genera-tion. To support this point, we further train a topicmodel based on LDA (Blei et al., 2003) by treat-ing each poem as a document. We train a 10-topic and a 20-topic LDA model and then reana-lyze the largest topic component of each sentencein a poem. Surprisingly, we find that only 20%and 10% consecutive sentences in a poem havethe same largest topic components respectivelythough we assume that each poem is generatedby the same topic when training the LDA model.The fact indicates that even given the former sen-tence, the style of the latter one could still be di-verse and flexible. Intuitively, poets will choosea style or topic in their minds and then write thenext sentence based on the style they choose inpoetry creation. Therefore, we propose to learn astylistic poetry generation model which can disen-tangle the poems in different styles and generate

3962

style-specific outputs as real human poets coulddo. Note that no explicit style labels are given tothe training data and thus previous supervised al-gorithms cannot be adapted for the purpose easily.

Recently, a few works have been proposedfor disentangled representation learning. Info-GAN (Chen et al., 2016) was proposed for gen-erating continuous image data conditioned on im-age labels and disentangled text generation (Huet al., 2017) focused on controllable generation inthe semi-supervised setting. Though inspired bythem, the motivation and proposed models of ourwork differ from these methods by a large mar-gin. To the best of our knowledge, we are the firsteffort at stylistic poetry generation in a fully unsu-pervised manner.

3 Method

We hope our generation model can generate mul-tiple outputs in various styles. Formally, ourmodel takes two arguments as input: input sen-tence sinput and style id k ∈ 1, 2 . . .K where Kis the total number of different styles. Then wecan enumerate each style id k and generate style-specific output sentence skoutput respectively.

In this section, we will start by introducing thebackground knowledge about mutual informationand sequence-to-sequence model with attentionmechanism. Then we propose our framework ofdecoder model for style disentanglement by tak-ing the mutual information as an additional regu-larization term. Finally, we will present the detailsof our implementations of each component in theframework.

3.1 Mutual Information

Inspired by previous works on image genera-tion (Chen et al., 2016) and semi-supervised gen-eration (Hu et al., 2017), we propose to incorpo-rate the concept of mutual information in informa-tion theory for style disentanglement. Given tworandom variables X and Y , the mutual informa-tion I(X,Y ) measures “the amount of informa-tion” obtained about one random variable givenanother one 1. Mutual information can also beinterpreted as a measurement about how similarthe joint probability distribution p(X,Y ) is to theproduct of marginal distributions p(X)p(Y ). The

1https://en.wikipedia.org/wiki/Mutual_information

definition of mutual information is

I(X,Y ) =

∫Y

∫X

p(X,Y ) logp(X,Y )

p(X)p(Y )dXdY. (1)

3.2 Sequence-to-Sequence Model withAttention Mechanism

We employ a widely used Encoder-Decoderframework (Sutskever et al., 2014) which wasfirstly proposed in machine translation as our ba-sis. Suppose sentence X = (x1x2 . . . xT ) andY = (y1y2 . . . yT ) are the input sentence (sourcesentence) and output sentence (target sentence) re-spectively where xi, yi for i = 1, 2 . . . T are char-acters and T is the total number of characters inthe sentence. We denote the character vocabularyas V .

Specifically, we use bidirectional LSTM (bi-LSTM) (Hochreiter and Schmidhuber, 1997;Schuster and Paliwal, 1997) model as the encoderto project the input sentence X into the vectorspace. Formally, the hidden state of LSTM arecomputed by

−→hi = LSTMforward(

−−→hi−1, e(xi)), (2)

←−hi = LSTMbackward(

←−−hi−1, e(xT−i+1)), (3)

for i = 1, 2 . . . T where−→hi and

←−hi are the i-th

hidden state of forward and backward LSTM re-spectively, e(xi) ∈ Rd is the character embeddingof character xi and d is the dimension of charac-ter embeddings. Then we concatenate correspond-ing hidden states of forward and backward LSTMhi = [

−→hi ,←−−−−hT−i+1] as the i-th hidden state of bi-

LSTM. In specific, we use the last hidden state hTas the embedding vector and feed it to the decoder.

The decoder module contains an LSTM decoderwith attention mechanism (Bahdanau et al., 2014)which computes a context vector as a weightedsum of all encoder hidden states to represent themost relevant information at each stage. The char-acter probability distribution when decoding the i-th character can be expressed as

p(yi|y1y2 . . . yi−1, X) = g(yi|si), (4)

where g(·) is a linear projection function with soft-max regularization, si is the i-th hidden state in thedecoder LSTM:

si = LSTMdecoder(si−1, [e(yi−1), ai]), (5)

for i = 2, . . . T and s1 = hT , where [ ] indi-cates concatenation operation, e(yi−1) is charac-ter embedding of yi−1 and ai is the context vector

3963

!

!!

x1 x2 xT

hTh2h1

One-hot Style

Representation

Concatenations1 s2 sT

Expected Character

Embedding!

Estimated Posterior

Style Distribution

LSTM Decoder

Mutual Information Maximization

Bi-LSTM Encoder

Input Sequence

Figure 2: An overview of style disentanglement by mutual information maximization.

learned by attention mechanism (Bahdanau et al.,2014)

ai = attention(si−1, h1:T ). (6)

3.3 Decoder Model with StyleDisentanglement

To accept two arguments, i.e., input sentence Xand style id k, we can directly concatenate theone-hot representation of style id one hot(k) andthe embedding vector hT obtained by bi-LSTMencoder and then feed the concatenated vector[one hot(k), hT ] instead of hT into the decodermodel without changing anything else.

However, there is no theoretical guaranteethat the output sentence generated by the de-coder is strongly correlated to the style id inputone hot(k). In other words, when the input styleid is changed, the output sentence would probablybe the same because no supervised loss is givento force the output sentences to follow the one-hot style representation. Therefore, we naturallycome up with the idea to add a regularization termto force a strong dependency relationship betweenthe input style id and generated sentence.

Without loss of generality, we assume that theinput style id is a uniformly distributed randomvariable Sty and Pr(Sty = k) = 1

K for k =1, 2 . . .K where K is the total number of styles.Recall that mutual information quantifies the mu-tual dependency between two random variables.Hence we propose to maximize the mutual in-formation between the style distribution Pr(Sty)and the generated sentence distribution Pr(Y ;X)given input sentence X to strengthen the depen-dency between them as shown in Fig. 2. The mu-

tual information is computed as

I(Pr(Sty),Pr(Y ;X))

=

K∑k=1

Pr(Sty = k)

∫Y |k;X

logPr(Y, Sty = k;X)

Pr(Sty = k) Pr(Y ;X)dY

=

K∑k=1

Pr(Sty = k)

∫Y |k;X

logPr(Y, Sty = k;X)

Pr(Y ;X)dY

−K∑

k=1

Pr(Sty = k) log Pr(Sty = k)

=

K∑k=1

Pr(Sty = k)

∫Y |k;X

logPr(Sty = k|Y )dY + logK

=

∫Y ;X

K∑k=1

Pr(Sty = k|Y )logP (Sty = k|Y )dY + logK.

(7)

Note that the input sequence X and style Styare both input arguments and thus independentwith each other. Therefore, posterior distributionPr(Sty = k|Y ;X) = Pr(Sty = k|Y ).

But the posterior probability distributionPr(Sty = k|Y ) is actually unknown, we cannotcompute the integration directly. Fortunately,with the help of variational inference maximiza-tion (Barber and Agakov, 2003), we can train aparameterized function Q(Sty = k|Y ) whichestimates the posterior distribution and maximizea lower bound of mutual information in Eq. 7instead:

I(Pr(Sty),Pr(Y ;X))− logK

=

∫Y ;X

K∑k=1

Pr(Sty = k|Y ) log Pr(Sty = k|Y )dY

=

∫Y ;X

K∑k=1

Pr(Sty = k|Y ) logQ(Sty = k|Y )dY

+

∫Y ;X

K∑k=1

Pr(Sty = k|Y ) logPr(Sty = k|Y )

Q(Sty = k|Y )dY

3964

=

∫Y ;X

K∑k=1

Pr(Sty = k|Y ) logQ(Sty = k|Y )dY

+

∫Y ;X

KL(Pr(Sty|Y ), Q(Sty|Y ))dY

≥∫Y ;X

K∑k=1

Pr(Sty = k|Y ) logQ(Sty = k|Y )dY

=

K∑k=1

Pr(Sty = k)

∫Y |k;X

logQ(Sty = k|Y )dY.

(8)

Here KL(Pr(·)||Q(·)) indicates the KL-divergence distance between probability dis-tribution Pr(·) and Q(·). The inequality comesfrom the fact that the KL-divergence is always noless than zero and is tight when Pr(·) = Q(·).

We use the lower bound parameterized by func-tion Q in Eq. 8 as an additional maximizationterm. Intuitively, the lower bound is maximized ifwe can perfectly infer the style id from generatedstyle-specific outputs by inference function Q. Italso indicates that generated outputs will heavilydepend on the input style id and outputs generatedby different styles are distinguishable, which fol-lows our motivation of stylistic poetry generation.Now we will introduce how to design the poste-rior distribution estimation function Q and com-pute the integration in the lower bound.

3.4 Posterior Distribution EstimationGiven an output sequence Y , the function Q esti-mates the probability distribution of the style ofsequence Y and therefore disentangles differentstyles. In this paper, we employ neural networkto parametrize the posterior distribution estima-tion function Q. Specifically, we first computethe average character embeddings of sequence Yand then use a linear projection with softmax nor-malizer to get the style distribution. Formally,Q(Sty|Y ) is computed as

Q(Sty|Y ) = softmax(W · 1T

T∑i=1

e(yi)), (9)

where W ∈ RK×d is the linear projection matrix.Then the last thing to do is to compute the in-

tegration over Y |k;X . However, the search spaceof sequence Y is exponential to the size of vocab-ulary. Hence it’s impossible to enumerate all pos-sible sequence Y for computing the integration.Also, it is not differentiable if we sample sequenceY according to generation probability for approx-imation. Therefore, we propose to use expected

character embedding to approximate the integra-tion.

3.5 Expected Character EmbeddingInspired by previous works (Kocisky et al., 2016),we use expected character embedding to approxi-mate the probabilistic space of output sequences:we only generate an expected embedding se-quence and suppose Y |k;X has one hundred per-cent probability generating this one. Formally,Eq. 4 gives a probability distribution of generat-ing the i-th character given previous ones. Thenthe “expected” generated character embedding is

expect(i; k,X) =∑c∈V

g(c|si)e(c), (10)

where expect(i; k,X) ∈ Rd represents the ex-pected character embedding at i-th output givenstyle id k and input sequence X and c ∈ V enu-merates all characters in the vocabulary.

Then expect(i; k,X) is fed into the LSTM indecoder to update the hidden state for generatingnext expected character embedding:

si+1 = LSTMdecoder(si, [expect(i; k,X), ai+1]). (11)

Finally, we use the expected embeddingsexpect(i; k,X) for i = 1, 2 . . . T as an approxi-mation of the whole probability space of Y |k;X .The lower bound in Eq. 8 can be rewritten as

Lreg =1

K

K∑k=1

log{softmax(W · 1T

T∑i=1

expect(i; k,X))[k]},

(12)

where x[j] represents the j-th dimension of vectorx.

We add the lower bound Lreg to the overalltraining objective as a regularization term. Thecomputing process is shown in Fig. 2. Then foreach training pair (X,Y ), we aim to maximize

Train(X,Y ) =

T∑i=1

log p(yi|y1y2 . . . yi−1, X) + λLreg,

(13)

where p(yi|y1y2 . . . yi−1, X) is style irrelevantgeneration likelihood and computed by settingone-hot style representation to an all-zero vector,and λ is a harmonic hyperparameter to balancethe log-likelihood of generating sequence Y andthe lower bound of mutual information. The firstterm ensures that the decoder can generate fluentand coherent outputs and the second term guaran-tees the style-specific output has a strong depen-dency on the one-hot style representation input.

3965

Note that unlike machine translation task wherethe trained model is desired to generate exactly thesame as the target, poetry generation encouragesnovelty as an important requirement. In Eq. 13, wecan also enumerate the style id representation as aone-hot vector instead of an all-zero one. How-ever, this will force the generation of all styles tobe close to the training target and discourage thediversity and novelty.

Moreover, our model is not task-specific: theregularization term Lreg can be added to othermodels conveniently for diverse or stylistic gen-erations. A Chinese quatrain has at most 4*7=28characters and clear rhyme requirements. As thefirst attempt on unsupervised style disentangle-ment, we find Chinese poetry generation is anideal choice for evaluation because we can bet-ter focus on stylistic generation rather than deal-ing with genre requirements. We will explore theapplicability of our model on other languages andtasks for future work.

4 Experiments

Following the experimental settings in previousworks (Zhang et al., 2017), we conduct humanjudgment to evaluate the performance of ourmodel and state-of-the-art baseline methods. Wewill first introduce the dataset, our model details,baseline methods and evaluation settings. Then wewill present the experimental results and give fur-ther analysis about the evaluation. Finally, we willpresent some example generations for case study.

4.1 Dataset and Model Details

We collect 168,000 poems (half wuyan: five char-acters per line, half qiyan: seven characters perline) over 1,000 years history of classical Chinesepoetries as our corpus. We randomly select 80%of the corpus as training set, 10% as validation setand leave the rest for test. We extract all consec-utive sentences in a poem as training pairs (X,Y )and feed them into our model.

For hyperparameter settings of our model, i.e.stylistic poetry generation (SPG), we set the di-mensions of character embeddings and encoderhidden states as d = 512. The total number ofstyles is set to K = 10. Hence the dimension ofdecoder hidden states is 512 + 512 + 10 = 1034.We pack the training pairs to mini-batches and thesize of mini-batches is set to 50. The harmonic hy-perparameter is set to λ = 0 for the first 50, 000

mini-batches as pretraining and λ = 1.0 for subse-quent batches. We use Adam optimizer (Kingmaand Ba, 2014) for stochastic gradient descent. Wealso employ dropout (Srivastava et al., 2014) strat-egy with dropout rate 0.2 to avoid overfitting prob-lem. We terminate the training process when theoptimization loss on validation set is stable, i.e.300, 000 mini-batches in total.

4.2 BaselinesWe consider the following state-of-the-art poetrygeneration models for comparison:• Seq2seq (Bahdanau et al., 2014) is the

sequence-to-sequence model with attention mech-anism. Note that seq2seq is the basis of our SPGmodel. We can better analyze the improvement bystyle disentanglement from the comparison withseq2seq.• Polish (Yan, 2016) proposed an iterative

schema to polish and refine the generated sen-tences for several times instead of a one-pass gen-eration.• Memory (Zhang et al., 2017) incorporated

external memory into poem generation for betternovelty. The memory can be seen as a regulariza-tion to constrain the behavior of the neural model.

Rule-based and template-based methods are notconsidered as our baselines as they have been al-ready thoroughly compared in (He et al., 2012;Yan, 2016).

4.3 Evaluation SettingsFollowing the settings in (Zhang et al., 2017), weemploy human judgment for evaluation. To bettercompare the ability to generate fluent and coherentpoems, we fix the first sentence as input and let allmodels generate three subsequent sentences. Thefirst sentences are randomly chosen from the po-ems in the test set. Therefore we also consider theoriginal poems written by real poets for compari-son.

Note that our SPG model need a manually spec-ified style id as input. For fully automatic poemgeneration, we use the posterior style estimationfunction Q(·) to infer the style of the first sen-tence and then generate next three sentences se-quentially using the same style.

As previous works (Manurung, 2004; Yi et al.,2017) did, we design four criteria for human judg-ment: Fluency (are the generated poems fluentand well-formed?), Coherence (is the topic ofthe whole quatrain consistent?), Meaningfulness

3966

Group MethodsFluency Coherence Meaningfulness Poeticness

wuyan qiyan wuyan qiyan wuyan qiyan wuyan qiyan

Group 1seq2seq 2.75 2.48 2.60 2.33 2.38 2.15 2.40 2.35

SPG 3.43 3.23 3.13 3.05 2.83 3.10 3.10 3.10

Group 2

memory 2.50 2.60 2.30 2.38 2.18 2.25 2.05 2.40polish 2.53 2.90 2.28 2.55 2.15 2.55 2.20 2.50SPG 3.38 3.53 3.38 3.30 3.13 3.25 3.20 3.20

human poet 3.85 3.68 4.05 3.83 4.00 3.83 3.53 3.75

Table 1: Human judgment results. We bold the best performing model in each group.

(does the poem convey some certain messages?)and Poeticness (does the poem have some poeticfeatures?). Each criterion needs to be scored on a5-point scale ranging from 1 to 5.

For each model, we generate 20 wuyan and 20qiyan quatrains given the first sentence. We invite10 experts who major in Chinese literature or aremembers of a poetry association to evaluate thesequatrains. We divide the baseline methods intotwo groups: In the first group, we compare SPGwith seq2seq (Bahdanau et al., 2014) to present theadvantages of style disentanglement; In the sec-ond group, we compare SPG with state-of-the-artpoetry generation methods, memory (Zhang et al.,2017) and polish (Yan, 2016), and the original po-ems written by poets to demonstrate the effective-ness of our algorithm. Note that the scores of hu-man judgment are relative, not absolute. Thus wecompare with seq2seq separately to avoid mutualinterference.

4.4 Experimental Results

We report the average scores of expert judgmentsin Table 1. From the experimental results, we havethe following observations:

Firstly, the experimental results in group1 demonstrate that SPG outperforms seq2seq,the basis of our model, by learning a style-disentangled generation model. The only differ-ence between seq2seq and our model is that weappend a one-hot style id to the encoder state andadd mutual information regularization in Eq. 12 tothe loss function. Note that our model is fully un-supervised: the one-hot style id conveys no mean-ingful message unless the mutual information reg-ularization is considered. In other words, the one-hot style id and mutual information regularizationcannot be torn apart. Therefore, the improvementsover seq2seq all come from the part of style dis-entanglement modeling because seq2seq and our

model share all the other components. The com-parisons in group 1 of Table 1 are sufficient toshow the effectiveness of style modeling.

Secondly, the experimental results in group 2show that SPG consistently outperforms two state-of-the-art poem generation methods. SPG is ableto generate poetries in diverse styles without los-ing fluency and coherency. Our advantages aretwo-fold: On one hand, SPG better fits the diver-sity characteristic of human poetry writing. In-tuitively, seq2seq learns a generation model thatmixes poems in various styles and is more likelyto generate meaningless common sentences. Bydisentangling poems in different styles, our modelcan learn a more compact generation model foreach style and write more fluent and meaningfulpoems. On the other hand, by fixing the style whengenerating three lines in a quatrain, SPG is able togenerate more coherent poems.

One can also imagine that the generation prob-ability distribution actually consists of severalpeaks where each peak identifies a cluster of po-ems in similar styles. A traditional model mixesall the stuff together and learns a one-peak genera-tion model which is more likely to generate mean-ingless common sentences. In contrast, our modeldisentangles different clusters (styles) and learns amore accurate and compact generation model foreach cluster (style). Hence our model can generatepoems with higher quality and beat the baselinesin terms of human evaluation. This observationdemonstrates the effectiveness and robustness ofour model.

Finally, there is still a large margin betweenSPG and human poets. In this paper, we focus onpoem generation in diverse styles. Other poetrycharacteristics are also important for the quality ofgenerations. We will consider applying our style-disentangle regularization on other poem modelsfor better performance in the future.

3967

Style id Keywords1 loneliness, melancholy2 the portrait of landscape3 sorrow during roaming4 hermit, rural scenes5 grand scenery, regrets about old events6 sorrow during drinking7 emotions towards life experience8 the portrait of hazy sceneries9 reminiscence, homesickness

10 sadness about seasons

Table 2: Representative keywords for poems generatedby each learned style.

Figure 3: Experimental results on style recognition.Each row represents the human annotation of corre-sponding style generations. The diagonal blocks arecorrect classifications. Darker block indicates higherprobability.

4.5 Interpretability of Learned Styles

In this subsection, we conduct experiments to un-derstand the semantic meanings of each learnedstyle. To this end, we first generate 50 poems ineach style and then manually select two or threekeywords to sketch the overall feelings of everystyle. The keywords of all 10 styles are listedin Table 2. Note that the learned styles may notstrictly align the traditional writing styles recog-nized by human such as romantic or pastoral be-cause our model is fully unsupervised.

Then we generate another 50 poems (5 poemsper style) and ask the experts to classify these po-ems into the 10 styles according to the relationshipbetween generated poems and style keywords inTable 2. The human annotation results are shownin Fig. 3.

We can see that many learned styles can be suc-cessfully recognized by human with a higher prob-ability, e.g. the first style can be correctly clas-sified by 80% probability. This observation indi-cates that the learned styles of SPG are meaningfuland recognizable through only two or three key-words. Also, the generated poems are diverse oth-erwise they cannot be differentiated and correctlyclassified.

4.6 Case Study

We present three poems generated by SPG withthe same first sentence for case study. We onlylist the results of three most representative stylesin Fig. 4 and put other generation examples in sup-plementary materials (in Chinese). We can see thatthe poems generated by different style inputs dif-fer a lot from each other and follow the style key-words. To conclude, SPG can generate fluent andcoherent poetries in diverse styles.

5 Conclusion

In this paper, we propose a stylistic poetry gen-eration (SPG) model to learn the ability to writepoems in different styles under the same poeticimagery. To the best of our knowledge, we arethe first effort at stylistic poetry generation in afully unsupervised manner. Therefore, our modelrequires no expensive expert style annotation forthousands of poems in the database. We innova-tively employ mutual information, a concept in in-formation theory, to develop our model. Exper-imental results show that SPG is able to gener-ate fluent and coherent poetries in diverse styleswithout losing fluency and coherency. The learnedstyles are meaningful and recognizable given onlytwo or three keywords. Our algorithm has been in-corporated into Jiuge2 poetry generation system.

For future works, we will consider adopting themutual information regularization for other textgeneration tasks which encourage stylistic gener-ation or diversity to improve their performances.Another intriguing direction is to refine our modelfor more task-specific scenarios. Besides, ourmodel simplifies the prior of style distribution asa uniform distribution. This assumption could befurther improved by an iteratively updating ap-proach.

2https://jiuge.thunlp.cn

3968

(a) Style 1: “loneliness, melancholy” (b) Style 4: “hermit, rural scenes” (c) Style 8: “the portrait of hazy scener-ies”

Figure 4: Examples generated by style 1,4 and 8 given the same first sentence. The keywords of the three stylesare listed for convenience.

Acknowledgments

We would like to thank Chongxuan Li, HuiminChen, Jiannan Liang, Zhipeng Guo and anony-mous reviewers for their insightful comments.This research is funded by the National 973project (No. 2014CB340501). It is also par-tially supported by the NExT++ project, the Na-tional Research Foundation, Prime Minister’s Of-fice, Singapore under its IRC@Singapore FundingInitiative.

ReferencesDzmitry Bahdanau, Kyunghyun Cho, and Yoshua Ben-

gio. 2014. Neural machine translation by jointlylearning to align and translate. arXiv preprintarXiv:1409.0473.

David Barber and Felix Agakov. 2003. The im algo-rithm: a variational approach to information maxi-mization. In Proceedings of the 16th InternationalConference on Neural Information Processing Sys-tems, pages 201–208. MIT Press.

David M Blei, Andrew Y Ng, and Michael I Jordan.2003. Latent dirichlet allocation. Journal of ma-chine Learning research, 3(Jan):993–1022.

Xi Chen, Yan Duan, Rein Houthooft, John Schulman,Ilya Sutskever, and Pieter Abbeel. 2016. Infogan:Interpretable representation learning by informationmaximizing generative adversarial nets. In NeuralInformation Processing Systems.

Pablo Gervas. 2001. An expert system for the compo-sition of formal spanish poetry. Knowledge-BasedSystems, 14(3-4):181–188.

Jing He, Ming Zhou, and Long Jiang. 2012. Generat-ing chinese classical poems with statistical machinetranslation models. In AAAI.

Sepp Hochreiter and Jurgen Schmidhuber. 1997.Long short-term memory. Neural computation,9(8):1735–1780.

Zhiting Hu, Zichao Yang, Xiaodan Liang, RuslanSalakhutdinov, and Eric P Xing. 2017. Controllabletext generation. arXiv preprint arXiv:1703.00955.

Diederik P Kingma and Jimmy Ba. 2014. Adam: Amethod for stochastic optimization. arXiv preprintarXiv:1412.6980.

Tomas Kocisky, Gabor Melis, Edward Grefenstette,Chris Dyer, Ling Wang, Phil Blunsom, andKarl Moritz Hermann. 2016. Semantic parsing withsemi-supervised sequential autoencoders. In Con-ference on Empirical Methods in Natural LanguageProcessing, pages 1078–1087.

Robert P Levy. 2001. A computational model of poeticcreativity with neural network as measure of adap-tive fitness. In Proceedings of the ICCBR-01 Work-shop on Creative Systems.

Hisar Manurung. 2004. An evolutionary algorithm ap-proach to poetry generation.

Yael Netzer, David Gabay, Yoav Goldberg, andMichael Elhadad. 2009. Gaiku: Generating haikuwith word associations norms. In Proceedings of theWorkshop on Computational Approaches to Linguis-tic Creativity, pages 32–39. Association for Compu-tational Linguistics.

Hugo Goncalo Oliveira. 2012. Poetryme: a versatileplatform for poetry generation. Computational Cre-ativity, Concept Invention, and General Intelligence,1:21.

Mike Schuster and Kuldip K Paliwal. 1997. Bidirec-tional recurrent neural networks. IEEE Transactionson Signal Processing, 45(11):2673–2681.

Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky,Ilya Sutskever, and Ruslan Salakhutdinov. 2014.Dropout: A simple way to prevent neural networks

3969

from overfitting. The Journal of Machine LearningResearch, 15(1):1929–1958.

Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014.Sequence to sequence learning with neural net-works. In Advances in neural information process-ing systems, pages 3104–3112.

Zhe Wang, Wei He, Hua Wu, Haiyang Wu, Wei Li,Haifeng Wang, and Enhong Chen. 2016. Chinesepoetry generation with planning based neural net-work. arXiv preprint arXiv:1610.09889.

Xiaofeng Wu, Naoko Tosa, and Ryohei Nakatsu. 2009.New hitch haiku: An interactive renku poem com-position supporting tool applied for sightseeing nav-igation system. In International Conference on En-tertainment Computing, pages 191–196. Springer.

Rui Yan. 2016. i, poet: Automatic poetry compositionthrough recurrent neural networks with iterative pol-ishing schema. In IJCAI, pages 2238–2244.

Rui Yan, Han Jiang, Mirella Lapata, Shou-De Lin,Xueqiang Lv, and Xiaoming Li. 2013. i, poet: Au-tomatic chinese poetry composition through a gen-erative summarization framework under constrainedoptimization. In IJCAI, pages 2197–2203.

Xiaopeng Yang, Xiaowen Lin, Shunda Suo, and MingLi. 2017. Generating thematic chinese poetry withconditional variational autoencoder. arXiv preprintarXiv:1711.07632.

Xiaoyuan Yi, Ruoyu Li, and Maosong Sun. 2017. Gen-erating chinese classical poems with rnn encoder-decoder. In Chinese Computational Linguistics andNatural Language Processing Based on NaturallyAnnotated Big Data, pages 211–223. Springer.

Jiyuan Zhang, Yang Feng, Dong Wang, Yang Wang,Andrew Abel, Shiyue Zhang, and Andi Zhang.2017. Flexible and creative chinese poetry gen-eration using neural memory. arXiv preprintarXiv:1705.03773.

Xingxing Zhang and Mirella Lapata. 2014. Chinesepoetry generation with recurrent neural networks.In Proceedings of the 2014 Conference on Em-pirical Methods in Natural Language Processing(EMNLP), pages 670–680.