optimized event storyline generation based on mixture ... · the key challenge to our storyline...

Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 726–735,Seattle, Washington, USA, 18-21 October 2013. c©2013 Association for Computational Linguistics

Optimized Event Storyline Generation based on Mixture-Event-AspectModel

Lifu HuangShenzhen Key Lab for Cloud Computing

Technology and Applications,Peking University Shenzhen Graduate School,

Shenzhen, Guangdong 518055, [email protected]

Lian’en Huang∗

Shenzhen Key Lab for Cloud ComputingTechnology and Applications,

Peking University Shenzhen Graduate School,Shenzhen, Guangdong 518055, P.R.China

[email protected]

Abstract

Recently, much research focuses on event s-toryline generation, which aims to produce aconcise, global and temporal event summaryfrom a collection of articles. Generally, eachevent contains multiple sub-events and the sto-ryline should be composed by the componen-t summaries of all the sub-events. However,different sub-events have different part-wholerelationship with the major event, which isimportant to correspond to users’ interestsbut seldom considered in previous work. Todistinguish different types of sub-events, wepropose a mixture-event-aspect model whichmodels different sub-events into local andglobal aspects. Combining these local/globalaspects with summarization requirements to-gether, we utilize an optimization method togenerate the component summaries along thetimeline. We develop experimental systems on6 distinctively different datasets. Evaluationand comparison results indicate the effective-ness of our proposed method.

1 Introduction

With the rapid growth of the World Wide Web, in-formation explosion has become an important issueto modern people. Those who search for informa-tion from the Internet often get lost and confusedby the overwhelmingly large collection of web doc-uments. So how to get a concise and global pic-ture for a given event subject is an urgent prob-lem to be solved. Although many document un-derstanding systems have been proposed, such as

∗Corresponding author

multi-document summarization systems, to gener-ate a compressed summary by extracting the ma-jor information from the collection of documents,they ignored the dynamic development informationof an event. Intuitively, each event is long-runningand contains multiple sub-events, including relatedevents. Users are likely to prefer a summary of alloccurrences of all the sub-events along the timelineof the event. This motivates us to study the task ofgenerating event storyline from a collection of webdocuments related to an event subject.

The research of event storyline summarization ispopular in recent years. Its task is to summarize acollection of web documents by extracting represen-tative information based on all the sub-events andgenerate a global summary. Generally, generatingsuch a global storyline is quite interesting for the fol-lowing main reasons: (1) It can help people catch thewhole incident based on an overall temporal struc-tured summary for a given subject, and understandthe cause, climax, development process and resultof an event. (2) It can also make people know whatother events are related, or the effect of this incidentto subsequent events, which can present the evolu-tion of an event along a timeline.

Though several methods of generating event sto-ryline have been proposed recently, there are stillsome problems unresolved. As event storyline sum-marization is a process to generate component sum-maries based on the multiple sub-events, which is d-ifferent from traditional summarization focusing ononly one subject, so how to exactly extract all thesub-events is the first challenge. Moreover, user-s tend to bias to the sub-events which have global

726

consistency with the given event subject, so the sub-events should not be considered equally when gen-erating the component summaries. It is also a greatchallenge to generate a qualified summary based onthe different types of sub-events. The componen-t summaries should be correlative across differen-t dates based on the global collection (Yan et al.,2011a).

Mei and Zhai (Mei and Zhai, 2005) proposed touse theme or topic to model different sub-events,which is to some extent similar to our method. Tobe different, in this paper we introduce “local/glob-al” property to distinguish different part-whole rela-tionship between the sub-events and the major even-t, which have not been considered before in story-line generation or summarization, to improve thequality of the storyline. The local/global proper-ty corresponds to the elements of an event, suchas the place, characters and other body informa-tion. These information reflects the relationship be-tween the sub-events and the major event. Somesub-events have distinctive body information and lit-tle relevance with each other. They generally occurfor a local period, which we name as “local-sub-events”. While other sub-events often share com-mon properties with each other and have close re-lationship with the major event and we call themas “global-sub-events”. Here we give some exam-ples to illustrate the difference. For the event “Con-necticut school shooting” which occurred on Dec.142012, its sub-events such as “Obama’s speech forthis massacre” or “Gun control Act” have little wordco-occurrences and distinctive event body informa-tion to each other, while the process and result of thistragedy can be regarded as global-sub-events whichhave a lot of word co-occurrences and share com-mon properties with the major event.

Inspired by these, to detect different types of sub-events based on word co-occurrences between sub-events and the major event, we propose a mixture-event-aspect (MEA) model to formalize differen-t types of sub-events into local/global aspects, whichare implicated with clusters of sentences. Then com-bining the local/global aspects with summarizationrequirements together, we utilized an optimizationapproach to get the optimal component summariesalong the timeline. We evaluate our method on 6 dis-tinctively different datasets. Performance compar-

isons among different system-generated storylinesdemonstrate the necessity to distinguish differen-t types of sub-events and also indicates the effective-ness of the proposed mixture-event-aspect model.

The rest of the paper is organized as follows. Webriefly review the related work in section 2. In sec-tion 3 we present the details of optimized event s-toryline generation based on mixture-event-aspectmodel. Experiments and results are discussed inSection 4. Finally we draw a conclusion of this s-tudy in Section 5.

2 Related Work

Our work is related to several lines of research in theliterature including multi-document summarization(MDS), topic detection and tracking (TDT), tempo-ral text mining (TXM) and temporal news summa-rization (TNS).

Multi-document summarization is a process togenerate a summary by reducing documents in sizewhile retaining the main information. To date, dif-ferent features and ranking strategies have been s-tudied. Radev et al. (Radev et al., 2004) proposedto implement MEAD as a centroid-based summa-rizer by combining several predefined features to s-core the sentence. LexPageRank (Erkan and Radev,2004) is the representative work which is based onPageRank (Page et al., 1999) algorithm. Some meth-ods have been proposed to extend the conventionalgraph-based models recently including multi-layergraph incorporated with different relationship (Wan,2008), ToPageRank based on the topic information(Pei et al., 2012) and DivRank (Mei et al., 2010) bal-ancing the prestige and diversity.

Topic detection and tracking (TDT) aims to groupnews articles based on the topics discussed in them,detect some novel and previously unreported eventsand track future events related to the topics (Wanget al., 2012). Kumaran and Allan (Kumaran and Al-lan, 2004) showed how performance on new eventdetection could be improved by the use of text clas-sification techniques as well as by using named en-tities in a new way. Makkonen et al. (Makkonenet al., 2004) proposed a method that incorporatedsimple semantics into TDT by splitting the term s-pace into groups of terms. Krause et al. Wang et al.(Wang et al., 2007) and Wang et al. (Wang et al.,

727

2009) worked on topic tracking from multiple newsstreams. Their methods extracted meaningful top-ics from multi-source news collections and trackeddifferent topics as they evolved from one to anotheralong the timeline.

Our work is also related to temporal text miningand temporal news summarization. The task of tem-poral news summarization is to generate news sum-maries along the timeline from massive data. Chieuet al. (Chieu and Lee, 2004) built a system that ex-tracted events relevant to a query from a collectionof related documents and placed such events alonga timeline. Yan et al. (Yan et al., 2011b) designedan evolutionary timeline summarization approach toconstruct a timeline of a topic by optimizing the rel-evance, coverage, coherence, and diversity. Lin et al.(Lin et al., 2012) explored the problem of generatingstorylines from microblogs for user input queries.They first proposed a language model with dynamicpseudo relevance feedback to obtain relevant tweet-s and then generated storylines via graph optimiza-tion.

3 Approach Details

In this section, we first propose a mixture-event-aspect model to detect local/global sub-events basedon part-whole relationship with the major event andthen present a new method to estimate the bursty ofeach aspect on a certain date. Afterwards we uti-lize an optimization method based on local/globalaspects to extract the qualified summary.

3.1 Mixture-Event-Aspect Model

The key challenge to our storyline generation taskis to detect and distinguish different types of sub-events contained in the article collection. In the col-lection, each sentence is assigned with a certain dateand sentences that are assigned with the same dateare grouped into the same sub-collection. Consid-ering the consistency of content between the sub-events and the major event, we model different sub-events into two types: local-sub-event and global-sub-event, and introduce local/global aspects cor-respondingly. Generally, local aspects which cor-respond to local-sub-events have distinctive word-s distribution from each other and sustain for a lo-cal context while the global aspects corresponding

to global-sub-events have coincident words distri-bution with the major event. To capture specificwords, Titov and McDonald (Titov and McDonald,2008) proposed a multi-grain topic model, relyingon word co-occurrences within short paragraphs andLi et al. (Li et al., 2010) proposed a entity-aspectmodel based on word co-occurrences within singlesentences. Inspired by these ideas, we rely on wordco-occurrences within local period context to detec-t mixed local and global aspects implicated in thewhole collection. We name this model as “Mixture-Event-Aspect (MEA)” model which can simultane-ously detect local/global aspects and cluster sen-tences and words into different aspects.

3.1.1 Model DescriptionOur mixture-event-aspect (MEA) model can be

extended from both the Probabilistic Latent Seman-tic Analysis (PLSA) (Hofmann, 1999) and Laten-t Dirichlet Allocation (LDA) (Blei et al., 2003). Wemodel two distinct types of aspects: global aspect-s and local aspects, based on their relationship withthe major event. The distribution of global aspects isfixed for the collection while the distribution of localaspects is fixed to a local period of sub-collections.That means a sentence is sampled either from themixture of the global aspects or from the local as-pects specific for the local context. Here we takethe event “Connecticut school shooting” as an exam-ple. For the sentence “On Sunday, President Obamacame to Connecticut to give a lecture, expressing hissorrow for ... and calling for an end to such inci-dents”, the words such as “Obama”, “lecture”, “ex-press” are only occurred for the local period of twodays and have no co-occurrence with other neigh-boring period sentences, so we sample the sentenceas a local aspect sentence. But for the sentence “Allschools in Newtown, the northeastern U.S. state ofConnecticut were in lockdown after a shooting wasreported at a local elementary school”, the word-s such as “Connecticut”, “shooting”, “elementary”have high co-occurrence frequency in the whole col-lection, so we sample the sentence as a global aspectsentence.

To detect aspects, we first divide words into twotypes: aspect words and background words. Back-ground words are commonly used in the whole even-t corpus while aspect words are clearly associat-

728

ed with the aspects of the sentences they occur in.Stop words are removed using a standard stop wordlist. In order to get the distribution of local aspect-s, we implement a mechanism called “Time Win-dow” which covers Sp sequential time-based sub-collections. We associate each time window witha distribution over local aspects and a distributiondefining preference of local aspects versus global as-pects.

draw φB ∼ Dir(β), ψc(v) ∼ Dir(λ), π ∼ Dir(γ)

draw φgl ∼ Dir(β) for Agl times

draw φloc ∼ Dir(β) for Aloc times

choose a distribution of global aspects θgl ∼ Dir(αgl)

For each time window v in collection c

choose θlocc,v ∼ Dir(αloc)

choose ρc,v ∼ Beta(αmix)

For each sentence s in collection c

choose window νc,s ∼ ψc

choose ηc,s ∼ ρc,νc,s

if ηc,s = gl, zc,s ∼ θgl

if ηc,s = loc, zc,s ∼ θlocc,v

For each word w of sentence s in collection c

draw yc,s,n ∼Multi(π)

draw wc,s,n ∼Multi(φB) if yc,s,n = 1

draw wc,s,n ∼Multi(φzc,s) if yc,s,n = 2

Figure 1: The Collection Generation Process

Formally, let C = {Ct|t = 1, 2, 3, ..., T} beT time based sub-collections related to the even-t subject, Ct represents the collection of sentenceswhich are assigned with the date t. Let v be a timewindow containing Sp sequential sub-collections,v = {Ct|t = i, i + 1, ..., i + Sp − 1}. We drawa background unigram language model which gen-erates words for all sub-collections, and draw Agl

global aspect unigram language models for globalaspects andAloc word distributions for local aspects.We assume these word distributions have a uniformDirichlet prior Dir(β). There is also a multinomi-al distribution π that controls in each sentence howoften the word occurs as a background word or anaspect word. π has a Dirichlet prior with parame-ter γ. We assign each window v with an distributionover local aspects and a distribution ρ defining pref-erence for local aspects versus global aspects. ρ has

a Beta prior αmix. A sentence can be sampled usingany window which is chosen according to a categor-ical distribution.

In Figure 2 the corresponding graphical model ispresented. This model allows for fast approximateinference with collapsed Gibbs sampling.

Figure 2: Mixture-Event-Aspect Model

Let SC denotes the number of sentences in col-lection C, Nc,s denotes the number of words in sen-tence s of collection c, and wc,s,n denotes the nth

word in sentence s. There are two kinds of hiddenvariables: zc,s for each sentence to indicate the as-pect a sentence belongs to, and yc,s,n for each wordto indicate whether a word is generated from thebackground model or the aspect model.

3.1.2 Inference via Gibbs Sampling

In order to estimate the hidden parameter-s in the model, we try to maximize distributionp(z,y|w;α, β, γ, λ), where z, y and w represent theset of all z, y and w variables, respectively. Giv-en a sentence s in the collection c, we apply GibbsSampling to estimate the conditional probability forlocal/global aspects using the following rules:

p(vc,s = vh, ηc,s = gl, zc,s = a|v′ , z′ , y, w) ∝nc

h∗+λ

nc(.)

+S∗pλ ·

nc,vhgl

+αmixgl

nc,vh(.)

+∑

r′∈gl,loc

αmix

r′

· ncgl,a+αgl

ncgl

+Aglαgl ·

∏L

l=1

∏E(l)−1

i=0(Ca

(l)+i+β)

ΠE(.)−1i=0 (Ca

(.)+i+Lβ)

729

p(vc,s = vh, ηc,s = loc, zc,s = a|v′ , z′ , y, w) ∝nc

h∗+λ

nc(.)

+S∗pλ · n

c,vhloc

+αmixloc

nc,vh(.)

+∑

r′∈gl,loc

αmix

r′

·n

c,vhloc,a

+αloc

nc,vhloc

+Alocαloc·

∏L

l=1

∏E(l)−1

i=0(Ca

(l)+i+β)∏E(.)−1

i=0(Ca

(.)+i+Lβ)

where nch∗ =#{si|vc,si = vi−sp+h∗+1}, denotes

the number of times a sentence si in collection c isassigned to its h∗th window and for each sentencesi, we have h = i− Sp + 1 + h∗. S∗

p is the numberof windows that contain sentence s. nc

(.) denotes thenumber of sentences in collection c. Sp representsthe number of dates a window covered. nc,vh

gl andnc,vh

loc are the number of sentences in window vh thatare assigned to global or local aspects. nc,vh

(.) is thenumber of sentences assigned to window vh. nc

gl,a

is the number of sentences in all global aspects thatare assigned to aspect a and nc,vh

loc,a is the number oflocal aspect sentences in the window vh are assignedto aspect a. Agl is the number of global aspects incollection C while Aloc is the number of local as-pects. E(l) represents the number of times of wordl occurs in the current sentence and is assigned tobe an aspect word, while E(.) is the total number ofwords in the current sentence that are assigned to bean aspect word.

Then we compute the assignments of the consid-ered word. We sample the hidden variables yc,s,n foreach word with the following rules:

p(yc,s,n = 1|z, y′) ∝

Cπ(1) + γ

Cπ(·) + 2 · γ

·CB

(wc,s,n) + β

CB(·) + L · β

p(yc,s,n = 2|z, y′) ∝

Cπ(2) + γ

Cπ(·) + 2 · γ

·Ca

(wc,s,n) + β

Ca(·) + L · β

where Cπ(1) and Cπ

(2) are the numbers of words as-signed to be background words and aspect words.Cπ

(·) is the total number of words. CB(·) is the total

number of background words and Ca(·) is the number

of words assigned to aspect a.By using the samples from Gibbs sampling, we

can effectively make the following estimations:

φBw = CB

w +β

CB· +L·β , φ

aw = Ca

w+βCa

w+L·β , ψh∗ =nc

h∗+λ

nc(.)

+S∗pλ

πy =Cπ

y +γ

Cπ· +2·γ , ρ

c,vη =

nc,vη +αmix

η

nc,v+∑

r∈(gl,loc)

αmixr

θgla =

ncgl,a+αgl

ncgl

+Aglαgl , θloca =

nc,vloc,a

+αloc

nc,vloc

+Alocαloc

The hyper-parameters like α, β, γ, λ can be estimat-ed using standard methods introduced in (Minka,2000).

3.2 Bursty Period DetectionWe borrow the definition of “bursty” from (Lappaset al., 2009) to measure the popularity of the eventon a certain date. Intuitively, each aspect have differ-ent bursties on different dates. In this section, we tryto obtain the temporal aspect sequences of an eventbased on the bursty periods of all the aspects. Dur-ing its bursty period, one aspect should (1)be morepopular than other aspects (2) be continuously morepopular than other time. Following these intuitions,we design a method to measure the bursty of eachaspect and get the bursty period.

Let Ak be the kth aspect obtained from themixture-event-aspect model, we estimate the burstyof Ak at a certain date t as follows.

bursty(Ak,t) = p(t|Ak) =p(Ak|t) · p(t)∑

t′p(Ak|t′)p(t′)

where p(Ak|t) is measured by the number of sen-tences assigned to aspect Ak in date t divided by thetotal number of sentences in date t. p(t) is estimatedby the total number of sentences in aspect Ak divid-ed by the overall number of sentences in the collec-tion C.

After getting the bursty of aspect Ak at each date,we can find the most popular date and expand onboth sides to obtain the whole burst period in whichthe bursties are higher than the neighboring aspectsand continuous higher than other dates.

3.3 Optimization-based Storyline GenerationWith the methods discussed in previous sections, wecan get the local/global aspect sequence. Each as-pect contains numbers of sentences and we are aim-ing to select the most representative ones to composethe final storyline. Considering users’ bias and thelength requirement, different aspects should have d-ifferent proportions in the last storyline. For glob-al aspects which correspond more to users’ interest,they should share a larger proportion in the final sto-ryline than local aspects. Thus, we use an optimiza-tion method to determine if a sentence is selected tobe an summary sentence or to be discarded based onthe multiple local/global aspects and finally get theoptimal storyline. We formalize this problem as se-lecting a subset of sentences S from the aspect Ak

730

to minimize the information loss.

arg min∑

Ak∈C,S∈Ak

∑z∈Ak−S,s∈S

O(z, s)

where O(z, s) is the cost function which measuresthe cost of representing sentence z with sentence s.Generally, this is an NP-hard problem (Cheung etal., 2009) but we can use POPSTAR, an implemen-tation of an approximate solution proposed by Re-sende and Werneck (Resende and Werneck, 2004).To model different costs between global or localaspects and determine the proportions of differen-t aspects in the final storyline, we utilize a func-tion ζ(s). When sentences z and s are local as-pect sentences, ζ(s) = χ, or, ζ(s) = 1 − χ. For-mally, we incorporate two kinds of decreasing/in-creasing logistic functions, ℓ1(x) = 1/(1 + ex) andℓ2(x) = ex/(1 + ex), to define the cost function as

O(z, s) = ζ(s) · ℓ1(S(s)) · ℓ2(S(z)) ·DKL(s, z)

where S(s) and S(z) are the ranking scores of sen-tences s and z among the aspectAk with LexPageR-ank algorithm. DKL(s, z) is used to measure thesimilarity between sentence s and z with Kullback-Leibler divergence here.

With this optimization method, we get the repre-sentative sentences of each aspect for the given eventsubject. Combining all the representative sentencestogether based on the aspect sequence, we finallygenerate the storyline.

4 Experiments and Evaluation

4.1 DatasetsTo evaluate our framework for event storyline gen-eration, we conduct our experiments on the dataset-s amounting to 12418 articles for 6 event subjectsfrom 6 famous news websites, which provide dateedited by professional editors. Each article consistsof three parts, title, publish-time and news content.Table 1 and Table 2 give the brief description of the12418 articles. To generate reference summary, weinvite 12 undergraduate students with good Englishability to read the sentences, and for each event sub-ject we ask two students to label human storylines.

4.2 Evaluation MetricsWe use the ROUGE1 (Lin and Hovy, 2003) (RecallOriented Understudy for Gisting Evaluation) toolkit

1http://www.isi/edu/licensed-sw/see/rouge/

Table 1: News sources of the 6 datasetsNews Sources Number of Articles

CNN 2357Fox News 1936

New York Times 2178ABC 2113

Washington Post 1405Xinhua 2429

Table 2: Event subjects of the datasetsEvent Subjects Number of Articles

Connecticut school shooting 1792The earthquake in Tokyo, Japan 2046The U.S. presidential election 2573

Sandy hurricane attacked America 1827American curiosity rover landed on Mars 1651

The 30th London Olympic Games 2529

to evaluate our framework, which has been widelyapplied for summarization evaluation. It evaluatesthe quality of a summary by counting the overlap-ping units between the candidate summary and ref-erence summaries. There are many kinds of ROUGEmetrics to measure the system-generated summa-rization such as ROUGE-N, ROUGE-L, ROUGE-W, and ROUGE-U, of which the most important oneis ROUGE-N with 3 sub-metrics: precision, recall,and F-score.

ROUGE − N =

∑S∈RS

∑N−gram∈S

Countmatch(N − gram)∑S∈RS

∑N−gram∈S

Count(N − gram)

where RS represents the reference summaries. N-gram∈RS in the metrics denotes the N-grams in ref-erence summaries. Countmatch(N − gram) is themaximum number of N-grams co-occurring in thecandidate summary and in the set of reference sum-maries. Count(N − gram) is the number of N-grams in the reference summaries.

The ROUGE toolkit can report separate scores for1, 2, 3, and 4-gram. In the experimental results wereport three ROUGE F-measure scores: ROUGE-1, ROUGE-2, ROUGE-W metrics. The higher theROUGE scores, the better the summary is.

4.3 Algorithms for ComparisonGiven a collection of news articles, we first decom-pose them into sentences, and then assign each sen-tence with a certain date, afterwards stop-words re-moving and words stemming are performed. Wechoose the following algorithms as baseline system-s. Specifically, baseline 2 and 3 are summarization

731

systems which are similar to our storyline genera-tion system. Then we choose baseline 4, 5 to evalu-ate the effectiveness of the proposed method. It mustbe said that all the systems are required to generatethe same number of summary words with the hu-man reference. We conduct the same preprocessingfor all algorithms for fairness.• Random : The method selects sentences ran-

domly from the sentence collection.• LexPageRank (LexRank): This method applies

the graph-based multi-document summarization al-gorithm which first constructs a sentence connec-tivity graph based on the cosine similarity and thenchooses top-ranked sentences with PageRank.• Chieu : This method was proposed by Chieu

(Chieu and Lee, 2004), utilizing interest and bursti-ness to rank sentences, and choosing the top-rankedquery related sentences to construct the timeline.• LDA+LexPageRank (LDALR) : This method

first applies standard LDA to detect latent topicsfrom the collection and clusters sentences to mul-tiple aspects, then utilizes PageRank to generate themost representative component summaries from allthe aspects.• MEA+LexPageRank (MEALR) : This method

applies the proposed mixture-event-aspect model tocluster sentences into multiple aspects and then uti-lizes PageRank to generate the most representativecomponent summaries from all the aspects.• MEA+Optimization (MEAOp) : This method

extracts local/global aspects with the proposedmixture-event-aspect model, and then utilizes theoptimization method to get the qualified summary.

Figure 3: Overall performance for comparison

Table 3: Results of different systems on 6 subjectsSubject1 Subject2

Systems R-1 R-2 R-W R-1 R-2 R-WRandom 0.234 0.037 0.188 0.242 0.039 0.192LexRank 0.317 0.045 0.257 0.326 0.051 0.262

Chieu 0.332 0.056 0.277 0.351 0.055 0.283LDALR 0.356 0.069 0.297 0.369 0.066 0.327MEALR 0.369 0.072 0.313 0.381 0.076 0.348MEAOp 0.381 0.075 0.331 0.398 0.081 0.364

Subject3 Subject4Systems R-1 R-2 R-W R-1 R-2 R-WRandom 0.258 0.042 0.191 0.234 0.036 0.179LexRank 0.339 0.049 0.272 0.309 0.043 0.242


Subject5 Subject6Systems R-1 R-2 R-W R-1 R-2 R-WRandom 0.222 0.034 0.166 0.264 0.045 0.195LexRank 0.309 0.042 0.237 0.349 0.054 0.276


4.4 Overall Performance Comparison

We experiment with all the baselines and our frame-work on the 6 datasets. We take the average F-score performance in terms of 3 ROUGE-F scores:ROUGE-1, ROUGE-2 and ROUGE-SU4. The over-all results are shown in Figure 3 and details are listedin Tables 3.

Figure 3 and Table 3 show the performance ofthese systems on the same datasets. The local/globaloptimization balance parameter χ = 0.5. From Fig-ure 3 and Table 3 we have following observations:• Generally, the Random gets the worst perfor-

mance;• The LexRank system outperforms Random al-

gorithm. This is due to the fact that LexRank ranksall the sentences based on eigenvector centrality andthe global relationship between sentences, whichtends to select the most informative sentences as thesummary.• The results of Chieu (Chieu and Lee, 2004) sys-

tem are better than those of LexRank. This maybe mainly for the reason that Chieu used the datedimension to filter away uninteresting sentences byparaphrasing and defined two different ranking mea-sures: interest and burtiness, to select top-ranked in-formative sentences.• The LDALR system outperforms the Chieu sys-

732

tem. This may be for the fact that Chieu’s method isactually based on flat clustering-based summariza-tion, which is not as effective as LDA topic modelto extract latent sub-events.

Figure 4: Examine the performance of the balance pa-rameter χ

• The MEALR system outperforms the LDAL-R system. This may be mainly for the reason thatMEALR utilizes the mixture-event-aspect model todetect the more salient sub-events based on the sub-whole relationship, which seems to satisfy users’bias to different sub-events.• The MEAOp system which utilizes our method

outperforms all the baselines, indicating the effec-tiveness of detecting different types of sub-eventswith mixture-event-aspect model and the necessityto distinguish different proportions of the compo-nent summaries based on local/global aspects.

4.5 Parameter Tuning

Figure 5: Aspect sequence of event “Connecticut schoolshooting” (X-axis is the number of days after Dec. 14,2012. Y-axis is bursty(Ak,t))

In this section, we compare the performance ofthe parameters. The hyper-parameters such as α, β,γ, λ can be estimated using standard methods intro-duced by Minka (Minka, 2000). So we mainly ex-amine the local/global optimization balance parame-ter χ. We try to evaluate the influence of this param-eter on the three kinds of ROUGE measure resultsrespectively. Figure 4 shows the performance of thebalance parameters χ. It is obvious that when thebalance parameter χ is set to 0.7 this method per-forms best.

4.6 Sample Output and Case Study

Figure 6: Bursties of sub-event “Gun control debate” (X-axis is the number of days after Dec. 14, 2012. Y-axis isbursty(Ak,t))

We take the event “Connecticut school shooting”as an example to show the usefulness of our method.Figure 5 shows the aspect sequence based on thebursty periods of all aspects. We select a sub-event “Gun control debate” and Figure 6 shows thebursties of this sub-event on the whole timeline. Ta-ble 4 shows part of the storylines for the event “Con-necticut school shooting” generated by human andour method. Through observation, we find that thepeak of the event “Connecticut school shooting” isaround the date when it occurred, and the sub-event“Gun control debate” has two bursty periods aroundthe two peaks. Compared with the human summary,our framework can extract the important sub-eventscontained in the collection, and satisfy users’ inter-est on different sub-events based on the part-wholerelationship with the event subject.

From the sample output and the human storylines,we also get some observations. (1) The componentsummary of global aspect tend to share larger pro-

733

Table 4: Selected part of storyline generated by MEAOp and humanStoryline Generated by human:December 14, 2012 Global AspectShooting massacre occurred in a primary school in Connecticut town, Sandy hoot Newton.20-year-old Adam lanza broke into the primary school after shotting his mother, and in 10 minutes shot more than 100 times, killing twenty childrenand eight adult, including himself.The youngest death was a children in preschool students.December 15, 2012 Global AspectPhotos of the teachers and students shoot in the Connecticut massacre are released as well as the shooter’s.The shooter was very smart but lonely.December 16-17, 2012 Local AspectPresident Obama arrived in the locality of school shooting, mourned for the victims and made a speech.December 18-20, 2012 Local AspectAmerican gun control bill was put on the agenda again.

Storyline Generated by MEAOp:December 14, 2012 Global AspectChildren and adults gunned down in Connecticut school massacre.20 children, six adults and the shooter are dead after shooting at Sandy Hoot Elementary School in Newtown, Connecticut.Three law enforcement officials say Adam Lanza, 20, was the shooter, and that he died apparently by his own hand.Suspect’s mother, Nancy Lanza, found dead in suspects home in Newtown, law enforcement source says.Ryan Lanza, older brother of Adam Lanza, questioned by police but not labeled a suspect.December 15-16, 2012 Global AspectVictims’ names released Saturday; all of the slain children were either 6 or 7 years old.Understanding school lockdowns in regards to Connecticut shooting.Connecticut gunman recalled as intelligent but remote.December 17, 2012 Local AspectPresident obama leads interfaith preyer vigil in newtown connecticut.A tearful Obama says “we’ve endured too many of these tragedies”.December 18-20, 2012 Local AspectModerate dems join gun control debate call for commission on us violence gains.Gun debate gains traction as some lawmakers say its time to act.

portion in the final storyline. This is mainly forthe reason that when researching for an event sub-ject, users bias more to the information about theglobal-sub-events that have closely connection andcoincident properties with the major event based onthe part-whole relationship. So it is really neces-sary to distinguish different sub-events with distinc-tive properties. (2) Our system performs better forthe persistent event, such as “The U.S. presidentialelection”. This may be for the fact that these eventsare usually long running and have more global-sub-events than local-sub-events.

5 Conclusion

In this work, we study the task of event storylinegeneration and present a novel method. We inno-vatively introduce the properties of different sub-events based on word co-occurrences to determinethe part-whole relationship with the major event anddevelop a mixture-event-aspect (MEA) model to for-malize different types of sub-events into local/globalaspects. Based on these local/global aspects, we uti-lize an optimization method to get the optimal com-

ponent summaries along the aspect sequence. Weconduct experiments with our method and variousbaselines on real web datasets. Through our exper-iments we notice that our method generates over-all better storyline than other baselines. This indi-cates the effectiveness to detect different types ofsub-events with the proposed mixture-event-aspectmodel and the necessity to distinguish different pro-portions of the component summaries based on lo-cal/global aspects.

Acknowledgments.We thank the anonymous reviewers for their

valuable and constructive comments. This workis financially supported by NSFC under the grantNO.61272340 and 60933004.

References

David M Blei, Andrew Y Ng, and Michael I Jordan.2003. Latent dirichlet allocation. the Journal of ma-chine Learning research, 3:993–1022.

Jackie Chi Kit Cheung, Giuseppe Carenini, and Ray-mond T Ng. 2009. Optimization-based content se-

734

lection for opinion summarization. In Proceedings ofthe 2009 Workshop on Language Generation and Sum-marisation, pages 7–14. ACL.

Hai Leong Chieu and Yoong Keok Lee. 2004. Querybased event extraction along a timeline. In Proceed-ings of the 27th annual international ACM SIGIR con-ference on Research and development in informationretrieval, pages 425–432. ACM.

G. Erkan and D.R. Radev. 2004. Lexpagerank: Prestigein multi-document text summarization. In Proceed-ings of EMNLP, volume 4.

Thomas Hofmann. 1999. Probabilistic latent semanticanalysis. In Proceedings of the Fifteenth conferenceon Uncertainty in artificial intelligence, pages 289–296. Morgan Kaufmann Publishers Inc.

Giridhar Kumaran and James Allan. 2004. Text clas-sification and named entities for new event detection.In Proceedings of the 27th annual international ACMSIGIR conference on Research and development in in-formation retrieval, pages 297–304. ACM.

Theodoros Lappas, Benjamin Arai, Manolis Platakis,Dimitrios Kotsakos, and Dimitrios Gunopulos. 2009.On burstiness-aware search for document sequences.In Proceedings of the 15th ACM SIGKDD internation-al conference on Knowledge discovery and data min-ing, pages 477–486. ACM.

Peng Li, Jing Jiang, and Yinglin Wang. 2010. Gen-erating templates of entity summaries with an entity-aspect model and pattern mining. In Proceedings ofthe 48th annual meeting of the Association for Com-putational Linguistics, pages 640–649. ACL.

C.Y. Lin and E. Hovy. 2003. Automatic evaluationof summaries using n-gram co-occurrence statistics.In Proceedings of the 2003 Conference of the NorthAmerican Chapter of the Association for Computa-tional Linguistics on Human Language Technology-Volume 1, pages 71–78. ACL.

Chen Lin, Chun Lin, Jingxuan Li, Dingding Wang, YangChen, and Tao Li. 2012. Generating event storylinesfrom microblogs. In Proceedings of the 21st ACM in-ternational conference on Information and knowledgemanagement, pages 175–184. ACM.

Juha Makkonen, Helena Ahonen-Myka, and MarkoSalmenkivi. 2004. Simple semantics in topic detec-tion and tracking. Information Retrieval, 7(3-4):347–368.

Qiaozhu Mei and ChengXiang Zhai. 2005. Discover-ing evolutionary theme patterns from text: an explo-ration of temporal text mining. In Proceedings of theeleventh ACM SIGKDD international conference onKnowledge discovery in data mining, pages 198–207.ACM.

Q. Mei, J. Guo, and D. Radev. 2010. Divrank: the inter-play of prestige and diversity in information networks.

In Proceedings of the 16th ACM SIGKDD internation-al conference on Knowledge discovery and data min-ing, pages 1009–1018. ACM.

Thomas Minka. 2000. Estimating a dirichlet distribu-tion.

Lawrence Page, Sergey Brin, Rajeev Motwani, and Ter-ry Winograd. 1999. The pagerank citation ranking:bringing order to the web.

Yulong Pei, Wenpeng Yin, et al. 2012. Generic multi-document summarization using topic-oriented infor-mation. In PRICAI 2012: Trends in Artificial Intel-ligence, pages 435–446. Springer.

D.R. Radev, H. Jing, M. Stys, and D. Tam. 2004.Centroid-based summarization of multiple documents.Information Processing & Management, 40(6):919–938.

Mauricio GC Resende and Renato F Werneck. 2004. Ahybrid heuristic for the p-median problem. Journal ofheuristics, 10(1):59–88.

Ivan Titov and Ryan McDonald. 2008. Modeling on-line reviews with multi-grain topic models. In Pro-ceedings of the 17th international conference on WorldWide Web, pages 111–120. ACM.

X. Wan. 2008. Document-based hits model for multi-document summarization. PRICAI 2008: Trends inArtificial Intelligence, pages 454–465.

Xuanhui Wang, ChengXiang Zhai, Xiao Hu, and RichardSproat. 2007. Mining correlated bursty topic pat-terns from coordinated text streams. In Proceedingsof the 13th ACM SIGKDD international conferenceon Knowledge discovery and data mining, pages 784–793. ACM.

Xiang Wang, Kai Zhang, Xiaoming Jin, and Dou Shen.2009. Mining common topics from multiple asyn-chronous text streams. In Proceedings of the SecondACM International Conference on Web Search andData Mining, pages 192–201. ACM.

Dingding Wang, Tao Li, and Mitsunori Ogihara. 2012.Generating pictorial storylines via minimum-weightconnected dominating set approximation in multi-viewgraphs. Proceddings of AAAI 2012.

Rui Yan, Liang Kong, Congrui Huang, Xiaojun Wan, X-iaoming Li, and Yan Zhang. 2011a. Timeline gener-ation through evolutionary trans-temporal summariza-tion. In Proceedings of the Conference on EmpiricalMethods in Natural Language Processing, pages 433–443. ACL.

Rui Yan, Xiaojun Wan, Jahna Otterbacher, Liang Kong,Xiaoming Li, and Yan Zhang. 2011b. Evolution-ary timeline summarization: a balanced optimizationframework via iterative substitution. In Proceedingsof the 34th international ACM SIGIR conference onResearch and development in Information Retrieval,pages 745–754. ACM.

735

optimized event storyline generation based on mixture ... · the key challenge to our storyline...

Documents