STED: Semi-Supervised Targeted-Interest Event Detectionin Twitter

Ting HuaDept. of Computer Science

Virginia Techtingh88@vt.edu

Feng ChenH.J. Heinz III College

Carnegie Mellon Universityfchen1@cmu.edu

Liang ZhaoDept. of Computer Science

Virginia Techliangz8@vt.edu

Chang-Tien LuDept. of Computer Science

Virginia Techctlu@vt.edu

Naren RamakrishnanDept. of Computer Science

Virginia Technaren@vt.edu

ABSTRACTSocial microblogs such as Twitter and Weibo are experiencing anexplosive growth with billions of global users sharing their daily ob-servations and thoughts. Beyond public interests (e.g., sports, mu-sic), microblogs can provide highly detailed information for thoseinterested in public health, homeland security, and financial anal-ysis. However, the language used in Twitter is heavily informal,ungrammatical, and dynamic. Existing data mining algorithms re-quire extensive manually labeling to build and maintain a supervisedsystem. This paper presents STED, a semi-supervised system thathelps users to automatically detect and interactively visualize eventsof a targeted type from twitter, such as crimes, civil unrests, anddisease outbreaks. Our model first applies transfer learning and labelpropagation to automatically generate labeled data, then learns a cus-tomized text classifier based on mini-clustering, and finally appliesfast spatial scan statistics to estimate the locations of events. Wedemonstrate STED’s usage and benefits using twitter data collectedfrom Latin America countries, and show how our system helps todetect and track example events such as civil unrests and crimes.

Categories and Subject DescriptorsH.3.3 [INFORMATION STORAGE AND RETRIEVAL]: Infor-mation Search and Retrieval

General TermsAlgorithms

KeywordsEvent Detection, Text Mining

1. INTRODUCTIONMicroblogs (e.g., Twitter and Weibo) have emerged as a disruptive

platform for people to share their daily activities and sentiments on

Permission to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full cita-tion on the first page. Copyrights for components of this work owned by others thanACM must be honored. Abstracting with credit is permitted. To copy otherwise, or re-publish, to post on servers or to redistribute to lists, requires prior specific permissionand/or a fee. Request permissions from permissions@acm.org.KDD’13, August 11–14, 2013, Chicago, Illinois, USA.Copyright 2013 ACM 978-1-4503-2174-7/13/08 ...$15.00.

ongoing events. The rich up-to-date sensing information allows dis-covering and tracking important events even earlier than news, withimportant applications such as public health and emergency manage-ment. Although identifying events from newspaper reports has beenwell studied, analyzing messages in Twitter requires more sophisti-cated techniques. Twitter messages are irregular, contain misspelledor non-standard acronyms, and are written in informal style. Ad-ditionally, tweets are filled with trivial events discussing daily life.Twitter’s noisy nature challenges traditional text-based event detec-tion methods and therefore specifically designed event detection ap-proaches are needed for Twitter text analysis.

Most previous work on Twitter event detection has focused ongeneral and large-scale (breaking news) events, such as the VirginiaTech shooting and the Southern Califorinia wild fires. Unsupervisedlearning techniques, such as clustering, topic modeling, and burst de-tection, are mainly utilized. However, they have limited capabilitiesto detect small-scale events, such as city-level or even street-levelprotests or strikes. Recently, new attention has been paid to eventdetection of a targeted topic (e.g., civil unrests, disease outbreaks, orcrimes). Supervised learning techniques are primarily applied, suchas support vector machines and random forest classifiers. Althoughthis work can detect small-scale events of the targeted topic, the re-quirements of expensive manual data labeling limits its efficiencyand scalability. How to determine whether a tweet is interest-relatedor not is far more than simple keyword filtering. For example, iftweets related to shooting crimes are required, feedback from Twit-ter for the query word ’shooting’ are motley: tweets like ’2 shot todeath, 1 wounded: A shooting erupted at Mexico City airport’ areindeed related to shooting crime, but tweets like ’Shooting a musicvideo’ in fact have nothing to do with gunfire.

In this demo, we propose a novel approach, semi-supervised targeted-interest event detection (STED), which takes users’ specific interestsas input, retrieves related tweets and summarizes events’ spatial andtemporal features into visualization results. The major contributionsare as follows:

• Automatical label creation and expansion: To avoid bur-densome human efforts required in previous work, we pro-pose a method capable of generating labeled data automatical-ly, which first transfers labels from newspapers to tweets, andfurther expands the initial label subspace using Twitter socialties.

• Customized text classifier for Twitter: Using tweet mini-clusters obtained by graph partitioning, we build a specializedsupport vector machine classifier for tweet analysis.

Figure 1: System Framework of STED

• Enhanced location estimation algorithms: Utilizing tweetsocial ties and fast spatial scan statistics, we propagate geo-labels within location clusters for event separation.

• Visualization and analysis: Provision of event clusters, his-torical statistics, and related-tweets via a friendly interface pro-motes effective and efficient usage for human analysis.

2. FRAMEWORK AND METHODSAs shown in Figure 1, the architecture of STED can be divided

into these parts. Using the Extracting and Label-Generating mod-ules, we transfer labels from news to Twitter to generate initial labeldata. Module Label Propagating utilizes Twitter social features toobtain extended label data. Graph-Partition module clusters initialsingle tweets into mini-tweet-cluster, and then training module builda Support Vector Machine(SVM) text classifier to identify targetedtopic related tweets. Finally, Spatial Scan and Location Propagatingmodules further group target topic related tweets into specific eventsaccording to location.

2.1 Automatical Label Creation and ExpansionIn this step, we first automatically transfer labels from news de-

scriptions to Tweets and further expand the initial label data by uti-lizing Twitter social ties like Retweet(RT), Hashtag(#), and Mention-s(@).

Term Extracting and Label Generating: We first collect domainspecific news descriptions, such as news about crime, from publicmedia. Though news reports are quite different from tweets in struc-ture and expression style, elements that can specify an event remainthe same: Named Entity and Action Word. By using NLTK 1, weextract Named Entity(noun) and Action Words(verb) from news de-scription as candidate query word set for tweets. Given a query set asinput, label generating module investigates Twitter data and selectstweets containing at least one Named Entity and one Action Word aspositive label data.

Label Propagating: Social-ties terms appear between tweets inthe form of Mentions(@), Retweets(RT), and Hashtag(#). Tweet-s sharing common terms are more likely to discuss the same topic.We use social-ties terms to expand initial labeled dataset L obtained.

1http://nltk.org/

First, we identify social-ties terms from labeled tweets, build Term-Tweet heterogeneous network S1, and remove less popular terms. Asshown in Figure 2(a), node degrees are approximately distributed inpower law where most tweets are connected to few terms with highdegree. These terms are expected to be more related to the event,while those low degree terms on the border are trivial. Then, we usethe remaining popular terms as query to retrieve tweets, build Term-Tweet heterogeneous network S2, and filter away terms with low a-bility to denote a specific event. Figure 2(b) illustrates an exampleof S2, a Hashtag-Tweet heterogeneous network, where the core ter-m of the central cluster is hashtag ’#mexico’, surrounded with morenewly found tweets (orange nodes) than initial labeled tweets (bluenodes). These terms should be filtered away by our system, sincethey are popular but shared by too many topics to represent specificinterests. Finally, we build Term-Tweet network with filtered termset connected to new found label tweets S3, as shown in Figure 2(c).Iterate process above, until no new tweet satisfying the condition-s can be found. Through Label Propagating module, we obtain anextended label dataset for further processing.

2.2 Twitter Text ClassificationIn this part, we first apply graph partitioning methods [5] to ob-

tain event-related words groups and generate tweet mini-clusters,and then use support vector machine for text classification.

Graph Partitioning: Given word w, we first build its waveletsignal represented by the following sequence.

fw = [ fw(T1), fw(T2), ..., fw(Tn)] (1)

where fw(Ti) is the TF-IDF score of word w during the period Ti . Inthis paper, to capture daily event emergence, we set duration of Ti tobe one hour and number of segment n to be 24. Then, we computethe auto correlation Aw for each word w and filter away trivial word-s(appearing evenly day by day). From above, we get subset Ψ of rareand note worthy words. Next, we calculate cross-correlation Xi j ofeach word-pair in Ψ and construct a correlation matrix Γ containingall word pairs. This correlation matrix Γ can be viewed as a graphand related-word clustering becomes a graph partition problem: Weapply graph partitioning [4] on correlation matrix Γ to obtain sub-graphs that words within one subgraph are highly similar in form ofhigh cross correlation, while words in different subgraphs have lowcross correlation. Finally, tweet clusters are generated by obtained

(a) Term-Tweet Heterogeneous Network ofLabel Tweets S1.

(b) Hashtag-Tweet Heterogeneous Net-work of Total Tweets Space S2.

(c) Term-Tweet Heterogeneous Network ofNew Found Tweets and Filtered Terms S3.

Figure 2: Big nodes represent terms: Red nodes are hashtags, blue nodes are mentions, and yellow nodes are Retweets. Small nodes denotetweets: blue ones are labeled tweets, orange nodes are newly found tweets from raw data. Edge (i, t) means tweet t contains term i.

word groups: tweet containing at least two items of word group Gican be considered as an item of tweet cluster Ci.

Classifier Training: The most important part for classifier train-ing is feature selection. Words appear less than threshold ζ are firstfiltered out. Next, we calculate TF-IDF scores for words and filterout trivial words such as ’people’, ’love’, which appear more fre-quently in total Twitter space than labelled tweets space. Besides,to avoid overfitting problem, most words from the Named Entity setE should be removed, since they enjoy high TF-IDF scores and willpotentially be assigned heavy weights in the SVM classifier, but onlyrepresent one specific event.

Figure 3: Example of Tweet Location Clusters. Red nodes denotethe highest density of tweets of locations.

2.3 Location EstimationTo estimate event locations, we first identify spatial clusters using

fast spatial scan methods [3] However, only 2% of tweets containsuch geographic information. To make best use of the minority oftweets with geo-labels as well as the majority without labels, we

further propagate geo-labels within each cluster to amplify spatialsignal capabilities.

Spatial Scan: Geo-locations of tweets about a certain event arelikely around the event’s occurring location. We apply spatial scanstatistics to detect significant spatial clusters, as shown in Figure 3.Specifically, we aggregate the count of event related tweets in citylevel and define the base of each city as the total count of the originaltweets. Then we apply fast subset scan [3] to identify a set of Hcandidate clusters with the largest Kulldorff’s statistics [2], which isdefined as

Kr = (Ca −Cr) lg(Ca −Cr

Ba −Br)+Cr lg(

Cr

Br)−Ca lg(

Ca

Ba) (2)

where Ca and Ba refer to the total count and base in the country,respectively; and Cr and Br refer to the count and base in the spatialregion r, which is a set of neighbor cities. The empirical p-value ofeach candidate cluster is estimated by random permutation testing,and the clusters with empirical p-values smaller than a threshold η(e.g.η = 0.05), are returned as significant clusters. The parameter His usually set greater than the maximum number of potential clustersthat may exist, and the insignificant clusters can be filtered out laterby randomization testing.

Location Label Propagating: Within each cluster, we further la-bel tweets that lack geo information using social ties. Tweets containcommon terms such as hashtag and mention are more likely to occurin the same location. We first compute a score ωi j =

mi j

Miby tweets

with geo-labels to denote the relativity of term i and location j, wheremi j is number of tweets contain term i as well as location j, and Miis the count of tweets contain term i.

lt = maxj∈ϕ

{1− ∏i∈φt

(1−ωi j)} (3)

Then, using Equation 3, we estimate location lt of unlabelled tweett, which contains a set φt of terms. We first compute the relativitybetween tweet t and each location j from location set ϕ, and thenpick up the biggest value as this tweet’s estimated location.

3. DEMONSTRATIONWe showcase STED system using Twitter data from Latin Amer-

ica as the example application. The considered database is morethan 400GB in size, from June, 2012 to Jan, 2013. With applicationto civil unrest event detection, STED achieved 72% in precision of

Figure 4: Interface of STED system

Figure 5: Historical Analysis Screenshot

and 74% in recall, with a lead time of 2.42 days ahead of traditionalmedia (e.g., news sources). We implemented the STED interface inPython which provides users with the following capilities:

• Map visualization of targeted-interest event clusters in city-level.

• Detailed information about each event cluster, including relat-ed tweets and word cloud summary.

• Graphical analyses of historical statistics, including spatial com-parison among regions and temporal trend within one region.

Figure 4 shows a screenshot of the STED interface. With STED, auser can search for events pertaining to their specific interests and an-alyze their spatial and temporal features. A targeted-interest includestime, location, topic and keywords. Users are allowed to choose dateand topic as well as typing in keywords, in the right part of inter-face. As an ongoing project, we have applied our method to detectinterests of crime and unrest. As shown in the screenshot, the user-s’ interest is in detecting events about type ’Civil Unrest’ in country’Mexico’, with keyword ’protest’, from date ’2012-07-06’ to ’2012-07-07’. After users click on the ’Search’ button, STED will returncorresponding event clusters of targeted-interest, shown as balloons.By clicking on one of the balloons, users can find detailed informa-tion of corresponding event from left part of the interface: tweets

ranked by their relativity to users’ interests and word cloud eventsummary denoting terms’ relative importance. System feedback ofgiven interest shown in the screenshot reveals that there was a march(Spanish word ’marcha’ in word cloud) held by YoSoy132 to protestpresident election results, which is also reported by public media [1].

It is also possible to study targeted interested events spatially andtemporally, by using the historical statistics analysis interface. Giv-en a city and historical period range, users can track interest-relatedevent trend of this city. In the bottom of Figure 5, we show interest-related event summary of given city ’Mexico City’, from historicaldate ’2012,July’ to ’2012,December’. Users can also compare spa-tial features of interested event. In the upper part of Figure 5, welist top-10 cities in given country ’Mexico’ with restrict to historicalevent number.

4. ACKNOWLEDGEMENT AND ANNOUNCEThis work is supported by the Intelligence Advanced Research

Projects Activity(IARPA) via Department of Interior National Busi-ness Center (DoI/NBC) contract number D12PC000337. US Gov-ernment is authorized to reproduce and distribute reprints for Gov-ernmental purposes notwithstanding any copyright annotation there-on. Disclaimer: The views and conclusions contained herein arethose of the authors and should not be interpreted as necessarily rep-resenting the official policies or endorsements, either expressed orimplied, of IARPA, DoI/NBC, or the US Government.

5. REFERENCES[1] L. Diaz, G. Stargardter, I. Grillo, and P. Cooney. Protesters

march against mexico’s president election.http://www.reuters.com/article/2012/07/07/us-mexico-election-protest-idUSBRE8660I820120707.

[2] M. Kulldorff. Spatial scan statistics: models, calculations, andapplications. pages 303–322. Springer, 1999.

[3] D. B. Neill. Fast subset scan for spatial pattern detection. WileyOnline Library, 2012.

[4] M. E. Newman. Fast algorithm for detecting communitystructure in networks. volume 69, page 066133. APS, 2004.

[5] J. Weng and B. Lee. Event detection in twitter. In ICWSM’11,volume 3, 2011.