A Latent Dirichlet Allocation Approach using Mixed Graph of Terms forSentiment Analysis

Mario Casillo †, Fabio Clarizia ‡, Francesco Colace ‡, Massimo De Santo ‡, Marco Lombardi ‡, Francesco Pascale ‡† University of Naples ”Federico II”, Italy

mario.casillo@unina.it‡ DIIn, University of Salerno, Italy

fcolace — desanto — malombardi — fpascale@unisa.it

Abstract

The spread of generic (as Twitter, Facebook orGoogle+) or specialized (as LinkedIn or Viadeo) socialnetworks allows to millions of users to share opinionson different aspects of life every day. Therefore thisinformation is a rich source of data for opinion miningand sentiment analysis. This paper presents a novelapproach to the sentiment analysis based on the LatentDirichlet Allocation (LDA) approach. The proposedmethodology aims to identify a word-based graphicalmodel (we call it a mixed graph of terms) for depictinga positive or negative attitude towards a topic. By theuse of this model it will be possible to automaticallymine from documents positive and negative sentiments.Experimental evaluation, on standard and real datasets,shows that the proposed approach is effective andfurnishes good and reliable results.

1. Introduction

In the 2010 Eric Schmidt, CEO of Google, said:Between the birth of the world and 2003, there werefive exabytes of information. Now, we create fiveexabytes every two days. See why its so painfulto operate in information markets?. Millions ofmessages appear daily thanks to blogs, microblogs,social networks or reviews collector sites. In general,this textual information can be divided in two maincategories: facts and opinions [22]. Facts are objectivestatements while opinions reflect peoples sentimentsabout products, other person and events and areextremely important when someone needs to evaluatethe feelings of other people before taking a decision.Before the wide diffusion of the Internet and Web 2.0,people used to share opinions and recommendationswith traditional approaches: asking friends, talking toexperts and reading documents. The Internet and webmade possible to find out opinions and experiencesfrom people being neither our personal acquaintancesnor well known professional critics. The interest,

that potential customers show in online opinions andreviews about products, is something that vendorsare gradually paying more and more attention to[29]. Companies are interested in what customerssay about their products as politicians are interestedin how different news media are portraying them.Therefore there is a lot of information on the webthat have to be properly managed in order to providevendors with highly valuable network intelligence andsocial intelligence to facilitate the improvement of theirbusiness. In this scenario, a promising approach isthe sentiment analysis: the computational study ofopinions, sentiments and emotions expressed in a text[13]. Its main aim is the identification of the agreementor dis-agreement statements that deal with positive ornegative feelings in comments or reviews. In theliterature, there are many approaches to the sentimentanalysis. A very broad overview of the existing workwas presented in [19]. The authors describe in detailthe main techniques and approaches for an opinionoriented information retrieval. Early work in this areawas focused on determining the semantic orientationof documents. In particular some approaches attemptto learn a positive-negative classifier at a documentlevel. [26] introduces the results of review classificationby considering the algebraic sum of the orientation ofterms as respective of the orientation of the documents.Starting from this approach other techniques have beendeveloped by focusing on some specific tasks as findingthe sentiment of words [27]. Baroni [3] proposed to ranka large list of adjectives according to a subjectivity scoreby employing a small set of manually selected subjectiveadjectives and computing the mutual information ofpairs of adjectives using frequency and co-occurrencefrequency counts on the web. The work of Turney[25] proposes an approach to measure the semanticorientation of a given word based on the strengths ofits association with a set of context insensitive positivewords minus the strengths of its association with a setof negative words. By this approach sentiment lexiconscan be built and a sentiment polarity score can be

Proceedings of the 52nd Hawaii International Conference on System Sciences | 2019

URI: https://hdl.handle.net/10125/59661ISBN: 978-0-9981331-2-6(CC BY-NC-ND 4.0)

Page 2226

assigned to each word [16][9]. Sentiment polarity scoremeans the strengths or degree of sentiment in a definedsentence pattern. Artificial Intelligence and probabilisticapproaches have also been adopted for sentimentmining. In [20] three machine learning approaches(Naive Bayes, Maximum Entropy and Support VectorMachines) has been adopted to label the polarity ofa movie reviews datasets. A promising approach ispresented in [21] where a novel methodology has beenobtained by the combination of rule based classification,supervised learning and machine learning. In [23]a SVM based technique has been introduced forclassifying the sentiment in a collection of documents.In [18], instead, a Naive Bayes classifier is used forthe sentiment classification of tweets corpora. Otherapproaches are inferring the sentiment orientation ofsocial media content and estimate sentiment orientationsof a collection of documents as a text classificationproblem [7]. More in general, sentiment relatedinformation can be encoded within the actual wordsof the sentence through changes in attitudinal shadesof word meaning using suffixes as discussed in [8].This has been investigated in [17] where a lexiconfor sentiment analysis has been obtained. In [28] aprobabilistic approach to sentiment mining is adopted.In particular this paper uses a probabilistic modelcalled Sentiment Probabilistic Latent Semantic Analysis(S-PLSA) in which a review, and more in general adocument, can be considered as generated under theinfluence of a number of hidden sentiment factors [11].The S-PLSA is an extension of the PLSA where itis assumed that there are a set of hidden semanticfactors or aspects in the documents related to documentsand words under a probabilistic framework. In [4] anapproach combining the ontological formalism and amachine learning technique has been introduced. Inparticular the proposed system uses domain ontology toextract the related concepts and attributes starting froma sentence and then labels the sentence itself as positive,negative or neutral by means of the Support VectorMachine (SVM) classifier. In this paper, we investigatethe adoption of a similar approach based on the LatentDirichlet Allocation (LDA). In LDA, each documentmay be viewed as composed by a mixture of varioustopics. This is similar to probabilistic latent semanticanalysis, except that in LDA the topic distribution isassumed to have a Dirichlet prior. By the use of the LDAapproach on a set of documents belonging to a sameknowledge domain, a Mixed Graph of Terms can beautomatically extracted [6] [15] . Such a graph containsa set of weighted word pairs, which we demonstrate tobe discriminative for sentiment classification. The mainreason of such discriminative power is that LDA based

topic modeling is essentially an effective conceptualclustering process and it helps discover semanticallyrich concepts describing the respective sentimentalrelationships. By means of applying these semanticallyrich concepts, that contain more useful relationshipindicators to identify the sentiment from messages, theproposed system can accurately discover more latentrelationships and make less errors in its predictions.The rationale of this paper is the following: section2 discusses the searching the sentiment by the use ofa Mixed Graphs of Terms from a document corpus.The section 3 introduces the proposed approach for thesentiment extraction and the section 4 discusses theexperimental results. Finally, conclusions and furtherworks are discussed.

2. Searching the sentiment by the use ofthe Mixed Graph of Terms

In this paper we explain how a complex structure,that we call a Mixed Graph of Terms (mGT), allows tocapture and represent the information contained in a setof documents that belong to a well-defined knowledgedomain. Such a graph can be automatically extractedfrom a document corpus and can be effectively used asa filter to employ in document classification as well asin sentiment extraction problems. Formally, a MixedGraph of Terms can be defined as a graph g = 〈N,E〉where:

• N = R,W is a finite set of nodes, covered bythe set R = r1, ..., rH whose elements are theaggregate roots and by the setW = w1, ..., wMcontaining the aggregates. Aggregate roots can bedefined as the words whose occurrence is mostimplied from the occurrence of all other wordsin the training corpus. Aggregates are defined asthe words most related to aggregate roots from aprobabilistic point of view.

• E = ERR, ERW is a set of edges, coveredby the set ERR = er1r2, ..., erH−1rH whoseelements are links between aggregate roots andby the set ERW = er1W1, ..., eeHwM whoseelements are links between aggregate roots andaggregates. As better explained further, twoaggregate roots are linked if strongly correlated(in a probabilistic sense):

erirj =

1 ifψij ≥ τ0 otherwise

(1)

Aggregate roots can be also linked to aggregates

Page 2227

if a relevant probabilistic correlation is present:

eriws =

1 ifρij ≥ µi0 otherwise

(2)

Details about mGT building and thresholds τ and µi willbe cleared in the next section. First we show how mGTcan be effectively applied for the sentiment mining fromtexts. The proposed method adopts the Mixed Graphof Terms for building a sentiment detector able to labela document according its sentiment. We propose anarchitecture composed by the following modules:

• Mixed Graph of Terms building module: thismodule builds a mixed graph of terms startingfrom a set of documents belonging to awell-defined knowledge domain and previouslylabeled according the sentiment expressed inthem. In this way the obtained mixed graphof terms contains information about the wordsand their co-occurrences so representing a certainsentiment in a well-defined knowledge domain.As described in section 2 thanks to the LDAapproach such a graph can be obtained by theuse of a set of few documents. In figure 1 themodule architecture and its main functional stepsare depicted. The output of this module is amixed graph of terms representing the documentsand their sentiment. By feeding this modulewith positive or negative training sets, it willbe possible to build mixed graphs of termsfor documents that express positive or negativesentiment in a well-defined domain.

• Sentiment Mining Module: this module extractsthe sentiment from a document thanks to the useof the Mixed Graph of Term as a sentiment filter.The input of this module is a generic document,the mixed graph of terms representing positiveand negative sentiment in a knowledge domainand the output is the sentiment detected in theinput document.

The sentiment extraction is obtained by acomparison between document and the mixed graphof terms according to the algorithm 1. The proposedalgorithm requires the use of an annotated lexicon, asfor example WordNet [2] o ItalWord- Net [1], for theretrieval of synonyms of the words contained in thedocument D and not in the reference mGT. The retrievedsynonyms are added to the vector W and analyzedaccording to the classification strategy. The proposedapproach is effective in an asynchronous sentimentclassification, but can work also in a synchronous way.

Figure 1.Mixed Graph of Terms Building Module.

In figure 2 and figure 3 the synchronous sentiment realtime classificatory architecture is depicted. For realtime working two new modules have been introduced:

Algorithm 1 Sentiment Mining Algorithm

Input: W = [w1, w2, ..., wn] the words that are ina Document D belonging a knowledge domainK; the mixed graph of terms mGT+ and mGT-obtained analyzing documents related to the knowledgedomain K expressing positive and negative sentiment;RW+ = [rw1; rw2, ..., rwt] the aggregator wordsthat are in mGT+; AW+ = [aw1, aw2, ..., awm]the aggregated words that are in mGT+;RW− = [rw1, rw2, ..., rwn] the aggregator wordsthat are in mGT-; AW− = [aw1, aw2, ..., rwp] theaggregated words that are in mGT, L an annotatedlexicon.Output: SentimentD =Positive;Negative;Neutral the sentimentexpressed in the document D.Algorithm Descriptionfp = 0;fn = 0;Determining the synonyms for each word belonging tothe vector Wfor i = 0 ! Length[W] do

WS = WS + Synset[L;W[i]];end forW = W +WS;Mining the sentiment from the documentfor i = 0 ! Length[W] do

for k = 0 ! Length[RW+] doif (RW+[k] == W[i]) then

fp = fp + 2;end if

end for

Page 2228

for k = 0 ! Length[RW+] doif (RW+[k] == W[i]) then

vfn = fn + 2;end if

end forfor k = 0 ! Length[AW+] do

if (AW+[k] == W[i]) thenfp = fp + 1;

end ifend forfor k = 0 ! Length[AW+] do

if (AW+[k] = W[i]) thenfn = fn + 1;

end ifend for

end forDetermining the Sentimentif (fp ¿ fn) then

SentimentD = Positive;else

if (fp ¡ fn) thenSentimentD = Negative;

elseSentimentD = Neutral;

end ifend if

Figure 2.Sentiment Analysis Classification System Architecture.

• Document Grabber. This module aims to collectdocuments from web sources (social networks,blogs and so on). These documents can becollected both for updating the training set andfor their classification according to the sentiment.The training set update is an important feature ofthe proposed approach. In this way, in fact, thevarious mGTs can be continuously updated andimprove their discriminating power introducingnew words and relations and deleting inconsistent

Figure 3.System Architecture for Synchronous Classification.

ones.

• Document Sentiment Classification. The newdocuments inserted into the training set have tobe classified by the support of an expert. Theaim of this module is to provide a user friendlyenvironment for the classification, according totheir sentiment, of the retrieved documents.

Figure 4.Proposed mGT extraction method.

3. Extracting a Mixed Graph of Terms

The aim of this section is to explain how aMixed Graph of Terms, which contains the mostsignificant word pairs, can be extracted from a corpusof documents. The extraction process is shown infigure 4 he input of the system is the set of documentsΩr(d1, ..., dM ) and the output is a vector of weightedword pairs g = w′1, ..., w′|τρ| where τρ is thenumber of pairs and w′n s the weight associated toeach pair (feature) tn = (υi, υj). Such a weightedword pairs structure can be suitably represented as amixed graph g of terms (mGT) figure 5 The mGTis made of several clusters, each containing a set of

Page 2229

Figure 5.mGT graphical representation.

words υs (aggregates) related to an aggregate root τia special word which represents the centroid of thecluster. How aggregate roots are selected will be clearfurther. The weight ρis can measure how a word isrelated to an aggregate root and can be expressed as aprobability: ρis = P (τi|υs) The resulting structure isa subgraph rooted on τi Moreover, aggregate roots canbe linked together building a centroids subgraph. Theweightψij can be considered as the degree of correlationbetween two aggregate roots and can also be expressedas a probability: ψij = P (τi, τj). Given the trainingset Ωr documents, the term extraction procedure isobtained first by computing all the probabilistic relationsbetween words and aggregate roots (ρis and ψij) andthen selecting the right subset of pairs τp from allthe possible ones. A mGT graph g is learnt from acorpus of documents as a result of two important phases:the Relations Learning stage, where graph relationweights are learnt by computing probabilities betweenword pairs (see Fig. 4); the Structure Learning stage,where the shape of an initial mGT graph, composedby all possible aggregate root and word levels, isoptimized by performing an iterative procedure which,given the number of aggregate roots H and the desiredmax number of pairs as constraints, chooses the bestparameter settings τ and µ = (µ1, ..., µH) defined asfollows:

1) τ : the threshold that establishes the number ofaggregate root/aggregate root pairs of the graph.A relationship between the aggregate root υi andaggregate root τj is relevant if ψij ≥ τ

2) µi: the threshold that establishes, for eachaggregate root i, the number of aggregateroot/word pairs of the graph. A relationshipbetween the word υs and the aggregate root τi is

relevant if ρis ≥ µi

3.1. Relations Learning

Since each aggregate root is lexically representedby a word of the vocabulary, we can write ρis =P (τi|υi) = P (υi|υs) and ψij = P (τi|τj) = P (υi|υs).Considering that P (υi, υj) = P (υi|υj)P (υj) all therelations between words result from the computationof the joint or the conditional probability ∀i, j,∈1, ..., |τ | (where |τ | is the size of the vocabulary τwhich contains all the indexed words from the corpus)and P (upsilonj) . An exact calculation of P (upsilonj)and an approximation of the joint, or conditional,probability can be obtained through a smoothed versionof the generative model introduced in [4] called LatentDirichlet Allocation (LDA), which makes use of Gibbssampling [10]. The original theory mainly proposesa semantic representation in which documents arerepresented in terms of a set of probabilistic topics z. Formally, we consider a word um of the documentdm as a random variable on the vocabulary τ andz as a random variable representing a topic between1,...,K. A document dm results from generating eachof its words. To obtain a word, the model considersthree parameters assigned: α, η and the number oftopics K. Given these parameters, the model choosesθm through P (θ|α) ∼ Dirichlet(α), the topic kthrough P (z|θm) ∼ Multinomial(θm) and βk ∼Dirichlet(η).Finally, the distribution of each wordgiven a topic is P (um|z, βz) ∼Multinomial(βz). Theoutput obtained by performing Gibbs sampling on a setof documents Ωr consists of two matrixes:

1) the words-topics matrix that contains |τ | X Kelements representing the probability that a wordυi of the vocabulary is assigned to topic k :P (u = υi|z = k, βk);

2) the topics-documents matrix that contains K XΩr elements representing the probability that atopic k is assigned to some word token within adocument dm : P (z = k|θm).

The probability distribution of a word within a documentdm of the corpus can be then obtained as:

P (um) =

K∑k=1

P (um|z = k, βk)P (z = k|θm) (3)

In the same way, the joint probability between twowords um and ym of a document dm of the corpuscan be obtained by assuming that each pair of words is

Page 2230

represented in terms of a set of topics z and then:

P (um, ym) =

K∑k=1

P (um, ym|z = k, βk)P (z = k|θm)

(4)Note that the exact calculation of Eq. 4 depends onthe exact calculation of P (Um, ym|z = k, βk) thatcannot be directly obtained through LDA. If we assumethat words in a document are conditionally independentgiven a topic, an approximation for Eq. 4 can be writtenas:

P (um, ym) ' (5)

'∑Kk=1 P (um|z = k, βk)P (ym|z = k, βk)P (z =

k|θm)

Moreover, Eq. 3 gives the probability distributionof a word um within a document dm of the corpus.To obtain the probability distribution of a word uindependently of the document we need to sum over theentire corpus:

P (u) =

M∑m=1

P (um)δm (6)

where δm is the prior probability for each document(∑Ωrm=1 δm = 1). In the same way, if we consider the

joint probability distribution of two words u and y, weobtain:

P (u, y) =

M∑m=1

P (um, yυ)δm (7)

Concluding, once we have P(u) and P(u,y) we cancompute P (υi) = P (u = υi) and P (υi, υj) = P (u =υi; y = υj), ∀i, j,∈ 1, ..., |τ | and so the relationslearning can be totally accomplished.

3.2. Structure Learning

Once each ψij and ρis is known ∀i, j, s, aggregateroot and word levels have to be identified in orderto build a starting mGT structure to be optimized asdiscussed later. The first step is to select from the wordsof the indexed corpus a set of aggregate roots r =(r1, ..., rH), which will be the nodes of the centroidssubgraph. Aggregate roots are meant to be the wordswhose occurrence is most implied by the occurrence ofother words of the corpus, so they can be chosen asfollows:

r1 = argmax∏j 6=1

P (υi|υj) (8)

Since relationships’ strenghts between aggregate rootscan be directly obtained from ψij the centroids subgraphcan be easily determined. Note that not all possiblerelationships between aggregate roots are relevant: thethreshold τ can be used as a free parameter foroptimization purposes. As discussed before, severalwords (aggregates) can be related to each aggregateroot, obtaining H aggregates’ subgraphs. The thresholdset µ = (µ1, ..., µH) can be used to select thenumber of relevant pairs for each aggregates’ subgraph.Note that a relationship between the word υs and theaggregate root ri is relevant if ρis ≥ µi, but the valueρis cannot be directly used to express relationships’strenghts between aggregate roots and words. In fact,being ρis conditional probability, it is always biggerthan ψis which is a joint probability. Therefore, oncepairs for the aggregates’ subgraph are selected usingρis relationships’ strenght are represented on the mGTstructure through ψis. Given H and the maximumnumber of pairs as constraints (i.e. fixed by the user),several mGT structure gt can be obtained by varyingthe parameters Λt = (τ, µ)t. As shown in Fig. 4an optimization phase is carried out in order to searchthe set of parameters Λt which produces the best mGTgraph. This process relies on a scoring function and asearching strategy that will be now explained. As wehave previously seen, a gt is a vector of features gt =b1t, ..., b|τsp|t in the space τsp and each documentof the training set Ωr can be represented as a vectordm = (ω1m, ..., ω|τsp|t) in the space τsp. A possiblescoring function is the cosine similarity between thesetwo vectors:

S(gt, dm) =

∑|τsp|n=1 bnt ∗ ωnm√∑|τsp|

n=1 b2nt ∗

√∑|τsp|n=1 ω

2nm

(9)

and thus the optimization procedure would consist insearching for the best set of parameters Λt such that thecosine similarity is maximized ∀dm. Therefore, the bestgt for the set of documents Ωr is the one that producesthe maximum score attainable for each document whenused to rank Ωr documents. Since a score for eachdocument dm is obtained, we have:

St = S(gt, d1), ..., S(gt, d|Ωr|) (10)

where each score depends on the specific set Λt =(τ, µ)t. To compute the best value of Λ we canmaximize the score value for each document, whichmeans that we are looking for the graph which bestdescribes each document of the repository from whichit has been learned. It should be noted that suchan optimization maximizes at the same time all |Ωr|

Page 2231

elements of St. Alternatively, in order to reduce thenumber of the objectives being optimized, we canat the same time maximize the mean value of thescores and minimize their standard deviation, whichturns a multiobjective problem into a two-objective one.Additionally, the latter problem can be reformulated bymeans of a linear combination of its objectives, thusobtaining a single objective function, i.e., Fitness (F),which depends on Λt,

F (Λt) = E|St| − σm|St| (11)

where E is the mean value of all the elements of Stand σm is the standard deviation. By summing up, theparameters learning procedure is represented as follows,

Λ∗ = argmaxtF (Λt) (12)

Since the space of possible solutions could growexponentially, |τsp| ≤ 300 has been considered.Furthermore, the remaining space of possible solutionshas been reduced by applying a clustering method, thatis the K-means algorithm, to all ψij and ρis values, sothat the optimum solution can be exactly obtained afterthe exploration of the entire space.

4. Experimental Results

In order to evaluate the performance of theproposed approach, two experimental phases have beenconducted. The first one has been carried out usinga standard dataset while the second one has beenapplied to ”real life” datasets collected from Twitterand Facebook; results obtained by the proposed methodhave been compared with the others in literature. Thefirst dataset used for the experimentation is the MovieReviews Dataset provided by Pang et al. in [20].This dataset consist of 1000 positive and 1000 negativereviews from the Internet Movie Database. Positivelabels were assigned to reviews that had a rating above3.5 stars and negative labels were assigned to the rest.The first step of the experimental campaign was aimedto find the best size for the training set. For achievingthis task nine training sets have been built selecting ina random way from the 10% to 90% of the positiveand negative comments that are in the full dataset.By the use of these training sets, the positive andnegative mixed Graphs of Terms have been built and thesentiment classification on the remaining comments hasbeen conducted. The process of training sets and mixedgraph of terms building and documents classificationhas been conducted ten times. The obtained results, interms of average accuracy, are depicted in figure 6. Asdepicted in the figure the value of accuracy improveswith the increase of the training set but the change is

very low after the adoption of a training set composedfrom the 50% of comments that are in the dataset. Afterthis phase a comparison with the results obtained on thesame dataset by other approaches that are in literaturehas been conducted (table 1). The proposed approachshows the best results in comparison with the other oneswhen the 50% of dataset is used as training set. Theother approaches usually adopt a larger dataset and inreal cases the training phase could be critical and timeconsuming.

Figure 6.The variation of the accuracy compared to the size of

the training set (on the x-axis 1 means 10set and so on)

Table 1. The accuracy obtained by the various

methods on the standard dataset.Reference Paper Methodology Accuracy

(20) Support Vector Machines 82.90%Naiive Bayes 81.50%

Maximum Entropy 81.00%(12) Support Vector Machines 86,20%(5) Ontology Supported Polarity Mining 72,20%(14) Bayesian Classification with the support of lexicons 81,42%(24) Formal Concept Analysis 77,75%

mGT Approach LDA 88,50%

The classification phase average lasts about a coupleof minutes using a Linux Ubuntu platform running ona 8GB RAM single CPU while the training phase lastsfrom about 5 minute to ten minutes depending from thesize of the training set. As previously said our approachgives effective results also in real time scenarios. Todemonstrate this aspect, an experimental campaign onposts coming from social networks has been conducted.In particular, 2.500 posts from Facebook and 10.000tweets from twitter have been collected. The posts ofFacebook has been collected from an official page ofa well-known mobile phone producer while the tweetshave been collected from the hashtags related to PompeiiArchaeological Park. A group of five experts labeledthe posts and the tweets according their sentiments usinga majority vote rule and deleting the neutral comments(table 2).

The methodologies based on the Naiive Bayes andSupport Vector Machine introduced in the paper [20]

Page 2232

Table 2. Considered datasets.Dataset Source Positive Negative Neutral

Mobile Phone Producer Facebook 864 759 877PompeiTweets Twitter 4783 3498 1719

Figure 7.Pompei Tweets: the variation of the accuracy compared

to the size of the training set (on the x-axis 1 means10% of training set and so on)

and [12] have been implemented and applied to thecollected data in order to compare results with the onesobtained by the proposed method. The first step wasaimed at building the training set. As previously saidfor achieving this task nine training sets have been builtselecting in a random way from the 10% to 90% ofthe positive and negative comments that are in the fulldataset. At the end of the training phase the selectedapproaches has been tested on the on the test setsobtaining the results depicted in figure 7 (dataset PompeiTweets), figure 8 (Mobile Phone Producer) and table 3.

Figure 8.Mobile Phone Producer: the variation of the accuracy

compared to the size of the training set (on the x-axis 1means 10% of training set and so on)

In the case of the dataset coming from twitter, theexperimental results show how the proposed methodoffers the best performance starting from a training setcomposed by the 30% of the collected dataset. Ingeneral the performances start to be interesting with atraining set composed by the 70% of the dataset: it

Table 3. The obtained results.Pompei Tweets Dataset Mobile Phone Producer Dataset

perc. Naiive Bayes SVM mGT Naiive Bayes SVM mGT10% 0.55 0.33 0.51 0.35 0.2 0.2820% 0.6 0.37 0.55 0.43 0.25 0.430% 0.65 0.42 0.59 0.52 0.31 0.5640% 0.72 0.45 0.71 0.58 0.37 0.6550% 0.75 0.59 0.78 0.65 0.59 0.760% 0.78 0.7 0.81 0.71 0.65 0.7670% 0.79 0.74 0.81 0.77 0.7 0.880% 0.8 0.75 0.81 0.8 0.74 0.8390% 0.8 0.77 0.82 0.82 0.79 0.85

is an expected result because the tweets are composedby a short number of words and so systems based onNaive Bayes and mGT has to learn from a great numberof examples. For the SVM we observe acceptableperformances when employing at least the 50% ofthe dataset. The poor results of the SVM, comparedtpo other methods, are due to the difficult to find awell-defined pattern for the correct classification andthis task is very difficult in the case of the tweets (wherethere is not a welldefined structure). In the case of thedataset collected from Facebook, the same approach hasbeen adopted for the classification of the posts. Also inthis case the proposed approach shows the best resultsstarting from a training set composed by the 30% ofthe dataset. In this case the performance of our systemimproved faster than the twitter case because Facebook’sposts contain a greater number of words so that the builtmGTs are more effective.

5. Conclusion

This paper proposes the use of the mixed graph ofterms, obtained by the use of Latent Dirchlet Allocationapproach, as tool for the sentiment classification ofdocuments. The method relies on building complexstructures called mGTs from documents labeledaccording their sentiment. Then, the classification of anew document can be conducted by using the referencemGTs. The proposed method was compared to the mainmethods in literature using standard and real datasets.In both cases the obtained results are better than thoseobtained by other approaches. Further developmentof this approach regards the introduction of annotatedlexicon, as SentiWordnet, for a better evaluation of thewords and the sentence structures.

References

[1] Italwordnet - http://www.ilc.cnr.it/iwndb/iwndb php/.

[2] Wordnet - http://wordnet.princeton.edu/.

[3] Marco Baroni and Stefano Vegnaduzzo. Identifyingsubjective adjectives through web-based mutualinformation. In In Proceedings of the 7th Konferenz zurVerarbeitung Natrlicher Sprache (German Conference

Page 2233

on Natural Language Processing KONVENS04, pages613-619, 2004.

[4] David M. Blei, Andrew Y. Ng, and Michael I.Jordan. Latent dirichlet allocation. J. Mach. Learn. Res.,3:993-1022, March 2003.

[5] Pimwadee Chaovalit and Lina Zhou. Movie reviewmining: a comparison between supervised andunsupervised classification approaches. In Proceedingsof the Proceedings of the 38th Annual HawaiiInternational Conference on System Sciences(HICSS’05) - Track 4 - Volume 04, HICSS ’05,pages 112.3-, Washington, DC, USA, 2005. IEEEComputer Society.

[6] F. Clarizia, F. Colace, M. De Santo, L. Greco, and P.Napoletano. Mixed graph of terms for query expansion.In Intelligent Systems Design and Applications (ISDA),2011 11th International Conference on, pages 581-586,2011.

[7] Richard Colbaugh and Kristin Glass. Estimatingsentiment orientation in social media for intelligencemonitoring and analysis. In Intelligence and SecurityInformatics (ISI), 2010 IEEE International Conferenceon, pages 135-137, may 2010.

[8] Andrea Esuli and Fabrizio Sebastiani. Sentiwordnet: Apublicly available lexical resource for opinion mining.In In Proceedings of the 5th Conference on LanguageResources and Evaluation (LREC06, pages 417-422,2006.

[9] Michael Gamon and Anthony Aue. Automaticidentification of sentiment vocabulary: Exploiting lowassociation with known sentiment terms. In Proceedingsof the ACL Workshop on Feature Engineering forMachine Learning in Natural Language Processing,pages 57-64, Ann Arbor, Michigan, June 2005.Association for Computational Linguistics.

[10] Thomas L. Griffiths, Joshua B. Tenenbaum, andMark Steyvers. Topics in semantic representation.Psychological Review, 114:2007, 2007.

[11] Thomas Hofmann. Probabilistic latent semantic analysis.In In Proc. of Uncertainty in Artificial Intelligence,UAI99, pages 289-296, 1999.

[12] F. Colace, M. De Santo, S. Lemma, M. Lombardi,A. Rossi, A. Santoriello, A. Terribile, and M.Vigorito, ”How to Describe Cultural HeritageResources in the Web 2.0 Era?”, 11th InternationalConference on Signal-Image Technology andInternet-Based Systems (SITIS), pp. 809-815, 2015.DOI:10.1109/SITIS.2015.50

[13] Bing Liu. Sentiment analysis and subjectivity. InHandbook of Natural Language Processing, SecondEdition. Taylor and Francis Group, Boca, 2010.

[14] Prem Melville, Wojciech Gryc, and Richard D.Lawrence. Sentiment analysis of blogs by combininglexical knowledge with text classification. InProceedings of the 15th ACM SIGKDD internationalconference on Knowledge discovery and data mining,KDD ’09, pages 1275-1284, New York, NY, USA, 2009.ACM.

[15] P. Napoletano, F. Colace, M. De Santo, and L.Greco. Text classification using a graph of terms. InComplex, Intelligent and Software Intensive Systems(CISIS), 2012 Sixth International Conference on, pages1030-1035, July.

[16] A. Neviarouskaya, H. Prendinger, and M. Ishizuka.Sentiful: A lexicon for sentiment analysis. AffectiveComputing, IEEE Transactions on, 2(1):22-36, jan.-june2011.

[17] A. Neviarouskaya, H. Prendinger, and M. Ishizuka.Sentiful: A lexicon for sentiment analysis. AffectiveComputing, IEEE Transactions on, 2(1):22 -36, jan.-june2011.

[18] Alexander Pak and Patrick Paroubek. Twitter as acorpus for sentiment analysis and opinion mining. InProceedings of the Seventh conference on InternationalLanguage Resources and Evaluation (LREC’10),Valletta, Malta, May 2010. European LanguageResources Association (ELRA).

[19] Bo Pang and Lillian Lee. Opinion mining and sentimentanalysis. Found. Trends Inf. Retr., 2(1-2):1-135, January2008.

[20] Bo Pang, Lillian Lee, and Shivakumar Vaithyanathan.Thumbs up? sentiment classification using machinelearning techniques. In Proceedings of EMNLP, pages79-86, 2002.

[21] Rudy Prabowo and Mike Thelwall. Sentiment analysis:A combined approach. Journal of Informetrics,3:143-157, 2009.

[22] Fabrizio Sebastiani and Consiglio Nazionale DelleRicerche. Machine learning in automated textcategorization. ACM Computing Surveys, 34:1-47,2002.

[23] Khin Phyu Phyu Shein. Ontology based combinedapproach for sentiment classification. In Proceedings ofthe 3rd International Conference on Communicationsand information technology, CIT’09, pages 112-115,Stevens Point, Wisconsin, USA, 2009. World Scientificand Engineering Academy and Society (WSEAS).

[24] Khin Phyu Phyu Shein. Ontology based combinedapproach for sentiment classification. In Proceedings ofthe 3rd International Conference on Communicationsand information technology, CIT’09, pages 112-115,Stevens Point, Wisconsin, USA, 2009. World Scientificand Engineering Academy and Society (WSEAS).

[25] P. Turney and M. Littman. Unsupervised learningof semantic orientation from a hundred-billion-wordcorpus. Technical report nrc technical report erb-1094,Institute for Information Technology, National ResearchCouncil Canada, 2002.

[26] Peter D. Turney. Thumbs up or thumbs down?: semanticorientation applied to unsupervised classification ofreviews. In Proceedings of the 40th Annual Meetingon Association for Computational Linguistics, ACL’02, pages 417-424, Stroudsburg, PA, USA, 2002.Association for Computational Linguistics.

[27] Theresa Wilson, Janyce Wiebe, and Rebecca Hwa. Justhow mad are you? finding strong and weak opinionclauses. In Proceedings of the 19th national conferenceon Artifical intelligence, AAAI’04, pages 761-767.AAAI Press, 2004.

[28] Xiaohui Yu, Yang Liu, Xiangji Huang, and Aijun An.Mining online reviews for predicting sales performance:A case study in the movie domain. IEEE Trans. onKnowl. and Data Eng., 24(4):720- 734, April 2012.

[29] Vivek Rajput and Sarika Bobde, ”Stock marketprediction using hybrid approach”, InternationalConference on Computing, Communicationand Automation (ICCCA), 2016, DOI:10.1109/CCAA.2016.7813694

Page 2234