1

On Argument Bundles in the Web ofExperiences

Xavier Ferrer1,2 and Enric Plaza1

1Artificial Intelligence Research Institute (IIIA),Spanish National Research Council (CSIC), CampusU.A.B., Bellaterra, Catalonia (Spain)2Universitat Autonoma de Barcelona, Bellaterra,Catalonia (Spain){xferrer,enric}@iiia.csic.es

In this paper we focus on a particular interesting type of webuser-generated content: people’s experiences. We extend ourprevious work on aspect extraction and sentiment analysisand propose a novel approach to creating a vocabulary of ba-sic level concepts with the appropriate granularity to charac-terize a set of products. This concept vocabulary is createdby analyzing the usage of the aspects over a set of reviews,and allows us to find those features with a clear positive andnegative polarity to create the bundles of arguments. The ar-gument bundles allow us to define a concept-wise satisfac-tion degree of a user query over a set of bundles using the no-tion of fuzzy implication, allowing the reuse of experiencesof other people to the needs a specific user.Keywords: Web of Experiences, sentiment analysis, argu-ments, aspect extraction, basic level concepts

1. Introduction

Our work is developed in the framework of the Webof Experiences [10]. This framework proposed to en-large the paradigm of Case-based Reasoning (CBR),based on solving new problems by learning from pastexperiences, and includes all forms of experiencesabout the real world expressed as user-contributed con-tent on the web. The final goal is to reuse this collectiveexperience in helping new people (the “users”) in tak-ing a more informed decision according to their prefer-ences, which can be different from the preferences ofthe individuals who have expressed their experienceson the web. The overall goal of the Web of Experi-ences (WoE) approach is constructing the relationshipbetween these two points: from numerous but diverseindividual experiences to a specific and personal user

request, and this paper presents a complete instantia-tion of this relationship.

In this WoE approach we focus on praxis and usage,and we want to analyze how individuals express certainexperiences about their daily life; in this paper we willfocus on the usage of digital cameras. A main goal is todiscover the vocabulary they use, which do not need tobe the same as the classical feature list describing thedifferent aspects of a camera (e.g. “has 45 AF points”),in order to elucidate the main pros and cons of eachcamera according to the user reviews.

To this end, we analyze textual reviews of user ex-periences with digital cameras and identify the set ofaspects the individuals utilize and the polarity of thesentiment words associated with them [15,16]. Aspectsare grouped in basic level concepts (BLC) [11], cre-ating a new concept vocabulary, to overcome the dis-parate granularity of the extracted aspects. Those con-cepts with a strong positive or negative polarity overthe set of reviews of a camera are considered the prosand cons, respectively, of that camera.

We call a bundle of arguments the set of main prosand cons of a camera. We take this approach, alreadyenvisioned in [10], because the pros and cons allowus to acquire and reuse the knowledge for other peo-ple with diverse individual preferences. To support thisreuse, we introduce the notion of query satisfaction bya bundle of arguments modeled using the notion offuzzy implication, in which a query expresses the newuser individual preferences (e.g. a travel photographerwill strongly prefer a camera with long battery life).

The paper is organized as follows: Section 2 de-scribes aspect extraction and the discovery of basiclevel concepts from user reviews. Section 3 presentsthree different types of argument bundles and and Sec-tion 4 defines the user query satisfaction degree. Evalu-ation results are presented in Section 5, related researchin Section 6, and conclusions in Section 7.

2. Aspects and Basic Level Concepts

In our previous work on social recommender sys-tems we harnessed knowledge from product reviews,

2

and characterized every product by a set of aspect-sentiment pairs extracted from its reviews [15]. Basedon these characterizations, we ranked and selected themost useful aspects for recommendation [16]. How-ever, even after identifying the most useful aspects forrecommendation, we still processed synonymous as-pects and aspects referencing the same concept (suchas sensor and cmos) as different aspects, adding noiseto the recommendation process.

In this work, we use a similar approach to [15] inorder to extract the set of salient aspects used to defineimportant characteristics of photographic digital cam-eras. We call aspect vocabulary A the set of extractedaspects from the reviews of a corpus K , and P the setof products described in K reviews. However, insteadof characterizing the products of the corpus directly bythe aspect vocabulary, we group them in basic levelconcepts (BLC). According to Rosch et al. [11], ba-sic level concepts are those that strike a tradeoff be-tween two conflicting principles of conceptualization:inclusiveness and discrimination. Rosch et al. foundthat there is a level of inclusiveness that is optimal forhuman beings in terms of providing optimum cogni-tive economy. This level of inclusiveness is called thebasic level, and concepts or categories at this level arecalled basic-level concepts. In Figure 1, the basic levelis found at the middle level of the hierarchy, and thebasic level concepts are ‘picture’, ‘battery’ and ‘lens’.The concept ‘image’ is the superordinate concept for‘picture’ (as are ‘energy storage’ and ‘optics’ to theirrespective BLCs), and it is more general: a ‘picture’ isconsidered here an image made using a camera, andused in common parlance more often than the technicalterm ‘photograph’; moreover, ‘image’ is considered anabstract term, a physical likeness or representation ofa person, animal, or thing, including painted canvases,and sculptures. ‘RAW picture’ is a subordinate conceptof the BLC ‘picture’, and therefore it is more specific.Notice that the BLC subordinate concepts are usuallyformed by combining the BLC concept name with an-other word that specifies the nature of the given BLC.For instance, ‘RAW picture’, or other concepts such as‘gray-scale picture’ and ‘color picture’ are also subor-dinate concepts of the BLC ‘picture’, because they aremore specific and can be abstracted to have the sameintended meaning as ‘picture’.

Research in the field of identifying basic level con-cepts is mostly oriented to improve the task of wordsense disambiguation. For instance, the class-basedword sense disambiguation [7] approach requires tomanually identify words in a corpus as belonging to

battery lenspicture

RAW picture

battery life

zoom lens

opticsenergy storageimage

Inclusiveness+

-

Specificity

+

-

Fig. 1. Examples of several levels of categorization on digital cam-eras.

one semantic class, that is interpreted as one BLC.Once the corpus is marked, several supervised classi-fiers are trained to assign the proper semantic class toeach ambiguous word. In our approach, we discover acollection of basic level concepts in an unsupervisedway from the review corpus, where each BLC assem-bles a set of aspects that, according to our analysis, areused in a similar way by the reviewers. As we show insection 2.1, we estimate this similarity by taking intoaccount semantic similarity and evaluating the coher-ence/incoherence of the sentiment values of the aspectsassembled in a given BLC. Synonymy is a special caseof aspects being used in a way that is indistinguishablefor our purposes.

Consider, for instance, these three aspects in A : pic-ture, pic and jpeg. One may surmise people using thosewords in reviews are in fact referring to the same ba-sic level concept, i.e. the picture obtained by my digi-tal camera. Thus, we could consider that different re-views in the corpus using those words are referring tothe same BLC, because they have the same intendedmeaning (they are indistinguishable for our purposes).

We will now present a method to discover a collec-tion of BLCs in order to to create a concept vocabularyC . The creation of a collection of basic level conceptsconsist of three steps: 1) identifying synonymous as-pects, 2) building a hierarchical clustering using a newsimilarity measure among aspects, and 3) creating aconcept vocabulary C of basic level concepts from thehierarchical clustering.

2.1. Hierarchical Clustering of Aspects

The first step is, considering a corpus K with an as-pect vocabulary A , to identify which aspects in A aresynonyms according to WordNet (a lexical database ofEnglish). To do so, we first need to identify the Word-

3

Net synsets of every aspect and identify if they are re-lated. The five more frequent aspects of A are manu-ally mapped to the corresponding WordNet synset re-lated to digital photography. For each of the remain-ing non-mapped aspects of A , we first search the corre-sponding WordNet synsets (if any) associated with thenoun word form, and then disambiguate it by identi-fying the synset with the shortest aggregated WordNetPath Distance [8] with respect to the five previouslymanually selected synsets. The aspects that have a syn-onymy relation among them are assembled into aspectgroups {G1, . . . ,Gm}. Aspects with no synonyms forma singleton group.

Next, we iteratively cluster the most similar groupsof aspects and create a dendrogram. To cluster the as-pect groups we use an unsupervised bottom-up hierar-chical clustering algorithm that takes the most similarpair of groups at each stage and puts them together ina higher level group.

We will now define similarity measures over aspectsand over groups. The similarity measure between twoaspects ai and a j is:

SimA(ai,a j) = a ·G(ai,a j)+b ·F(ai,a j)+ g ·L(ai,a j)

where a, b and g are weighting parameters in [0,1]such that a+b+g= 1. The values of SimA are in [0,1].Functions G, F, and L estimate aspect similarity bythree different criteria, bounded in [0,1].

Semantic Similarity (G): Compares two aspect co-occurrence vectors to estimate their similarity[12]. The co-occurrents of an aspect ai are theother aspects that have a first order co-occurrencewith ai within a sentence window. By passingthis window over the entire corpus we obtain, foreach aspect ai, a vector of its co-occurrent as-pects. The vector of co-occurrent aspects repre-sent the global context of the aspect with respectto the other aspects in A , and we use it to estimatethe semantic similarity between aspects. That is tosay, we consider that two aspects are semanticallyclose if the co-occurrence vectors of both aspects,with respect to all other aspects in A , are simi-lar. Figure 2 shows the co-occurrent relations be-tween some of the top most frequent aspects in theaspect vocabulary of DSLR cameras, where thesize of the nodes represent the frequency of occur-rence of the aspects over the reviews of the DSLRcorpus (bigger nodes are more frequent), and theedges the strength of the co-occurrence between

lensvideo

picture

shoot

quality

image

focus

photofeature

iso

light

point

body

price

photography

shooting

sensorbutton

shutter

noise

zoom

Fig. 2. Co-occurrences between pairs of the most frequent aspects ofthe aspect vocabulary of DSLR cameras.

aspects (the wider the edge the more times thoseaspects co-occur in the same sentence). Both theaspect vocabularies and the corpora will be intro-duced later in Section 5.

String Similarity (F): Uses the Jaro-Winkler distance[14] to estimate the string similarity between twoaspects. This string similarity compares the char-acters of two strings giving more importance tothe left-most characters of the words (to boost as-pects with similar lemmas).

Taxonomic Similarity (L): PhotoDict is a small tax-onomy of camera-related terms, where similar-ity is measured as the shortest path betweentwo terms. We automatically generated PhotoDictfrom a camera related website [4].

From aspect similarity we define the similarity SimGbetween two groups of aspects Gn and Gm:

SimG(Gn,Gm) =1

|Gn||Gm|

|Gn|

Âi=1

|Gm|

Âj=1

SimA(ai,a j)

There is a special treatment of compound nouns inclustering. Since compound nouns are formed by twoor more words (e.g. image quality), we group themwith the most frequent aspect in the compound.

The result of the hierarchical clustering is a dendro-gram (or clustering tree) of aspects; Figure 3 showsa portion of the resulting dendrogram of the DSLRaspect vocabulary (see Section 5), considering only arepresentative subset.

4

Fig. 3. Representative portion of the dendrogram for the DSLR cameras, from which the concept vocabulary for DSLR cameras is created.

Since hierarchical clustering gives multiple parti-tions over the initial elements at different levels, wehave to select a single partition over A to create a con-cept vocabulary C .

2.2. Concept Discovery

We are interested in selecting a partition from thehierarchical clustering dendrogram such that that theparts constitute basic level concepts (BLC) of digitalcameras. This selection should be guided by the usageof the aspects occurring in a corpus of reviews K . Theselected partition will become our concept vocabularyC by assigning every aspect in A to a concept in C .Therefore, every concept in C is formed by a set ofaspects that are, in their usage, coherent around a basiclevel concept.

We will consider that an aspect group G is a goodcandidate for being a BLC when, for all aspects in G,the average polarity of the sentences related to eachaspect cohere with respect to each product.

To select the best partition, we cut the dendrogram atdifferent levels. Then, for each partition P, we analyzethe coherence degree of the sentiment values in eachaspect group in P. If the sentiments of the aspects of agroup G cohere into a clear positive, negative, or neu-tral value, we consider G a potential basic level con-cept. For instance, let picture, photo and image be threeaspects in a group. If those three aspects are used bypeople to refer to the same concept (‘picture obtainedby my digital camera’), then the sentiment values ofthose aspects with respect to the reviews of each prod-uct should have a high coherence degree.

Thus, we define the Partition Ranking score R(P) ofa partition P by aggregating the sentiment coherenceof the aspect groups in P, as follows:

R(P) =1|P| Â

Gi2PIS(Gi)

where Gi 2 P and |P| is the number of aspect groupsin P. The higher R(P), the better the partition P.

IS(G) estimates the coherence degree of an aspectgroup G as the average sentiment similarity among theaspects in G using the average cosine similarity:

IS(G) =1

|G| · (|G|�1)

|G|

Âi=1

|G|

Âj=1, j 6=i

cos(D(ai),D(a j))

where cos(D(ai),D(a j)) is the cosine of the angle be-tween aspect vectors D(ai) and D(a j). Finally, an as-pect vector D(a) is:

D(a) = (Sav(pi,a))i21,...,|P |

where p1, . . . , p|P | is the set of products P described ina review corpus K and Sav(pi,a)2 [0,1] is the normal-ized sentiment average over the set of sentences fromthe reviews of product pi in which aspect a occurs.Therefore, the aspect vector D(a) contains the aver-age polarity of aspect a, considering all sentences fromuser reviews over P . Notice that, by comparing two as-pect vectors D(ai) and D(a j), we can assess the polar-ity coherence between two aspects ai and a j over theset of products of a corpus K .

The concept vocabulary C we took corresponds tothe partition with greater R(P) (in our experiments weconsidered only partitions with 35 to 45 groups, a rea-sonable concept vocabulary size).

5

Concept Aspects in Concept

Storage storage, capacity, sd card, sdhc card, cf card

Buttonlag, shutter release, shutter speed, shutter lag,

shutter button, button, button layoutBattery battery, battery life, battery pack

Table 1Three example basic level concepts and their group of aspects.

3. Bundle of Arguments

In this section we characterize the a product by abundle of arguments that use concepts in C .

Let p 2 P be a product described in the reviews of acorpus K , and C 2C a concept. We define Occ(p,C) asthe set of sentences from the reviews that describe indi-vidual experiences with product p in which any of theaspects that form the concept C appears. By analyzingthe sentiment values of Occ(p,C), we infer whetherthe people’s experiences about a concept C of a prod-uct p have a positive or negative overall sentiment. Ifthe overall polarity of the occurrences of C over the re-views of p is positive, we consider C to be a pro argu-ment (Arg+) of p. If the overall polarity is negative, weconsider C a con argument (Arg�) of p. Finally, if theoverall polarity is not clearly positive or negative, weconsider C a moot argument (Arg0). An argument:

Arg = hp,C,si

is formed by a product p 2 P , a concept C 2 C and anaggregated sentiment s calculated by aggregating thesentiment values of Occ(p,C). Pro, con, and moot ar-guments are defined as follows:

Arg =

8><

>:

Arg+ if Arg.s > dArg� if Arg.s <�dArg /0 if �d Arg.s d

(1)

where d is a threshold that determines when argumentsare considered clearly pros or cons —and moot other-wise. We will see later that d depends on the bundletype (dG,ds,dF ).

By considering the pro, con and moot arguments of aproduct p over the set of all concepts in C , we obtain acharacterization about what people like or dislike aboutp. The union of all these arguments form the bundle ofarguments B of a product p:

B(p) =[

C2CArghp,C,si

In this work we consider three different methods tocreate a bundle of arguments: Gini (BG), Agreement(Bs), and Cardinality (BF ) bundles. Each bundle type(BG, Bs and BF ) is built by a different sentiment ag-gregation measure. As we will see later, the parameterD defines as moot those arguments with a very smallOcc(p,C).

3.1. Gini Bundle of Arguments (BG)

An argument in BG has the form hp,C,SG(p,C)i,where the polarity value SG is calculated using the av-erage sentiment Sav(p,C) together with the Gini Co-efficient [17] to penalize it according to the degree ofdispersion of sentiment values:

SG(p,C) =

8><

>:0

if |Occ(p,C)|< Dor �dG < S(p,C)< dG,

S(p,C) otherwise.

where S(p,C) = Sav(p,C)(1�Gini(p,C)).Notice that, when |Occ(p,C)|< D, we consider that

we do not have enough reviews of product p with con-cept C and we assign a neutral sentiment value. Sim-ilarly, when �dG < Sav(p,C) · (1�Gini(p,C)) < dG,we consider that the polarity is not strong enough todefine an argument as a pro or a con, and we assign aneutral sentiment value (recall Equation 1). dG is set to0.1 in the experiments.

3.2. Agreement Bundle of Arguments (Bs)

Let Dev(p,C) be the standard deviation of the sen-timent values of Occ(p,C). The agreement sentimentmeasure Ss(p,C) is the sentiment average of the sen-timent values of the sentences in Occ(p,C), for thoseconcepts whose Dev(p,C) < dmax. This measure usestwo threshold parameters dmax and ds. First, dmax spec-ifies the maximum acceptable standard deviation overthe distribution of sentiment values in Occ(p,C): whenDev(p,C)> dmax we consider that we have no groundsfor an informed decision on the overall polarity of Cwith respect to product p. Second, ds specifies thethreshold for an argument sentiment value to be con-sidered pro, con, or moot argument (see Equation 1).

An argument in Bs has the form hp,C,Ss(p,C)i,where Ss is defined as follows:

Ss(p,C) =

8><

>:0

if Dev(p,C)> dmax

or |Occ(p,C)|< D,Sav(p,C) otherwise.

6

Fig. 4. Sentiment value distribution (a) button concept and (b) lens concept for Pentax K-5. Notice that values have a higher degree of dispersionin (a) than in (b). Furthermore, sentiment values for lens concept in Pentax K-5 (b) are clearly positive, with an average sentiment s = 0.362.

Parameter ds is set to 0.1 in the experiments. Similarlyas before, when |Occ(p,C)| < D we consider that wedo not have enough reviews of product p with conceptC and we assign a neutral sentiment value.

Figure 4 presents the sentiment value distribution oftwo arguments of Pentax K-5, button (a) and lens (b).The button argument of the Pentax K-5 has a sentimentvalue deviation s = 0.542, showing a high dispersionof sentiment values for concept button among the re-views of Pentax K-5. Since the deviation of the senti-ment values of button is higher than dmax, we have noclear overall polarity. On the other hand, the deviationof the sentiment values of lens is lower than dmax andhas a positive average sentiment (0.235 > ds). There-fore, concept lens is considered part of a pro argumentwith respect to Pentax K-5.

3.3. Cardinality Bundle of Arguments (BF )

The cardinality bundle is created by comparing thenumber of positive (O+) versus negative (O�) occur-rences of a concept C in Occ(p,C).

O+(p,C) = |{x 2 Occ(p,C)|s(C,x)> 0}|

O�(p,C) = |{x 2 Occ(p,C)|s(C,x)< 0}|

where s(C,x) is the sentiment value in [�1,1] of con-cept C in sentence x 2 Occ(p,C). The comparison ofpositive versus negative number of occurrences of con-cept C in the reviews of product p is the functionO(p,C) 2 [�1,1]:

O(p,C) =

✓2 · O+(p,C)

O+(p,C)+O�(p,C)

◆�1

Thus, an argument in BF has the form hp,C,SF(p,C)iwhere SF is:

Fig. 5. Gini Argument Bundle BG of the Canon EOS Rebel T4i cam-era. Pros are in green and above and cons in red and below. Lettersize corresponds to the strength of the argument sentiment, largermeans stronger polarity in sentiment value.

SF(p,C) =

8><

>:0

if O(p,C) = 0or |Occ(p,C)|< D,

O(p,C) otherwise.

Notice that SF(p,C) takes values on (0,1] if O+ > O�,and in [�1,0) if O+ <O�. Also, when |Occ(p,C)|<Dwe consider that we do not have enough occurrencesof product’s p concept C to make an informed deci-sion, and we assign a neutral value to the sentiment ofthe argument. In the experiments we use dF = 0 (re-call Equation 1) as the threshold that determines if acardinality argument is a pro, con or moot.

Figure 5 shows the pro and con arguments of theGini bundle of arguments (BG) of the Canon EOSRebel T4i (moot arguments are not represented). Eachword represents an argument: pros in green and above,and cons in red and below.

As a final step in order to create a collection of bun-dles (BG, Bs, BF ) for each product in P , we first need,considering the whole P in a corpus K , to rescalethe sentiment values of the arguments that form the

7

bundles of the collection of bundles in a way that themost positive argument sentiment about a concept hasa sentiment 1, and the most negative a sentiment -1.We rescale the rest of the sentiment values accordingly(see [4] for more details). This way, considering a col-lection of product bundles of a given type, the productwith the best sentiment over a concept has a sentimentvalue of 1. When all arguments of a bundle B(p) arerescaled we call it a normalized bundle B(p).

4. User Query over Product Bundles

A user query defines the preferences of a user ex-pressed using the concept vocabulary C . Since not allpreferences are equally important for the user, everypreference over a concept has a utility value. Given theproducts P characterized with the normalized bundlesof arguments B(p), we need to determine which p 2 Phas a higher level of query satisfaction.

We define a user query Q = {(Cj,U(Cj))} j=1,...,kwith k |C |. Each concept utility pair (Cj,U(Cj)) ex-presses a preference of user over a concept Cj and itsstrength with a utility degree U(Cj) 2 [0.5,1].

For instance in a query Q= {(lens,0.9),(video,0.6)},the user prefers high quality lens and video, althoughthe quality of the lens is more important than that of thevideo. Furthermore, lens and video are more importantthan any other arguments characterizing the camera.

We will now define the Degree of Query Satisfac-tion, DS(Q,B)), that determines the degree to whicha normalized bundle B satisfies a user query Q, usingthe framework of fuzzy logic. Since t-norms and impli-cations in fuzzy logic are defined in the interval [0,1][5], we need to rescale the sentiment values of all argu-ments that form all normalized product bundles from [-1,1] to [0,1]. We do so by applying the linear mappingbs = s+1

2 . For example, consider a normalized argumenthp, lens, 0.83i 2 B(p), the sentiment of the rescaledargument will be bs = 0.915, and the resulting rescaledargument is hp, lens, 0.915i 2 bB(p); the neutral value0 in [-1,1] is mapped to the neutral value 0.5 in [0,1].

The degree of query satisfaction is defined by ag-gregating the degrees to which an argument, with re-spect to concept C, satisfies a user preference with re-spect to the same concept C. Therefore, we first de-fine a concept-wise satisfaction degree, using the no-tion of fuzzy implication associated to the t-norm prod-uct ()⌦). The fuzzy implication U(Cj))⌦ bs j modelsthis notion of degree of satisfaction: if the sentimentbs j of an argument related with concept Cj is higher

Q1 Preferences (C1, 0.7) (C2, 0.6) DS

bBF (D7100) 0.75 1.00bBF (EOS70D) 0.97 0.50U(Cj) for bBF (D7100) 1.00 1.00 1.00

U(Cj) for bBF (EOS70D) 1.00 0.83 0.83

Q2 Preferences (C1, 0.7) (C2, 0.6) (C3, 0.9) DS

bBF (D7100) 0.75 1.00 0.64bBF (EOS70D) 0.97 0.50 1.00U(Cj) for bBF (D7100) 1.00 1.00 0.72 0.72U(Cj) for bBF (EOS70D) 1.00 0.83 1.00 0.83

Table 2Degree of satisfaction of two cameras for each individual preferenceand the overall DS for the query Q1 and the Q2, where C1 = pic-ture, C2 = resolution, C3 = video, and DS is the degree of QuerySatisfaction.

than the user preference U(Cj), then the user prefer-ence is satisfied (and satisfaction degree is 1). On theother hand, if the sentiment bs j provided by an argumentis lower than the user preference U(Cj), then the userpreference is satisfied to a degree lower than 1:

U(Cj))⌦ bs j =

(1 if U(Cj) bs j,

bs jU(Cj)

otherwise.(2)

where bs j is the rescaled sentiment value of argumenthp,Cj, bs ji and bs j

U(Cj)is the satisfaction degree.

Given a query Q with k preferences we can now in-fer k concept-wise satisfaction degrees with respect toa bundle bB(p). We need now to aggregate these k satis-faction degrees into an overall degree of bundle satis-faction (DS(Q,bB(p))). We do so using the conjunctionof these resulting concept-wise satisfaction degrees.Since conjunction in fuzzy logic are represented by t-norms, we use the product t-norm (in consonance withthe product implication used in Eq. 2):

DS(Q,bB(p)) =k

’j=1

(U(Cj))⌦ bs j)

where bs j is the rescaled sentiment value of argumenthp,Cj, bs ji of the argument bundle bB(p), and bB is arescaled argument bundle (either bBG, bBs or bBF ).

Table 2 shows the degree of satisfaction of two userqueries Q1 and Q2 against the cardinality bundles oftwo cameras: Nikon D7100 and Canon EOS70D (sen-timent values are rescaled). The first query is createdby a user who likes to go hiking and is looking fora camera to capture landscape and nature while valu-

8

Fig. 6. Value-triangle of user preferences for the three concepts pic-ture, resolution and video compared with the value-triangles of bun-dle bBF (EOS70D) and bundle bBF (D7100).

ing fine detail. Assume her query is Q1 = {(picture,0.7),(resolution, 0.6)} because she wants a camerawith good image quality and resolution. Table 2 showson the first two rows the argument sentiment values ofthe two cameras corresponding to the concepts appear-ing in the query. The second two rows show the satis-faction degree of the two cameras for each preferenceand the third row overall DS for the query. Notice thatsatisfaction is 1 when the argument sentiment value ishigher than the query’s utility value for that concept.

The second half of Table 2 shows another query,Q2 = {(picture, 0.7),(resolution, 0.6),(video, 0.9)}.Q2 is created by a user that, besides hiking, also lovesrecording video. Figure 6 shows a comparison be-tween the user preferences regarding concepts ‘pic-ture’, ‘video’ and ‘resolution’, and the sentiment ofthe corresponding arguments of the two camera bun-dles, bBF(D7100) and bBF(EOS70D). Now, according touser reviews, Canon EOS70D has an outstanding videoquality (1.0), while Nikon D7100 has an average qual-ity video (0.64). Because of this newly added prefer-ence now the higher ranking camera for Q2 is CanonEOS70D instead of Nikon D7100, the best rankingcamera for Q1. This is clear when we observe Figure6, where EOS70D value-triangle is closer to the userpreferences value-triangle.

5. Evaluation

We evaluate the argument bundles of products fromthree different digital camera categories extracted fromAmazon: Digital SLR, Compact, and Point & Shootcameras. For this purpose we compare the bundles ofarguments we create in the three categories with a set

of bundles created from DPReview.com, a renownedwebsite specialized in digital cameras. Moreover, weare keen to study if there are significant differences be-tween the sets of pros and cons depending the threetypes of bundles of arguments, BG, Bs and BF , and forthe three camera categories. We also assess the impactthat the number of reviews of each product has overthe quality of their argument bundles. Furthermore, weevaluate the precision and recall of the product bun-dles by comparing them with the expert evaluations ofproducts presented in DPReview. Finally, we present aranking strategy for product bundles and compare therankings of products obtained with each bundle type(BG, Bs, BF ) with two external product rankings (thoseof DPReview and Amazon).

This section is structured as follows: Section 5.1 in-troduces the three camera corpora and the correspond-ing aspect vocabularies used to evaluate this experi-ment: The DSLR corpus KD, the Compact corpus KD,and the Point & Shoot KD. Then, in Section 5.2, wecreate three concept vocabularies (CD, CC and CP) forthe tree corpora (as described in Section 2). Section5.3 explains how we create and compare the bundlesof arguments of the cameras in the corpora KD, KCand KP. Section 5.4 evaluates the bundles of argumentsof the DSLR cameras by comparing them with thoseof DPReview. Finally, in Section 5.5 we compare therankings of products obtained with each bundle typewith the rankings extracted from DPReview and Ama-zon.com.

5.1. Review Corpora and Aspect Vocabularies

During September 2015, we extracted more than100,000 reviews from Amazon.com corresponding to2,264 digital cameras from three different categories:Digital SLR, Compact System Cameras, and Point &Shoot. We filtered out those products that were olderthan January 1st 2010 and had less than 15 differ-ent user-generated reviews. Then, we united all syn-onymous products leaving us data for 102 productsin the DSLR category, 95 in Compact category, and599 products in Point & Shoot category. Finally, wegrouped the resulting product reviews in three corpora:KD for DSLR cameras, KC for Compact cameras, andKP for Point & Shoot cameras. Each corpus is formedby a set of product-reviews pairs (pi, Rev(pi)), wherepi is a digital camera and Rev(pi) is the set of reviewsabout camera pi. Table 3 shows the quantity of prod-ucts and reviews on each corpus. Notice that the num-

9

Category KD KC KP

No. of Products 102 95 599No. of Reviews 7,552 6,334 84,138Avg. Reviews/Product 74.03 66.67 140.46

Table 3The three corpora: DSLR (KD), Compact (KC), and P&S (KP).

Concept Vocabularies #Concepts #AspectsCD 41 225CC 39 197CP 39 179

Table 4Number of concepts and aspects for each concept vocabulary.

ber of products (and reviews) in KP is much larger thanthose in KD and KC.

For each of the three corpora of digital cameras,we created a corresponding aspect vocabulary, as de-scribed in our previous work on social recommendersystems [15] (see also Section 2). Thus, we createdthree distinct aspect vocabularies AD, AC, and AP,corresponding to the corpora KD, KC, and KP. Thethree aspect vocabularies consist of set of salient as-pects used to define important characteristics of photo-graphic digital cameras. Moreover, as we show in [4],the three aspect vocabularies contain different sets ofaspects.

5.2. Creation of the Concept Vocabularies

For each aspect vocabulary (AD, AC and AP), wecreate a hierarchical clustering dendrogram as ex-plained in Section 2.2 (one for each camera cate-gory). Then, we select the partition with highest R(P)from each dendrogram, removing those groups Gi 2 Pwhose |Occ(P ,Gi)| 100, and only considering par-titions with 35 to 45 groups, a reasonable concept vo-cabulary size for our purposes. The selected partitionP consists of a collection of groups of aspects, that areconsidered the collection of basic level concept (BLC)for that corpus. These sets of BLCs form the conceptvocabularies, CD, CC and CP, for each camera category.Each vocabulary determines the set of concepts that weuse to interpret the reviews from each corpus.

Table 4 shows the quantity of concepts and aspectsthat form the three concept vocabularies: CD (with 41concepts), CC (with 39 concepts), and CP (with 40 con-cepts). When creating the concept vocabularies, wediscarded those aspect groups from the selected parti-tion that had less than 100 occurrences in the reviews

Top 10 in CD Top 10 in CC Top 10 in CP

picture image picturelens lens zoom

video price videoscreen video pricefocus focus batterybutton zoom screen

iso button pricephotography screen focus

price battery buttonbattery sensor flash

Table 5Top 10 most frequent concepts of CD, CC , and CP.

of a corpus. We interpret the low number of occur-rence of the aspects within those concepts as an indica-tor that they are not deemed important by people whendescribing their experiences with digital cameras.

Table 5 shows the top 10 most frequent conceptsin CD, CC, and CP, where the name of a concept cor-responds to the most frequent aspect included in thatconcept. These 10 more frequent concepts are simi-lar in the three camera corpora, with a few exceptionssuch as concept ‘zoom’, and ‘iso’ (deemed importantfor DSLR cameras, but not for Compact and Point &Shoot cameras). However, the concepts with lower fre-quency are less similar among the vocabularies. Thedifference comes not only from the name of the con-cept but from the aspects grouped under a concept indifferent vocabularies. For instance DSLR and Com-pact differentiate some picture-related concepts suchas ‘iso’, ‘noise’ or ‘resolution’, while in Point & Shootthey are conflated in the concept ‘picture’ [4]. A simi-lar situation is observed in concept ‘button’: this con-cept groups aspects ‘button shutter’, ‘button’ and ‘shut-ter’ in the three concept vocabularies CD, CC and CP.However, the aspects ‘button layout’ and ‘release shut-ter’ are only found in concept ‘button’ of CD, and notin concept ‘button’ of CC and CP; while aspects ‘menubutton’, ‘lag’ and ‘shutter lag’ are only found in con-cept ‘button’ of CC. Furthermore, some concepts onlyexist in one of the three vocabularies, e.g. ‘waterproof’is only present in CP.

5.3. Comparing Argument Bundles Types

For each product of the corpora KD, KC and KP, wecreate three argument bundle types (BG, Bs and BF ).As described in Section 3, the argument bundles of theproducts of a corpus Kx are expressed with the cor-

10

GiniBundle BG

AgreementBundle Bs

CardinalityBundle BF

DSLR (KD)Avg # pros 11.82 15.61 16.06Avg # cons 0.59 4.08 3.19Avg # moots 28.59 21.31 21.75

Compact (KC)Avg # pros 9.42 12.70 13.16Avg # cons 0.47 3.66 2.90Avg # moots 29.11 22.64 22.94

Point & Shoot (KP)Avg # pros 9.64 18.49 19.01Avg # cons 0.82 6.59 5.58Avg # moots 28.54 13.92 14.41

Table 6Average number of pro, con and moot arguments for the three bundletypes BG, Bs, and BF for KD, KC , and KP.

responding concept vocabulary Cx. So, the argumentbundles of the products in KD are created using theDSLR concept vocabulary CD.

In this section we study the differences between thethree bundle types BG, Bs and BF for the products ofeach camera corpus KD, KC, and KP. Since the cri-teria to establish an argument as pro, con, or mootvaries between the three bundle types, the quantity ofpros, cons, and moot arguments obtained by each bun-dle type may differ. Table 6 presents a comparison be-tween the average quantity of pro, con and moot ar-guments of each bundle type for DSLR cameras KD,Compact cameras KC, and Point & Shoot cameras KP.Notice that the values presented in the table are aver-ages over all products of the same corpus. As we willshow later, products with more reviews usually havemore pros and cons —and much less moots.

In Table 6, the Agreement (Bs) and Cardinality(BF ) bundles have a similar average number of prosand cons, while Gini (BG) bundles have slightly lowerquantity of pros and cons for the three camera corpora(DSLR, Compact and Point & Shoot). The Gini av-erage tends to move the argument sentiment value to-wards 0 when there is dispersion in the distribution ofsentiment values, and thus more arguments tend to bemoots. The highest number of pros in a bundle type isobtained with the cardinality bundle BF of the Point &Shoot corpus KP, while the least number of pros is ob-tained by the Gini bundle BG of the Compact corpusKC.

On the other hand, the quantity of pros is higher thanthe quantity of cons for all bundle types (BG, Bs, andBF ) for the three corpora. Either the SmartSA senti-

ment analysis system is biased towards positive senti-ments, or the reviews of our three corpora contain morepositive than negative sentences referencing the con-cepts of the concept vocabularies. Notice that the quan-tity of pro and con arguments is directly related to theaverage quantity of reviews per product, presented inTable 3. For instance, the bundles of arguments of thePoint & Shoot corpus KP, with an average quantity ofreviews per product of 140.46, contain more pro andcon arguments than the bundles of the other two cor-pora KC and KP, with an average quantity of reviewsper product of 74.03 and 66.67, respectively.

Next we analyze which concepts are found in thepro arguments of the three bundle types for each prod-uct. Figure 7 shows, for each product in the horizon-tal axis, the quantity of concepts in pro arguments thatare present in 1, 2 or 3 bundle types. The left verticalaxis (#Concepts) shows that most pros (almost 8 outof 10) are present in 2 or 3 bundle types of a product,a good indicator of the consistency of our approach.This means that a pro argument concept in a BG is alsolikely to be present in a pro argument in Bs, or BF ,or both. The right vertical axis (#Pro concepts occur-rences) shows the number of concept occurrences inthe reviews of each product, displayed as a triangle inFig. 7. The number of pro arguments in a product bun-dle is directly related to the number of concept occur-rences in the reviews of that product. Similar results areobtained when analyzing concepts in con arguments.

5.4. Bundle of Arguments Evaluation

To evaluate the quality of bundles, we compared theproduct pros and cons textual descriptions we found onthe DPReview website with the bundles of argumentsof the 15 products of the DSLR cameras KD with thehighest number of reviews. The DPReview pros andcons of a product are in two separate lists with itemtexts such as ‘good detail and color in JPEGs at baseISO (pro)’ or ‘buggy Live View / Movie Mode (con)’.In order to compare the DPReview pro and con itemswith our bundles of arguments, we first manually iden-tify the concepts, from our DSLR vocabulary CD, ref-erenced in each item text. For instance, we considerthat the DPReview sentence ‘good detail and color inJPEGs at base ISO’ refers positively to the concepts‘jpeg’, ‘color’ and ‘picture’, whilst ‘buggy Live View /Movie Mode’ refers negatively to concepts ‘live view’and ‘video’. Those sentences from DPReview that didnot clearly refer to a concept in CD were ignored. Bygrouping the vocabulary concepts present in the DPRe-

11

Fig. 7. Quantity of pros shared between the three bundles of arguments BG, Bs and BF of the top 50 products with more reviews of DSLR corpusKD, together with the number of occurrences of the pro concepts in the reviews of the product.

view pro and con items of a product, we create thesets of DPReview pro and con arguments (Prosd p andConsd p), but without associating any numeric valueof sentiment —only the positive or negative nature ofthe polarity expressed in the text. We compare thoseDPReview Prosd p and Consd p sets with the pros andcons of the three different bundles of arguments of eachproduct —without taking into account the sentimentvalues, only whether the concept is in a pro or in a con.

Table 7 presents the average precision, recall and F2-score between the sets of pros and cons of the threebundle types of the 15 DSLR cameras with more re-views and those of DPReview. We use the F2-score toweight recall higher than precision, since we are keento study whether the three different bundle types iden-tify as pros and cons the same concepts listed in DPRe-view. Furthermore, we analyze the percentage of con-tradictions, which are those concepts selected as prosin our bundles of arguments but considered cons inDPReview or vice versa. A low rate of contradictionsis a good indicator of the quality of the bundles if wetake DPReview as a standard for comparison.

The argument bundle type that performs best for thepro arguments of the selected DSLR products shownin Table 7 is the cardinality bundle BF , with an av-erage recall of 0.822 and an F2-score of 0.733. Thismeans that the 82.2% of the arguments listed as prosof product a p in DPReview also form part of the prosof the cardinality bundle BF(p). On the other hand,

Precision Recall F2-score Contr

ProsBG 0.567 0.644 0.627 0.004

Bs 0.506 0.761 0.691 0.135BF 0.513 0.822 0.733 0.065

ConsBG 0.333 0.046 0.056 0.046Bs 0.285 0.558 0.468 0.132BF 0.388 0.488 0.464 0.165

Table 7Measure on precision, recall, F2-score and contradictions betweenpros and cons of bundles BG, Bs and BF , of DSLR products fromcorpus KD, with respect to DPReview pros and cons.

the sets of cons of all three bundles of arguments per-form poorly. The reason is the difference in granular-ity between our concept vocabulary and those conceptsused in DPReview. For us, the granularity level is givenby our concept vocabulary, while DPReview sentencesaddress concepts that are at different levels of granu-larity. Furthermore, the granularity of DPReview sen-tences varies whether the sentence is a pro or a con.Pro sentences in DPReview tend to be more general:‘camera buttons and dials are useful and easily config-urable’, while con sentences tend to be more specific:‘the video dial is not easily accessible’. Although forus both sentences reference the same concept (‘button’in this example), the DPReview pro sentence addressesa more general view of the buttons of the camera thanthe con sentence.

12

Furthermore, the precision values of the 3 bundletypes are lower than 0.6, suggesting that the sets ofpros of the bundles of arguments are richer in conceptscompared to the lists of DPReview. This is not surpris-ing, since the sets of DPReview pros and cons are notexhaustive but a short list of the concepts that stand outfrom their point of view. The average quantity of proarguments in a bundle is 12-14, while the average proset size of DPReview identified arguments is 7-9.

Finally, notice the number of contradictions betweenthe bundles of arguments and the DPReview sets islow. Nevertheless, we are interested in studying whichconcepts occur more often in contradictions.

The most common contradictions between the bun-dles and the set of pro and con extracted from DPRe-view for the 15 selected products are: ‘battery’ (10),‘viewfinder’ (5), ‘recording’ (5) and ‘button’ (3). InDPReview, ‘battery’ is often selected as a pro, howeverit is usually present in con arguments in our bundles.That is because in the reviews people usually complainabout the battery of a camera, while they do not seemto express positive opinions on cameras with a goodbattery (it would seem it is taken as a given). Other fre-quent contradictions are ‘viewfinder’, ‘recording’ and‘button’. This is because in DPReview those are com-monly selected as cons for having suboptimal behaviorin certain types of situations (e.g. ‘the video dial is noteasily accessible’) while the overall opinions about therest of the buttons are positive. Therefore, our bundleswill capture this average higher granularity sentimentof ‘button’. Similar situations are observed for ‘record-ing’ and ‘viewfinder’ concepts.

5.5. Bundle Evaluation by Product Ranking

In this section we are interested in comparing thebundles of arguments with the camera descriptions ofDPReview photography website. We have seen that thesets of pro and con arguments between the argumentsof the DSLR products and DPReview characterizationsare similar in Table 7. Now we want to evaluate howgood our experiential characterizations of products are,compared to those created by DPReview experts.

To do so, we define the function F(B(pi),B(p j)),that estimates the degree in which a normalized bundleB(pi) is better than another normalized bundle B(p j):

F(B(pi),B(p j)) =1

2|C |

|C |

Âk=1

sik � s j

k

where sik and s j

k are the sentiment values of argumentshpi,Ck,si

ki and hp j,Ck,sjki in the normalized bundles of

pi and p j. F is the average of these differences, a valuein [�1,1]. If the value of F(B(pi),B(p j)) is in (0,1],it means that B(pi) is better than B(p j). If the valueof F(B(pi),B(p j)) is in [�1,0), it means that B(pi) isworse than B(p j).

Using F, we create five rankings with the productsof each camera corpus: one for each normalized bun-dle type BG,Bs and BF , a DPReview ranking based onthe DPReview overall score, and an Amazon rankingbased on each product star rating. At the end, we havefive different rankings for DSLR cameras (BG rank, Bsrank, BF rank, DPReview rank and Amazon rank), fivecamera rankings for Compact cameras, and five cam-era rankings for Point & Shoot cameras.

In case two or more products of the same cameracategory had the same DPReview score, such as Olym-pus E620 and Nikon D3100, both cameras with a scoreof 72 out of 100, we only kept the product with mostreviews, in this example the Nikon D3100. This left uswith 10 different DSLR cameras, 10 Compact cameras,and 10 Point & Shoot cameras for the rankings.

Let us now compare the five DSLR rankings. Thetop 3 products for the BG ranking are Nikon D7100,Pentax K-5 and SonySLT A-55. The top 3 products forBs are Nikon D7100, SonySLT A-99 and SonySLT A-55, and the top 3 ranked products for BF are NikonD7100, SonySLT A-99 and Pentax K-5. Notice thatNikon D7100 is the top product in the three bun-dle types. Nikon D7100 is also 1st (with a scoreof 85 points) in the DPReview ranking, followed bySonySLT A-99 and Pentax K-5. The top products ofCompact and Point & Shoot camera categories are alsosimilar for the 3 bundle types and DPReview.

Table 8 shows the Spearman rank correlation ofthe 3 rankings of bundle types with the DPReviewscore ranking and the Amazon star ranking, for DSLR,Compact and Point & Shoot cameras. We added a6th random ranking to facilitate comparison. The ran-dom ranking correlation was obtained by averaging theSpearman correlations of 10.000 randomly generatedproduct rankings with DPReview ranking and Ama-zon ranking. The ‘Avg. DPReview Ranking’ of Table 8shows the average Spearman rank correlation betweenthe DPReview ranking and the Amazon ranking of thethree camera categories.

The results show that, for the DSLR camera cate-gory, the BG ranking has the highest Spearman correla-tion with DPReview ranking (correlation of 0.80), fol-lowed by the cardinality ranking BF (correlation with

13

DPReview Rank Amazon RankDSLR Spearman Rank Correlation

BG Ranking 0.80 0.39Bs Ranking 0.70 0.62BF Ranking 0.76 0.61

Compact Spearman Rank CorrelationBG Ranking 0.65 0.54Bs Ranking 0.62 0.55BF Ranking 0.68 0.57

Point & Shoot Spearman Rank CorrelationBG Ranking 0.57 0.33Bs Ranking 0.86 0.32BF Ranking 0.83 0.16

Avg. Random Ranking 0.27 0.27Avg. DPReview Ranking 1 0.23

Table 8Spearman rank correlation between the top 10 DSLR, Compact andPoint & Shoot bundle rankings with DPReview product ranking andAmazon star ratings ranking.

DPReview of 0.76). For Compact cameras, BF rankingobtained the highest Spearman correlation with DPRe-view ranking (correlation of 0.68). Finally, for Point& Shoot cameras, the bundle ranking that obtained thebest correlation with DPReview ranking was Bs (cor-relation of 0.86), closely followed by BF (correlationof 0.83). These values tell us that there is a very strongcorrelation between DPReview and the rankings of ourthree bundle types, a good indicator of the high qual-ity of the bundles. This is specially true with the rank-ings created from the cardinality bundles BF , whichobtained an average Spearman correlation of 0.76 be-tween the DSLR, Compact and Point & Shoot camerasand the corresponding DPReview rankings. The corre-lations for Bs and BG rankings are also strong, beingnotably higher than the random ranking correlations.

On the other hand, notice that the ranking correla-tions between the bundle rankings and Amazon starrankings are in comparison lower than the averageSpearman correlation between the bundle rankings andthe DPReview ranking. In fact, the average Spear-man correlation between the three bundle rankings andthe Amazon ranking is around 0.40, showing there isno strong similarity between the star-rating rankingand the rankings of the bundles acquired from the re-views. Furthermore, the Amazon star-based rankingdoes not correlate with the DPReview score rankingeither (Spearman correlation of 0.23), indicating thatthere exists a notable difference between the star rat-ing ranking of Amazon and the DPReview ranking. Infact, the Amazon ranking has a higher average corre-

lation with the random ranking (0.27) than with Point& Shoot BF ranking (0.16). These results seem to in-dicate that two people expressing similar argumentsabout a product can give different star-rating values (asan overall score). Nevertheless, the fact is that Ama-zon’s star rating seems unsuitable to test the quality ofthe argument bundles.

6. Related Work

There are numerous applications that gather knowl-edge from user reviews, usually oriented to help otherusers make more informed decisions in the area of rec-ommendation systems and CBR. The most commonapproach consists in characterizing a set of productsby considering product aspects (also called features)mentioned in the reviews [1,2]. In this process, the setof aspects selected to characterize a product togetherwith the sentiment analysis of the sentences have a cru-cial role in the final recommendation [13,3,6]. A re-lated work on creating BLC is [7], but they manuallytag a corpus with the classes (concepts) to which wordsbelong, and then use supervised learning —while wediscover the BLCs in an unsupervised way.

Another focus is identifying the sets of aspects withhigher positive/negative polarity to give insights intothe reason why items have been chosen [9]. Those ap-proaches need previously to group the aspects to re-duce the granularity in order to provide useful recom-mendations, often solved by clustering aspects usingbackground knowledge to simplify the process. Ourapproach is different in the sense that we create basiclevel concepts [11] by exploring the usage of the as-pects among the user reviews in an unsupervised way.

Using these basic level concepts, we build the bun-dles of arguments by identifying the pro and con con-cepts over the set of reviews of a product. Finally,we define a concept-wise satisfaction degree of a userquery over a set of bundles using the notion of fuzzyimplication [5]. This bundles both characterize themain pros and cons and supports reusing experiencesof other people to the needs a specific user.

7. Conclusions and Future Work

In this paper we extend our previous work on aspectextraction and sentiment analysis and propose an un-supervised method to create a vocabulary of basic levelconcepts with the appropriate granularity to character-

14

ize a set of cameras. This concept vocabulary is use-ful to practically reuse other people’s experiences withdigital cameras because it abstracts the concrete termsused in the reviews as given by the aspect extractionapproach. By analyzing the usage of the concepts overthe reviews of each product, we find those concept oc-currences that have a clearly positive or negative po-larity and create their argument bundles. We presentthree different types of argument bundles, based on dif-ferent aggregation criteria, each one defining the prosand cons of a product. The argument bundles allowus to define a satisfaction degree, interpreted in fuzzylogic and modeled with a fuzzy implication operator,between products and a user query.

An evaluation of the three types of argument bun-dles is performed and compared with the expert de-scriptions of the DPReview website, showing that thebundles of arguments identify pros and cons very simi-lar to those used in DPReview. Moreover, the cardinal-ity bundle ranking proved to correlate with the overallDPReview score ranking over the subset of the mostfrequent products, while Amazon star rating rankingdoes not correlate with either of them.

The characterization of products by means of thebundles of arguments and BLC is promising. We haveobserved that the quality of a product bundle is relatedto the quantity of reviews of that product: the productswith more reviews have a richer vocabulary of pro andcon arguments, while products with fewer reviews hadmore moots. This can be due to two reasons that opennew lines for future work. First, improving the detec-tion of aspects (for instance, considering also 3-gramaspects) could improve the argument bundles of thoseproducts with fewer reviews. Second, improving thesentiment analysis of reviews by developing a domainspecific sentiment dictionary for digital cameras couldenhance the accuracy of the arguments’ sentiment.

Acknowledgements

This research has been partially supported by NA-SAID (CSIC Intramural 201550E022). We thank LluısGodo and Pere Garcıa for their insightful comments.

References

[1] L. Chen, G. Chen, and F. Wang. Recommender systems basedon user reviews: the state of the art. User Modeling and User-Adapted Interaction, 25(2):99–154, 2015.

[2] R. Dong, M. Schaal, M. O’Mahony, K. McCarthy, andB. Smyth. Mining features and sentiment from review experi-ences. In Case-Based Reasoning Research and Development,pages 59–73, 2013.

[3] R. Dong, M. Schaal, M. O’Mahony, K. McCarthy, andB. Smyth. Opinionated product recommendation. In Case-Based Reasoning Research and Development, volume 7969 ofLecture Notes in Computer Science, pages 44–58. Springer,2013.

[4] X. Ferrer. Concept Discovery and Argument Bundles in theWeb of Experiences. PhD thesis, Universitat Autonoma deBarcelona, 2017.

[5] P. Hajek, L. Godo, and F. Esteva. A complete many-valuedlogic with product-conjunction. Archive for mathematicallogic, 35(3):191–208, 1996.

[6] M. Hu and B. Liu. Mining opinion features in customer re-views. In Proc. of the National Conf. on Artificial Intelligence,pages 755–760, 2004.

[7] R. Izquierdo, A. Suarez, and G. Rigau. Word vs. class-basedword sense disambiguation. J. Artif. Intell. Res. (JAIR), 54:83–122, 2015.

[8] L. Meng, R. Huang, and J. Gu. A review of semantic similar-ity measures in wordnet. Int. Journal of Hybrid InformationTechnology, 6(1):1–12, 2013.

[9] K. Muhammad, A. Lawlor, R. Rafter, and B. Smyth. Greatexplanations: Opinionated explanations for recommendations.In Case-Based Reasoning Research and Development, pages244–258. Springer, 2015.

[10] E. Plaza. Semantics and experience in the future web. In Ad-vances in Case-Based Reasoning: 9th European Conference,ECCBR 2008, volume 5239 of Lecture Notes in Artificial Intel-ligence, pages 44–58. Springer Verlag, 2008.

[11] E. Rosch. Human categorization. Studies in cross-cultural psy-chology, 1:1–49, 1977.

[12] S. Sani, N. Wiratunga, S. Massie, and R. Lothian. Term similar-ity and weighting framework for text representation. In Case-Based Reasoning Research and Development, pages 304–318.Springer, 2011.

[13] P. Turney. Thumbs up or thumbs down?: semantic orienta-tion applied to unsupervised classification of reviews. In Proc.Annual Meeting on Association for Computational Linguistics,pages 417–424, 2002.

[14] W. Winkler. The state of record linkage and current researchproblems. In Statistical Research Division, US Census Bureau.Citeseer, 1999.

[15] Y. Y.Chen, X. Ferrer, N. Wiratunga, and E. Plaza. Sentimentand preference guided social recommendation. In Case-BasedReasoning Research and Development, pages 79–94. Springer,2014.

[16] Y. Y.Chen, X. Ferrer, N. Wiratunga, and E. Plaza. Aspect selec-tion for social recommender systems. In Case-Based Reason-ing Research and Development, pages 60–72. Springer, 2015.

[17] S. Yitzhaki. Relative deprivation and the Gini coefficient. TheQuarterly Journal of Economics, pages 321–324, 1979.