analysis, design and implementation of a multi-criteria recommender system based on aspect...
TRANSCRIPT
Analysis, design and implementation of a Multi-
Criteria Recommender System based on Aspect
Extraction and Sentiment Analysis techniques
Instructors: Student:
Prof. Giovanni Semeraro Davide GIANNICO
Dott. Marco de Gemmis
Department of Computer ScienceMaster of Computer Science (Msc)
3
Need: necessity of the user to be supported during
complex decisional processes
Tendency: development of on-line platforms which provide
recommendations to the user and where the community
expresses the own opinion for a item class
Some stats 315M unique views per month & 200M reviews(TripAdvisor, 2014);
168M unique views per month & 35M reviews (Amazon, 2013);
139M unique views per month & 67M reviews (Yelp, 2014).
Scenery
Opportunity
4
Take advantage of the reviews informative power
incorporating such information in the recommendation
process.
Concept: make «value» from raw data.
Recommender Systems
5
Recommender Systems (RecSys) are decision support and
information filtering tools
The main goal is helping users which access to a data
source for discovering information or items that could be
interesting for them
Recently, RecSys area has focused on Multi-Criteria
RecSys[AMK11]
[AMK11] G. Adomavicius, N. Manouselis, Y. Kwon. ‘Multi-Criteria Recommender Systems’. In:
Recommender Systems Handbook, pp 769-803, Springer US, 2011
Multi-Criteria Recommender Systems
6
Techniques which provide recommendations to the user,
modeling the utility concept espressed by the user for an
item as a ratings vectore on several criteria
Focus: multi-criteria item-based collaborative filtering
Item-to-item similarity matrix
1.Item-to-item matrix calculus
2. Rating prediction
multi-static-criteria ratings
Input:
multi-criteria ratings
Output: recommendations
𝑠𝑖𝑚 Itemi, Itemj= …
…
IB Collaborative Filtering RecSys
Discussed Problem
7
State-of-the-art Approach: default criteria taxonomies on which users can rate
Limits and challenges:
Proposed criteria are not exhaustive respect to userpreferences need to «customize» the criteria
Vagueness of the criteria need of fine-grained criteria
Proposed Approach
8
Automatic identification of criteria from the reviews
using Aspect Extraction techniques
Sentiment Analysis for associating a preference level to
the extracted criteria (implicit rating)
Extension of multi-criteria item-based recommendation
algorithms exploiting the information extracted
Aspect Extraction e Sentiment Analysis
Step[V11]
9
Aspect
Extraction
Sentiment
Analysis
review<Main Aspect,Sub-Aspect,Rilevance,
Sentiment Score,holder>
<location,station, 0.8, 0.9, >
<staff, courtesy, 0.9, -0.6, >
<…, …, …, …, >
Opinion Retrieval
Engine (ORE) system
[V11] V. Giuliani. ‘Analisi, progettazione e sviluppo di un framework per l'opinion retrieval basato su tecniche
automatiche di aspect extraction’. In Tesi di Laurea in Informatica Magistrale, 2011
(multi-ORE-criteria ratings)
Algorithm A: #multi-ORE-criteria (1/2)
10
review dataset
multi-ORE-criteria ratings
Aspect
Extraction
Sentiment
Analysis
ORE System
Multi-Criteria Item-Based Collaborative Filtering RecSys which
uses the extracted ratings from the reviews for calculating the
item-to-item similarity matrix
Algorithm A: #multi-ORE-criteria (2/2)
11
multi-ORE-ratings Item-item similarity
matrix
(Multi-dimensional Pearson
coefficient)
𝑠𝑖𝑚 𝐼𝑡𝑒𝑚𝑖, Item𝑗 =
𝑢∈U,c=1..k 𝑅𝑢,𝑐,𝑖 − 𝑅 𝑖)(𝑅𝑢,𝑐,𝑖 − 𝑅 𝑗
𝑢∈U,c=1…k 𝑅𝑢,𝑐,𝑖 − 𝑅 𝑖
2𝑅𝑢,𝑐,𝑖 − 𝑅 𝑗
2
(Rating Prediction)
𝑃𝑢,𝑖 = 𝑎𝑙𝑙 𝑠𝑖𝑚𝑖𝑙𝑎𝑟 𝑖𝑡𝑒𝑚𝑠 𝑗(𝑠𝑗 ∗ 𝑅𝑢)
𝑛( , , 0.8 )
( , , 0.6 )
predicted ratings
Algorithm B: Multi-criteria Aspect-based Recommender
system based on sentimenT Analysis (#MARTA) (1/2)
12
Item #1Aspect
Extraction
Sentiment
Analysis
Position
Food
Staff-courtesy
Breakfast-buffet
<position, *,0.9, ..>(review#1)
<staff,courtesy, -0.6, ..>(review#2)<position, *,0.7, ..>(review#2)
(Item#1 summary)
If <aspect> 𝜖 summary
add <aspect> to the running average
else
add <aspect> to summary
Item#1 review set
(Item#1 multi-ORE-ratings)
…
…
Multi-criteria CF RecSys in which the item-to-item similarity matrix
is calculated considering each item description (summary)
13
Position
Food
Staff-courtesy
Breakfast-buffet
…
Food
Room-clean
Pool
Item-item similarity matrix
Item #1 summary
Item #2 summary
(Multi-dimensional
similarity measure)
(Rating Prediction)
( , , 0.9 )
( , , 0.7 )
predicted ratings
Algorithm B: Multi-criteria Aspect-based Recommender
system based on sentimenT Analysis (#MARTA) (2/2)
Room-view
…
𝑠𝑖𝑚 Itemi, Itemj= …
Design
17
Research questions
Q1) #multi-ORE-criteria algorithm overtakes the single-criteria(#single-criteria) and multi-criteria techniques (#multi-static-criteria), which use default criteria, results ?
Q2) Multi-criteria Aspect-based Recommender system based on sentimenT Analysis(#MARTA) algorithm overtakes the state-of-the-art approaches results?
10-Fold Cross Validation. For each run the training set isused for calculating the similarity matrix
Evaluation of the approaches on the respective test set
Evaluation metrics: MAE, RMSE, Precision, Recall and F-Measure
Statistical validation using the Wilcoxon test
Dataset
18
Amazon
TripAdvisor
Yelp
#ratings #users #items #ratings/
user
#ratings/
item
sparsity
355.949 2.850 2.820 124,73 126,06 0,96
#ratings #users #items #ratings/
user
#ratings/
item
sparsity
229.905 45.981 11.537 5 19,9 0,99
#ratings #users #items #ratings/
user
#ratings/
item
sparsity
208.135 1.386 1.580 150,16 131,73 0,90
Q1 Results (#multi-ORE-criteria vs #single-
criteria and #multi-static-criteria)
19
Dataset TripAdvisor
MAE
SVD 0,87
#single-criteria 1,32
#multi-static-criteria 1,10
#Multi-ORE-criteria 0,96
0,87
1,32
1,10
0,96
0,00
0,20
0,40
0,60
0,80
1,00
1,20
1,40
MA
E
F-Measure
SVD 0,59
#single-criteria 0,52
#multi-static-criteria 0,53
#multi-ORE-criteria 0,71
0,59
0,52 0,53
0,71
0,00
0,10
0,20
0,30
0,40
0,50
0,60
0,70
0,80
F-M
easu
re
Q2 Results (#MARTA)
20
DatasetYelp (sparsity: 0,99)
MAE
SVD 0,91
#single-criteria 1,18
#multi-ORE-criteria 1,13
#MARTA 0,85
0,91
1,181,13
0,85
0,00
0,20
0,40
0,60
0,80
1,00
1,20
1,40
MA
E
F-Measure
SVD 0,58
#single-criteria 0,49
#multi-ORE-criteria 0,55
#MARTA 0,63
0,58
0,49
0,55
0,63
0,00
0,10
0,20
0,30
0,40
0,50
0,60
0,70
F-M
easu
re
Q2 Results (#MARTA)
21
Dataset TripAdvisor (sparsity: 0,96)
MAE
SVD 0,87
#single-criteria 1,32
#multi-ORE-criteria 0,96
#MARTA 0,87
0,87
1,32
0,960,87
0,00
0,20
0,40
0,60
0,80
1,00
1,20
1,40
MA
E
F-Measure
SVD 0,59
#single-criteria 0,51
#multi-ORE-criteria 0,71
#MARTA 0,58
0,59
0,51
0,71
0,58
0,00
0,10
0,20
0,30
0,40
0,50
0,60
0,70
0,80
F-M
easu
re
Q2 Results (#MARTA)
22
Dataset Amazon (sparsity: 0,90)
MAE
SVD 0,71
#single-criteria 0,65
#multi-ORE-criteria 0,55
#MARTA 0,71
0,710,65
0,55
0,71
0
0,1
0,2
0,3
0,4
0,5
0,6
0,7
0,8
MA
E
F-Measure
#SVD 0,62
#single-criteria 0,65
#multi-ORE-criteria 0,74
#MARTA 0,63
0,62
0,65
0,74
0,63
0,54
0,56
0,58
0,60
0,62
0,64
0,66
0,68
0,70
0,72
0,74
0,76
F-M
easu
re
Analysis of the results
23
Q1) #multi-ORE-criteria algorithm overtakes the single-
criteria(#single-criteria) and multi-criteria techniques (#multi-
static-criteria), which use default criteria, results ?
#multi-ORE-criteria showed better results than #multi-
static-criteria and #single-criteria (MAE and F1)
Q2) Multi-criteria Aspect-based Recommender system based on
sentimenT Analysis(#MARTA) algorithm overtakes the state-of-
the-art approaches results?
#MARTA showed good results especially on sparse
dataset
Conclusions
24
Indipendence from the domain
Influenced by the review quality
Good performance especially on the item classification
(relevant or not)