Evaluating Similarity Measures: A Large-Scale Study in the orkut Social Network Ellen Spertus spertus@google.com

Download Evaluating Similarity Measures: A Large-Scale Study in the orkut Social Network Ellen Spertus spertus@google.com

Post on 29-Mar-2015

215 views

Category:

Documents

2 download

Embed Size (px)

TRANSCRIPT

<ul><li>Slide 1</li></ul> <p>Evaluating Similarity Measures: A Large-Scale Study in the orkut Social Network Ellen Spertus spertus@google.com Slide 2 Recommender systems What are they? Example: Amazon Slide 3 Controversial recommenders What to do when your TiVo thinks youre gay, Wall Street Journal, Nov. 26, 2002 http://tinyurl.com/2qyepg Slide 4 Controversial recommenders What to do when your TiVo thinks youre gay, Wall Street Journal, Nov. 26, 2002 http://tinyurl.com/2qyepg Slide 5 Controversial recommenders What to do when your TiVo thinks youre gay, Wall Street Journal, Nov. 26, 2002 http://tinyurl.com/2qyepg Slide 6 Controversial recommenders Wal-Mart DVD recommendations http://tinyurl.com/2gp2hm Slide 7 Controversial recommenders Wal-Mart DVD recommendations http://tinyurl.com/2gp2hm Slide 8 Controversial recommenders Wal-Mart DVD recommendations http://tinyurl.com/2gp2hm Slide 9 Googles mission To organize the world's information and make it universally accessible and useful. Slide 10 communities Slide 11 Community recommender Goal: Per-community ranked recommendations How to determine? Slide 12 Community recommender Goal: Per-community ranked recommendations How to determine? Implicit collaborative filtering Look for common membership between pairs of communities Slide 13 Terminology Consider each community to be a set of members B: base community (e.g., Pizza) R: related community (e.g., Cheese) Similarity measure Based on overlap |BR| Slide 14 Example: Pizza Slide 15 Slide 16 Terminology Consider each community to be a set of members B: base community (e.g., Wine) R: related community (e.g., Linux) Similarity measure Based on overlap |BR| Also depends on |B| and |R| Possibly asymmetric Slide 17 Example of asymmetry Slide 18 Similarity measures L1 normalization L2 normalization Pointwise mutual information Positive correlations Positive and negative correlations Salton tf-idf Log-odds Slide 19 L1 normalization Vector notation Set notation Slide 20 L2 normalization Vector notation Set notation Slide 21 Mutual information: positive correlation Formally, Informally, how well membership in the base community predicts membership in the related community Slide 22 Mutual information: positive and negative correlation Slide 23 Salton tf-idf Slide 24 LogOdds0 Formally, Informally, how much likelier a member of B is to belong to R than a non-member of B is. Slide 25 LogOdds0 Formally, Informally, how much likelier a member of B is to belong to R than a non-member of B is. This yielded the same rankings as L1. Slide 26 LogOdds Slide 27 Predictions? Were there significant differences among the measures? Top-ranked recommendations User preference Which measure was best? Was there a partial or total ordering of measures? Slide 28 Recommendations for I love wine (2400) Slide 29 Experiment Precomputed top 12 recommendations for each base community for each similarity measure When a user views a community page Hash the community and user ID to Select an ordered pair of measures to Interleave, filtering out duplicates Track clicks of new users Slide 30 Click interpretation Slide 31 Slide 32 Overall click rate (July 1-18) Total recommendation pages generated: 4,106,050 Slide 33 Overall click rate (July 1-18) Slide 34 Slide 35 Analysis For each pair of similarity measures M a and M b and each click C, either: M a recommended C more highly than M b M a and M b recommended C equally M b recommended C more highly than M a Slide 36 Results Clicks leading to joins L2 MI1 MI2 IDF L1 LogOdds All clicks L2 L1 MI1 MI2 IDF LogOdds Slide 37 Positional effects Original experiment Ordered recommendations by rank Second experiment Generated recommendations using L2 Pseudo-randomly ordered recommendations, tracking clicks by placement Tracked 1.3 M clicks between September 22-October 21 Slide 38 Results: single row (n=28108) Namorado Para o Bulldog Slide 39 Results: single row (n=28,108) p=.12 (not significant) Slide 40 Results: two rows (n=24,459) Slide 41 p</p>