![Page 1: Recommendations as a Conversation with the User](https://reader035.vdocuments.mx/reader035/viewer/2022062703/554d9052b4c905525e8b4575/html5/thumbnails/1.jpg)
1Recruiting SolutionsRecruiting SolutionsRecruiting Solutions
Recommendations as aConversation with the User
Daniel TunkelangPrincipal Data Scientist at LinkedIn
Daniel
![Page 2: Recommendations as a Conversation with the User](https://reader035.vdocuments.mx/reader035/viewer/2022062703/554d9052b4c905525e8b4575/html5/thumbnails/2.jpg)
2
Introductions
![Page 3: Recommendations as a Conversation with the User](https://reader035.vdocuments.mx/reader035/viewer/2022062703/554d9052b4c905525e8b4575/html5/thumbnails/3.jpg)
3
Let’s talk about how we talk with machines…
![Page 4: Recommendations as a Conversation with the User](https://reader035.vdocuments.mx/reader035/viewer/2022062703/554d9052b4c905525e8b4575/html5/thumbnails/4.jpg)
4
Clifford Nass’s secret:
1) Find a conclusion by a social science researcher.
2) Change
“People do X when interacting with other people.”
to
“People do X when interacting with a computer.”
3) Profit!
![Page 5: Recommendations as a Conversation with the User](https://reader035.vdocuments.mx/reader035/viewer/2022062703/554d9052b4c905525e8b4575/html5/thumbnails/5.jpg)
5
Let’s work on our relationship.
![Page 6: Recommendations as a Conversation with the User](https://reader035.vdocuments.mx/reader035/viewer/2022062703/554d9052b4c905525e8b4575/html5/thumbnails/6.jpg)
6
Core Message
Recommendations are a conversation with the user.
1) Consider asking vs. guessing.
2) Ask good questions.
3) It's ok to make mistakes…
if you have a good explanation
and adapt to feedback.
![Page 7: Recommendations as a Conversation with the User](https://reader035.vdocuments.mx/reader035/viewer/2022062703/554d9052b4c905525e8b4575/html5/thumbnails/7.jpg)
7
Our goal:
http://www.wilsoninfo.com/computerclipart.shtml
![Page 8: Recommendations as a Conversation with the User](https://reader035.vdocuments.mx/reader035/viewer/2022062703/554d9052b4c905525e8b4575/html5/thumbnails/8.jpg)
8
Overview
1) Theory
2) Examples
3) Action Items
![Page 9: Recommendations as a Conversation with the User](https://reader035.vdocuments.mx/reader035/viewer/2022062703/554d9052b4c905525e8b4575/html5/thumbnails/9.jpg)
9
1) Theory
![Page 10: Recommendations as a Conversation with the User](https://reader035.vdocuments.mx/reader035/viewer/2022062703/554d9052b4c905525e8b4575/html5/thumbnails/10.jpg)
10
Pragmatics: the Study of Conversation
Paul Grice
![Page 11: Recommendations as a Conversation with the User](https://reader035.vdocuments.mx/reader035/viewer/2022062703/554d9052b4c905525e8b4575/html5/thumbnails/11.jpg)
11
Grice’s Maxims of Conversation
Maxim 1: Quality
Maxim 2: Quantity
Maxim 3: Relation
Maxim 4: Manner
H. P. Grice, "Logic and conversation” [1975]
![Page 12: Recommendations as a Conversation with the User](https://reader035.vdocuments.mx/reader035/viewer/2022062703/554d9052b4c905525e8b4575/html5/thumbnails/12.jpg)
12
Maxim 1: Quality
![Page 13: Recommendations as a Conversation with the User](https://reader035.vdocuments.mx/reader035/viewer/2022062703/554d9052b4c905525e8b4575/html5/thumbnails/13.jpg)
13
Quality: Above All, the Truth
Xiao, Bo and Benbasat, Izak. 2011. "Product-Related Deception in E-Commerce: A Theoretical Perspective," MIS Quarterly, (35: 1) pp.169-195.
![Page 14: Recommendations as a Conversation with the User](https://reader035.vdocuments.mx/reader035/viewer/2022062703/554d9052b4c905525e8b4575/html5/thumbnails/14.jpg)
14
Don’t Lie
1) Don’t use “recommended” when you really mean
“sponsored” or “excess inventory”.
2) Optimize for the user’s utility.
3) Apply a standard of evidence (quality, quantity) that
you believe in.
![Page 15: Recommendations as a Conversation with the User](https://reader035.vdocuments.mx/reader035/viewer/2022062703/554d9052b4c905525e8b4575/html5/thumbnails/15.jpg)
15
Maxim 1: Quantity
![Page 16: Recommendations as a Conversation with the User](https://reader035.vdocuments.mx/reader035/viewer/2022062703/554d9052b4c905525e8b4575/html5/thumbnails/16.jpg)
16
Right Amount of Information
1) Exchange small units of information.
2) If recommendations supplement other content,
consider overall cognitive load.
3) Provide short, meaningful explanations.
![Page 17: Recommendations as a Conversation with the User](https://reader035.vdocuments.mx/reader035/viewer/2022062703/554d9052b4c905525e8b4575/html5/thumbnails/17.jpg)
17
Maxim 3: Relation
![Page 18: Recommendations as a Conversation with the User](https://reader035.vdocuments.mx/reader035/viewer/2022062703/554d9052b4c905525e8b4575/html5/thumbnails/18.jpg)
18
Relevant to the User
1) Offer value to the user.
2) Respect task context.
3) Don’t be obnoxious.
![Page 19: Recommendations as a Conversation with the User](https://reader035.vdocuments.mx/reader035/viewer/2022062703/554d9052b4c905525e8b4575/html5/thumbnails/19.jpg)
19
Maxim 4: Manner
![Page 20: Recommendations as a Conversation with the User](https://reader035.vdocuments.mx/reader035/viewer/2022062703/554d9052b4c905525e8b4575/html5/thumbnails/20.jpg)
20
Relevant to the User
1) Eschew obfuscation.
2) Avoid ambiguity.
3) Be brief.
4) Be orderly.
![Page 21: Recommendations as a Conversation with the User](https://reader035.vdocuments.mx/reader035/viewer/2022062703/554d9052b4c905525e8b4575/html5/thumbnails/21.jpg)
21
Another Perspective
Gary Marchionini
![Page 22: Recommendations as a Conversation with the User](https://reader035.vdocuments.mx/reader035/viewer/2022062703/554d9052b4c905525e8b4575/html5/thumbnails/22.jpg)
22
Human-Computer Information Retrieval
Empower people to explore large-scale information
but demand that
people also take responsibility for this control
by expending cognitive and physical energy.
Marchionini, G., “Toward Human-Computer Information Retrieval” [2006]
![Page 23: Recommendations as a Conversation with the User](https://reader035.vdocuments.mx/reader035/viewer/2022062703/554d9052b4c905525e8b4575/html5/thumbnails/23.jpg)
23
Principles of HCIR
1) Do more than deliver relevant information:
facilitate sensemaking.
2) Increase user responsibility and control:
require and reward effort.
3) Adapt to increasingly knowledgeable users over time.
4) Be engaging and fun to use!
![Page 24: Recommendations as a Conversation with the User](https://reader035.vdocuments.mx/reader035/viewer/2022062703/554d9052b4c905525e8b4575/html5/thumbnails/24.jpg)
24
Facilitate Sensemaking
![Page 25: Recommendations as a Conversation with the User](https://reader035.vdocuments.mx/reader035/viewer/2022062703/554d9052b4c905525e8b4575/html5/thumbnails/25.jpg)
25
Require and Reward Effort
http://www.posterenvy.com/catalog/ask_why.jpg
![Page 26: Recommendations as a Conversation with the User](https://reader035.vdocuments.mx/reader035/viewer/2022062703/554d9052b4c905525e8b4575/html5/thumbnails/26.jpg)
26
Adapt to User Knowledge
![Page 27: Recommendations as a Conversation with the User](https://reader035.vdocuments.mx/reader035/viewer/2022062703/554d9052b4c905525e8b4575/html5/thumbnails/27.jpg)
27
Be Engaging!
http://bluenile.com/
![Page 28: Recommendations as a Conversation with the User](https://reader035.vdocuments.mx/reader035/viewer/2022062703/554d9052b4c905525e8b4575/html5/thumbnails/28.jpg)
28
Applying the theory to…
1) Personalized Recommendations
2) Social Recommendations
3) Item Recommendations
![Page 29: Recommendations as a Conversation with the User](https://reader035.vdocuments.mx/reader035/viewer/2022062703/554d9052b4c905525e8b4575/html5/thumbnails/29.jpg)
29
Personalized Recommendations
1) Be transparent about model so users gain insight.
2) Allow users to modify models to correct mistakes.
3) Solicit just enough information to provide value.
![Page 30: Recommendations as a Conversation with the User](https://reader035.vdocuments.mx/reader035/viewer/2022062703/554d9052b4c905525e8b4575/html5/thumbnails/30.jpg)
30
Social Recommendations
1) Identify the right set of similar users.
2) Allow users to manipulate the social lens.
3) Accommodate users who break your model.
![Page 31: Recommendations as a Conversation with the User](https://reader035.vdocuments.mx/reader035/viewer/2022062703/554d9052b4c905525e8b4575/html5/thumbnails/31.jpg)
31
Item Recommendations
1) Explain recommendations to users.
2) Watch out for non-sequiturs (e.g., diapers -> beer).
3) Play well with user-controlled filtering and sorting.
![Page 32: Recommendations as a Conversation with the User](https://reader035.vdocuments.mx/reader035/viewer/2022062703/554d9052b4c905525e8b4575/html5/thumbnails/32.jpg)
32
2) Examples
![Page 33: Recommendations as a Conversation with the User](https://reader035.vdocuments.mx/reader035/viewer/2022062703/554d9052b4c905525e8b4575/html5/thumbnails/33.jpg)
33
![Page 34: Recommendations as a Conversation with the User](https://reader035.vdocuments.mx/reader035/viewer/2022062703/554d9052b4c905525e8b4575/html5/thumbnails/34.jpg)
34
Initial User Experience
![Page 35: Recommendations as a Conversation with the User](https://reader035.vdocuments.mx/reader035/viewer/2022062703/554d9052b4c905525e8b4575/html5/thumbnails/35.jpg)
35
“It just takes 2 minutes…”
![Page 36: Recommendations as a Conversation with the User](https://reader035.vdocuments.mx/reader035/viewer/2022062703/554d9052b4c905525e8b4575/html5/thumbnails/36.jpg)
36
Asking Before Guessing
![Page 37: Recommendations as a Conversation with the User](https://reader035.vdocuments.mx/reader035/viewer/2022062703/554d9052b4c905525e8b4575/html5/thumbnails/37.jpg)
37
Let’s try some answers:
![Page 38: Recommendations as a Conversation with the User](https://reader035.vdocuments.mx/reader035/viewer/2022062703/554d9052b4c905525e8b4575/html5/thumbnails/38.jpg)
38
Uh oh…
![Page 39: Recommendations as a Conversation with the User](https://reader035.vdocuments.mx/reader035/viewer/2022062703/554d9052b4c905525e8b4575/html5/thumbnails/39.jpg)
39
Expressing my gustibus…
![Page 40: Recommendations as a Conversation with the User](https://reader035.vdocuments.mx/reader035/viewer/2022062703/554d9052b4c905525e8b4575/html5/thumbnails/40.jpg)
40
New Star Trek = Yes; New Star Wars = No
![Page 41: Recommendations as a Conversation with the User](https://reader035.vdocuments.mx/reader035/viewer/2022062703/554d9052b4c905525e8b4575/html5/thumbnails/41.jpg)
41
Testing my patience…
![Page 42: Recommendations as a Conversation with the User](https://reader035.vdocuments.mx/reader035/viewer/2022062703/554d9052b4c905525e8b4575/html5/thumbnails/42.jpg)
42
Bring on the quality!
![Page 43: Recommendations as a Conversation with the User](https://reader035.vdocuments.mx/reader035/viewer/2022062703/554d9052b4c905525e8b4575/html5/thumbnails/43.jpg)
43
And continue the conversation.
![Page 44: Recommendations as a Conversation with the User](https://reader035.vdocuments.mx/reader035/viewer/2022062703/554d9052b4c905525e8b4575/html5/thumbnails/44.jpg)
44
Learning from Netflix
1) Ask the user for help up front. But not too much help.
2) Pay attention to what the user tells you!
3) Give users value early and often.
75% of Netflix views result from recommendations
![Page 45: Recommendations as a Conversation with the User](https://reader035.vdocuments.mx/reader035/viewer/2022062703/554d9052b4c905525e8b4575/html5/thumbnails/45.jpg)
45
![Page 46: Recommendations as a Conversation with the User](https://reader035.vdocuments.mx/reader035/viewer/2022062703/554d9052b4c905525e8b4575/html5/thumbnails/46.jpg)
46
Initial User Experience
![Page 47: Recommendations as a Conversation with the User](https://reader035.vdocuments.mx/reader035/viewer/2022062703/554d9052b4c905525e8b4575/html5/thumbnails/47.jpg)
47
Seed with an artist…
![Page 48: Recommendations as a Conversation with the User](https://reader035.vdocuments.mx/reader035/viewer/2022062703/554d9052b4c905525e8b4575/html5/thumbnails/48.jpg)
48
Or track or genre.
![Page 49: Recommendations as a Conversation with the User](https://reader035.vdocuments.mx/reader035/viewer/2022062703/554d9052b4c905525e8b4575/html5/thumbnails/49.jpg)
49
Goo Goo G'joob!
![Page 50: Recommendations as a Conversation with the User](https://reader035.vdocuments.mx/reader035/viewer/2022062703/554d9052b4c905525e8b4575/html5/thumbnails/50.jpg)
50
Ease user into recommendation space…
![Page 51: Recommendations as a Conversation with the User](https://reader035.vdocuments.mx/reader035/viewer/2022062703/554d9052b4c905525e8b4575/html5/thumbnails/51.jpg)
51
And go wild!
![Page 52: Recommendations as a Conversation with the User](https://reader035.vdocuments.mx/reader035/viewer/2022062703/554d9052b4c905525e8b4575/html5/thumbnails/52.jpg)
52
Shared Product: Personalized Stream
![Page 53: Recommendations as a Conversation with the User](https://reader035.vdocuments.mx/reader035/viewer/2022062703/554d9052b4c905525e8b4575/html5/thumbnails/53.jpg)
53
Positive and Negative Feedback
![Page 54: Recommendations as a Conversation with the User](https://reader035.vdocuments.mx/reader035/viewer/2022062703/554d9052b4c905525e8b4575/html5/thumbnails/54.jpg)
54
Learning from Pandora
1) Get meaningful input from user in one step.
2) Explain recommendations to users.
3) Solicit feedback and act on it immediately.
![Page 55: Recommendations as a Conversation with the User](https://reader035.vdocuments.mx/reader035/viewer/2022062703/554d9052b4c905525e8b4575/html5/thumbnails/55.jpg)
55
![Page 56: Recommendations as a Conversation with the User](https://reader035.vdocuments.mx/reader035/viewer/2022062703/554d9052b4c905525e8b4575/html5/thumbnails/56.jpg)
56
My home page…
![Page 57: Recommendations as a Conversation with the User](https://reader035.vdocuments.mx/reader035/viewer/2022062703/554d9052b4c905525e8b4575/html5/thumbnails/57.jpg)
57
Explanations and Humility
![Page 58: Recommendations as a Conversation with the User](https://reader035.vdocuments.mx/reader035/viewer/2022062703/554d9052b4c905525e8b4575/html5/thumbnails/58.jpg)
58
Explain What and Why
![Page 59: Recommendations as a Conversation with the User](https://reader035.vdocuments.mx/reader035/viewer/2022062703/554d9052b4c905525e8b4575/html5/thumbnails/59.jpg)
59
Recommendations as a Starting Point
![Page 60: Recommendations as a Conversation with the User](https://reader035.vdocuments.mx/reader035/viewer/2022062703/554d9052b4c905525e8b4575/html5/thumbnails/60.jpg)
60
Learning from Amazon
1) Show the factors that drive your conclusions.
2) Distinguish different kinds of recommendations.
3) Combine recommendations with user control.
Amazon: 35% of sales result from recommendations
![Page 61: Recommendations as a Conversation with the User](https://reader035.vdocuments.mx/reader035/viewer/2022062703/554d9052b4c905525e8b4575/html5/thumbnails/61.jpg)
61
3) Action Items
![Page 62: Recommendations as a Conversation with the User](https://reader035.vdocuments.mx/reader035/viewer/2022062703/554d9052b4c905525e8b4575/html5/thumbnails/62.jpg)
62
Increase explainability.
Explanations can be even more important than the recommendations themselves.
Herlocker et al., “Explaining collaborative filtering recommendations” [2000]
Sinha and Swearingen, “The role of transparency in recommender systems” [2002]
Tintarev and Masthoff, “Effective explanations of recommendations: User-centered design” [2007]
(via Òscar Celma’s book, Music Recommendation and Discovery: The Long Tail, Long Fail, and Long Play in the Digital Music Space)
![Page 63: Recommendations as a Conversation with the User](https://reader035.vdocuments.mx/reader035/viewer/2022062703/554d9052b4c905525e8b4575/html5/thumbnails/63.jpg)
63
Some models more explainable than others.
1) Consider decision trees and rule-based systems.
2) Avoid using latent, unlabeled features.
3) If the model is opaque, use examples as surrogates.
![Page 64: Recommendations as a Conversation with the User](https://reader035.vdocuments.mx/reader035/viewer/2022062703/554d9052b4c905525e8b4575/html5/thumbnails/64.jpg)
64
Make a good first impression.
Your user’s first experience is critical.
Use popularity as a default if it makes sense.
Solicit one valuable piece of information as quickly and painlessly as possible.
“Do you like the taste of beer?”
http://blog.okcupid.com/index.php/the-best-questions-for-first-dates/
![Page 65: Recommendations as a Conversation with the User](https://reader035.vdocuments.mx/reader035/viewer/2022062703/554d9052b4c905525e8b4575/html5/thumbnails/65.jpg)
65
Design feedback into your system.
You can make mistakes, if users can easily fix them.
Challenging if models use offline computation.
Respond instantly; generalize as quickly as possible.
Agarwal and Chen, “Machine Learning for Large Scale Recommender Systems” [ICML 2011 Tutorial]
![Page 66: Recommendations as a Conversation with the User](https://reader035.vdocuments.mx/reader035/viewer/2022062703/554d9052b4c905525e8b4575/html5/thumbnails/66.jpg)
66
Integrate recommendations with search.
Recommend next steps, not just items.
In a task context, recommendations are just another source of information scent.
Be careful in integrating offline recommendations with online features like search and navigation.
Pirolli, Information Foraging Theory: Adaptive Interaction with Information [2007]
![Page 67: Recommendations as a Conversation with the User](https://reader035.vdocuments.mx/reader035/viewer/2022062703/554d9052b4c905525e8b4575/html5/thumbnails/67.jpg)
67
Summary
Recommendations are a conversation with the user.
1) Consider asking vs. guessing.
2) Ask good questions.
3) It's ok to make mistakes…
if you have a good explanation
and adapt to feedback.