the human manifoldevents.csml.ucl.ac.uk/userdata/lunch_talks/2013_10_15_tg.pdf · 2013-11-19 ·...
TRANSCRIPT
![Page 1: The Human Manifoldevents.csml.ucl.ac.uk/userdata/lunch_talks/2013_10_15_tg.pdf · 2013-11-19 · Mapping the Human Manifold Scientific potential - Understanding: Better understand](https://reader034.vdocuments.mx/reader034/viewer/2022050205/5f585dac4405147d2c42603f/html5/thumbnails/1.jpg)
The Human Manifold
Thore Graepel
MSR Cambridge: Yoram Bachrach and Pushmeet KohliMSR Redmond: Kristin Lauter and Michael NährigBing: Milad Shokouhi, Bin BiCambridge University: Michal Kosinski and David Stillwell
On the Predictability of Human Online Behaviour
and its Consequences
![Page 2: The Human Manifoldevents.csml.ucl.ac.uk/userdata/lunch_talks/2013_10_15_tg.pdf · 2013-11-19 · Mapping the Human Manifold Scientific potential - Understanding: Better understand](https://reader034.vdocuments.mx/reader034/viewer/2022050205/5f585dac4405147d2c42603f/html5/thumbnails/2.jpg)
Overview
Predicting human attributes from online behaviour Motivation and methodology
Results and interpretation
Promises Personalization of web services
Personalization of query auto-suggest
Perils Consequences for online privacy
Confidential Machine Learning
Conclusions
![Page 3: The Human Manifoldevents.csml.ucl.ac.uk/userdata/lunch_talks/2013_10_15_tg.pdf · 2013-11-19 · Mapping the Human Manifold Scientific potential - Understanding: Better understand](https://reader034.vdocuments.mx/reader034/viewer/2022050205/5f585dac4405147d2c42603f/html5/thumbnails/3.jpg)
Overview
Predicting human attributes from online behaviour Motivation and methodology
Results and interpretation
Promises Personalization of web services
Personalization of query auto-suggest
Perils Consequences for online privacy
Confidential Machine Learning
Conclusions
![Page 4: The Human Manifoldevents.csml.ucl.ac.uk/userdata/lunch_talks/2013_10_15_tg.pdf · 2013-11-19 · Mapping the Human Manifold Scientific potential - Understanding: Better understand](https://reader034.vdocuments.mx/reader034/viewer/2022050205/5f585dac4405147d2c42603f/html5/thumbnails/4.jpg)
The Digital Traces: Online Data
![Page 5: The Human Manifoldevents.csml.ucl.ac.uk/userdata/lunch_talks/2013_10_15_tg.pdf · 2013-11-19 · Mapping the Human Manifold Scientific potential - Understanding: Better understand](https://reader034.vdocuments.mx/reader034/viewer/2022050205/5f585dac4405147d2c42603f/html5/thumbnails/5.jpg)
High-Dimensional Observation Space
www.marketersstudio.com
![Page 6: The Human Manifoldevents.csml.ucl.ac.uk/userdata/lunch_talks/2013_10_15_tg.pdf · 2013-11-19 · Mapping the Human Manifold Scientific potential - Understanding: Better understand](https://reader034.vdocuments.mx/reader034/viewer/2022050205/5f585dac4405147d2c42603f/html5/thumbnails/6.jpg)
Mapping the Human Manifold Scientific potential - Understanding:
Better understand human behaviour
Understand commonality and individual differences among people
Obtain psychometric measurements at an unprecedented scale
Business potential - Predicting: Develop fine-grained and predictive psycho-demographic user profiles
Increase user satisfaction by deep personalization for products and services
Increase revenue by providing more engaging ads and recommendations
How: Find mapping to interpretable dimensions Personality, Intelligence, Happiness, etc.
![Page 7: The Human Manifoldevents.csml.ucl.ac.uk/userdata/lunch_talks/2013_10_15_tg.pdf · 2013-11-19 · Mapping the Human Manifold Scientific potential - Understanding: Better understand](https://reader034.vdocuments.mx/reader034/viewer/2022050205/5f585dac4405147d2c42603f/html5/thumbnails/7.jpg)
Openness
• Appreciation of art, emotion, adventure, and variety of experience
Conscientiousness
• Self-discipline, act dutifully, and aim for achievement
Extraversion
• Energy, positive emotions, seek social stimulation
Agreeableness
• Compassionate and cooperative rather than suspicious
Neuroticism
• Experience unpleasant emotions easily, such as anger and anxiety
Big Five Personality traits
![Page 8: The Human Manifoldevents.csml.ucl.ac.uk/userdata/lunch_talks/2013_10_15_tg.pdf · 2013-11-19 · Mapping the Human Manifold Scientific potential - Understanding: Better understand](https://reader034.vdocuments.mx/reader034/viewer/2022050205/5f585dac4405147d2c42603f/html5/thumbnails/8.jpg)
General Intelligence - IQ
![Page 9: The Human Manifoldevents.csml.ucl.ac.uk/userdata/lunch_talks/2013_10_15_tg.pdf · 2013-11-19 · Mapping the Human Manifold Scientific potential - Understanding: Better understand](https://reader034.vdocuments.mx/reader034/viewer/2022050205/5f585dac4405147d2c42603f/html5/thumbnails/9.jpg)
Satisfaction with Life Scale (Happiness)
To what degree to you agree with the following statements?
In most ways my life is close to ideal.
The conditions of my life are excellent.
I am satisfied with my life.
So far I have gotten the important things I want in life.
If I could live my life over, I would change almost nothing.
![Page 10: The Human Manifoldevents.csml.ucl.ac.uk/userdata/lunch_talks/2013_10_15_tg.pdf · 2013-11-19 · Mapping the Human Manifold Scientific potential - Understanding: Better understand](https://reader034.vdocuments.mx/reader034/viewer/2022050205/5f585dac4405147d2c42603f/html5/thumbnails/10.jpg)
A Treasure Chest of Data: MyPersonality
Facebook App since 2008
Over 8 Million psychometric test results Personality
Intelligence
Happiness
Volunteered user profiles Relationship status, age, gender
Facebook Likes
Friendship network
www.myPersonality.org
![Page 12: The Human Manifoldevents.csml.ucl.ac.uk/userdata/lunch_talks/2013_10_15_tg.pdf · 2013-11-19 · Mapping the Human Manifold Scientific potential - Understanding: Better understand](https://reader034.vdocuments.mx/reader034/viewer/2022050205/5f585dac4405147d2c42603f/html5/thumbnails/12.jpg)
History of Psychometrics Work Pushmeet Kohli pioneers personality direction in
project with intern Michal Kosinski, 2009 Internal tech-report “Personality and Online Behaviour of the Crowd”
Collaboration re-kindled, work resumed, 2011 Bachrach, Kosinski, Graepel, Kohli, and Stillwell, “Personality and Patterns of
Facebook Usage”, in ACM Web Sciences 2012
Kosinski, Stillwell, Kohli, Bachrach, and Graepel, “Personality and Website Choice”, in ACM Web Sciences 2012
Kosinski, Stillwell, and Graepel, “Private traits and attributes are predictable from digital records of human behaviour”, in PNAS 2013
Shokouhi, Bi, Kosinski, Graepel, “Inferring the demographics of search users”, in WWW 2013
![Page 13: The Human Manifoldevents.csml.ucl.ac.uk/userdata/lunch_talks/2013_10_15_tg.pdf · 2013-11-19 · Mapping the Human Manifold Scientific potential - Understanding: Better understand](https://reader034.vdocuments.mx/reader034/viewer/2022050205/5f585dac4405147d2c42603f/html5/thumbnails/13.jpg)
PNAS WWW 2013
![Page 14: The Human Manifoldevents.csml.ucl.ac.uk/userdata/lunch_talks/2013_10_15_tg.pdf · 2013-11-19 · Mapping the Human Manifold Scientific potential - Understanding: Better understand](https://reader034.vdocuments.mx/reader034/viewer/2022050205/5f585dac4405147d2c42603f/html5/thumbnails/14.jpg)
The Magic of Machine Learning
![Page 15: The Human Manifoldevents.csml.ucl.ac.uk/userdata/lunch_talks/2013_10_15_tg.pdf · 2013-11-19 · Mapping the Human Manifold Scientific potential - Understanding: Better understand](https://reader034.vdocuments.mx/reader034/viewer/2022050205/5f585dac4405147d2c42603f/html5/thumbnails/15.jpg)
High-Dimensional Observation Space
![Page 16: The Human Manifoldevents.csml.ucl.ac.uk/userdata/lunch_talks/2013_10_15_tg.pdf · 2013-11-19 · Mapping the Human Manifold Scientific potential - Understanding: Better understand](https://reader034.vdocuments.mx/reader034/viewer/2022050205/5f585dac4405147d2c42603f/html5/thumbnails/16.jpg)
The Magic of Machine Learning
![Page 17: The Human Manifoldevents.csml.ucl.ac.uk/userdata/lunch_talks/2013_10_15_tg.pdf · 2013-11-19 · Mapping the Human Manifold Scientific potential - Understanding: Better understand](https://reader034.vdocuments.mx/reader034/viewer/2022050205/5f585dac4405147d2c42603f/html5/thumbnails/17.jpg)
Mapping the Manifold
![Page 18: The Human Manifoldevents.csml.ucl.ac.uk/userdata/lunch_talks/2013_10_15_tg.pdf · 2013-11-19 · Mapping the Human Manifold Scientific potential - Understanding: Better understand](https://reader034.vdocuments.mx/reader034/viewer/2022050205/5f585dac4405147d2c42603f/html5/thumbnails/18.jpg)
The Magic of Machine Learning
![Page 19: The Human Manifoldevents.csml.ucl.ac.uk/userdata/lunch_talks/2013_10_15_tg.pdf · 2013-11-19 · Mapping the Human Manifold Scientific potential - Understanding: Better understand](https://reader034.vdocuments.mx/reader034/viewer/2022050205/5f585dac4405147d2c42603f/html5/thumbnails/19.jpg)
Making Predictions
Intelligence
![Page 20: The Human Manifoldevents.csml.ucl.ac.uk/userdata/lunch_talks/2013_10_15_tg.pdf · 2013-11-19 · Mapping the Human Manifold Scientific potential - Understanding: Better understand](https://reader034.vdocuments.mx/reader034/viewer/2022050205/5f585dac4405147d2c42603f/html5/thumbnails/20.jpg)
Overview
Predicting human attributes from online behaviour Motivation and methodology
Results and interpretation
Promises Personalization of web services
Personalization of query auto-suggest
Perils Consequences for online privacy
Confidential Machine Learning
Conclusions
![Page 21: The Human Manifoldevents.csml.ucl.ac.uk/userdata/lunch_talks/2013_10_15_tg.pdf · 2013-11-19 · Mapping the Human Manifold Scientific potential - Understanding: Better understand](https://reader034.vdocuments.mx/reader034/viewer/2022050205/5f585dac4405147d2c42603f/html5/thumbnails/21.jpg)
![Page 22: The Human Manifoldevents.csml.ucl.ac.uk/userdata/lunch_talks/2013_10_15_tg.pdf · 2013-11-19 · Mapping the Human Manifold Scientific potential - Understanding: Better understand](https://reader034.vdocuments.mx/reader034/viewer/2022050205/5f585dac4405147d2c42603f/html5/thumbnails/22.jpg)
Prediction Accuracy: Binary variables
![Page 23: The Human Manifoldevents.csml.ucl.ac.uk/userdata/lunch_talks/2013_10_15_tg.pdf · 2013-11-19 · Mapping the Human Manifold Scientific potential - Understanding: Better understand](https://reader034.vdocuments.mx/reader034/viewer/2022050205/5f585dac4405147d2c42603f/html5/thumbnails/23.jpg)
What is “Area under the Curve” (AUC)
![Page 24: The Human Manifoldevents.csml.ucl.ac.uk/userdata/lunch_talks/2013_10_15_tg.pdf · 2013-11-19 · Mapping the Human Manifold Scientific potential - Understanding: Better understand](https://reader034.vdocuments.mx/reader034/viewer/2022050205/5f585dac4405147d2c42603f/html5/thumbnails/24.jpg)
How many Likes for good accuracy? About 50% of users in this
sample had at least 100 Likes and about 20% had at least 250 Likes.
Baseline (random guessing) is 50% for binary variables –Gender and Political View
![Page 25: The Human Manifoldevents.csml.ucl.ac.uk/userdata/lunch_talks/2013_10_15_tg.pdf · 2013-11-19 · Mapping the Human Manifold Scientific potential - Understanding: Better understand](https://reader034.vdocuments.mx/reader034/viewer/2022050205/5f585dac4405147d2c42603f/html5/thumbnails/25.jpg)
Prediction Accuracy: Numeric Variables
![Page 26: The Human Manifoldevents.csml.ucl.ac.uk/userdata/lunch_talks/2013_10_15_tg.pdf · 2013-11-19 · Mapping the Human Manifold Scientific potential - Understanding: Better understand](https://reader034.vdocuments.mx/reader034/viewer/2022050205/5f585dac4405147d2c42603f/html5/thumbnails/26.jpg)
![Page 27: The Human Manifoldevents.csml.ucl.ac.uk/userdata/lunch_talks/2013_10_15_tg.pdf · 2013-11-19 · Mapping the Human Manifold Scientific potential - Understanding: Better understand](https://reader034.vdocuments.mx/reader034/viewer/2022050205/5f585dac4405147d2c42603f/html5/thumbnails/27.jpg)
What is there to like? Intelligence
![Page 28: The Human Manifoldevents.csml.ucl.ac.uk/userdata/lunch_talks/2013_10_15_tg.pdf · 2013-11-19 · Mapping the Human Manifold Scientific potential - Understanding: Better understand](https://reader034.vdocuments.mx/reader034/viewer/2022050205/5f585dac4405147d2c42603f/html5/thumbnails/28.jpg)
Which Likes? Happiness
![Page 29: The Human Manifoldevents.csml.ucl.ac.uk/userdata/lunch_talks/2013_10_15_tg.pdf · 2013-11-19 · Mapping the Human Manifold Scientific potential - Understanding: Better understand](https://reader034.vdocuments.mx/reader034/viewer/2022050205/5f585dac4405147d2c42603f/html5/thumbnails/29.jpg)
Which Likes? Extraversion
![Page 30: The Human Manifoldevents.csml.ucl.ac.uk/userdata/lunch_talks/2013_10_15_tg.pdf · 2013-11-19 · Mapping the Human Manifold Scientific potential - Understanding: Better understand](https://reader034.vdocuments.mx/reader034/viewer/2022050205/5f585dac4405147d2c42603f/html5/thumbnails/30.jpg)
Which Likes? Agreeableness
![Page 31: The Human Manifoldevents.csml.ucl.ac.uk/userdata/lunch_talks/2013_10_15_tg.pdf · 2013-11-19 · Mapping the Human Manifold Scientific potential - Understanding: Better understand](https://reader034.vdocuments.mx/reader034/viewer/2022050205/5f585dac4405147d2c42603f/html5/thumbnails/31.jpg)
Overview
Predicting human attributes from online behaviour Motivation and methodology
Results and interpretation
Promises Personalization of web services
Personalization of query auto-suggest
Perils Consequences for online privacy
Confidential Machine Learning
Conclusions
![Page 32: The Human Manifoldevents.csml.ucl.ac.uk/userdata/lunch_talks/2013_10_15_tg.pdf · 2013-11-19 · Mapping the Human Manifold Scientific potential - Understanding: Better understand](https://reader034.vdocuments.mx/reader034/viewer/2022050205/5f585dac4405147d2c42603f/html5/thumbnails/32.jpg)
How can we apply this to Bing users? Predictors are based on Facebook Likes
Bing users are characterized by search history
Train on common representation to transfer knowledge: Open Directory Categories
![Page 33: The Human Manifoldevents.csml.ucl.ac.uk/userdata/lunch_talks/2013_10_15_tg.pdf · 2013-11-19 · Mapping the Human Manifold Scientific potential - Understanding: Better understand](https://reader034.vdocuments.mx/reader034/viewer/2022050205/5f585dac4405147d2c42603f/html5/thumbnails/33.jpg)
How good are transferred predictors?
77%
84%
74%
80%
Age Gender
Facebook Likes Bing Queries
![Page 34: The Human Manifoldevents.csml.ucl.ac.uk/userdata/lunch_talks/2013_10_15_tg.pdf · 2013-11-19 · Mapping the Human Manifold Scientific potential - Understanding: Better understand](https://reader034.vdocuments.mx/reader034/viewer/2022050205/5f585dac4405147d2c42603f/html5/thumbnails/34.jpg)
Political attitudes in the US
Pearson Correlation: 0.72
![Page 35: The Human Manifoldevents.csml.ucl.ac.uk/userdata/lunch_talks/2013_10_15_tg.pdf · 2013-11-19 · Mapping the Human Manifold Scientific potential - Understanding: Better understand](https://reader034.vdocuments.mx/reader034/viewer/2022050205/5f585dac4405147d2c42603f/html5/thumbnails/35.jpg)
Distribution of Christians in the US
Pearson correlation: 0.39
![Page 36: The Human Manifoldevents.csml.ucl.ac.uk/userdata/lunch_talks/2013_10_15_tg.pdf · 2013-11-19 · Mapping the Human Manifold Scientific potential - Understanding: Better understand](https://reader034.vdocuments.mx/reader034/viewer/2022050205/5f585dac4405147d2c42603f/html5/thumbnails/36.jpg)
What can we do with this knowledge? Personalize our products and
services based on inferred demographics & psychometrics
Bing: Re-rank search results, query suggestions, query auto-suggests, ads
Auto-Complete: Given query prefix, re-rank compatible query suggestions
Features MRR Gain
Age +1.8 %
Gender +2.6 %
Location +3.1 %
Short history (session) +0.9 %
Long history +4.4 %
All features +7.9 %
Results: Milad Shokouhi, SIGIR 2013
MRR: Mean Reciprocal Rank
![Page 37: The Human Manifoldevents.csml.ucl.ac.uk/userdata/lunch_talks/2013_10_15_tg.pdf · 2013-11-19 · Mapping the Human Manifold Scientific potential - Understanding: Better understand](https://reader034.vdocuments.mx/reader034/viewer/2022050205/5f585dac4405147d2c42603f/html5/thumbnails/37.jpg)
Overview
Predicting human attributes from online behaviour Motivation and methodology
Results and interpretation
Promises Personalization of web services
Personalization of query auto-suggest
Perils Consequences for online privacy
Confidential Machine Learning
Conclusions
![Page 38: The Human Manifoldevents.csml.ucl.ac.uk/userdata/lunch_talks/2013_10_15_tg.pdf · 2013-11-19 · Mapping the Human Manifold Scientific potential - Understanding: Better understand](https://reader034.vdocuments.mx/reader034/viewer/2022050205/5f585dac4405147d2c42603f/html5/thumbnails/38.jpg)
![Page 39: The Human Manifoldevents.csml.ucl.ac.uk/userdata/lunch_talks/2013_10_15_tg.pdf · 2013-11-19 · Mapping the Human Manifold Scientific potential - Understanding: Better understand](https://reader034.vdocuments.mx/reader034/viewer/2022050205/5f585dac4405147d2c42603f/html5/thumbnails/39.jpg)
![Page 40: The Human Manifoldevents.csml.ucl.ac.uk/userdata/lunch_talks/2013_10_15_tg.pdf · 2013-11-19 · Mapping the Human Manifold Scientific potential - Understanding: Better understand](https://reader034.vdocuments.mx/reader034/viewer/2022050205/5f585dac4405147d2c42603f/html5/thumbnails/40.jpg)
Reactions
![Page 41: The Human Manifoldevents.csml.ucl.ac.uk/userdata/lunch_talks/2013_10_15_tg.pdf · 2013-11-19 · Mapping the Human Manifold Scientific potential - Understanding: Better understand](https://reader034.vdocuments.mx/reader034/viewer/2022050205/5f585dac4405147d2c42603f/html5/thumbnails/41.jpg)
Questions for User Privacy
• Are users aware to what degree we can infer their personal traits and attributes from their digital traces?
• If they were, would they make that data public?
• Do these predictions just summarize what users are signalling to their friends using Facebook Likes?
• Should companies/services be allowed to use this information for commercial purposes?
• Can technology help to mitigate these challenges?
![Page 42: The Human Manifoldevents.csml.ucl.ac.uk/userdata/lunch_talks/2013_10_15_tg.pdf · 2013-11-19 · Mapping the Human Manifold Scientific potential - Understanding: Better understand](https://reader034.vdocuments.mx/reader034/viewer/2022050205/5f585dac4405147d2c42603f/html5/thumbnails/42.jpg)
Overview
Predicting human attributes from online behaviour Motivation and methodology
Results and interpretation
Promises Personalization of web services
Personalization of query auto-suggest
Perils Consequences for online privacy
Confidential Machine Learning
Conclusions
![Page 43: The Human Manifoldevents.csml.ucl.ac.uk/userdata/lunch_talks/2013_10_15_tg.pdf · 2013-11-19 · Mapping the Human Manifold Scientific potential - Understanding: Better understand](https://reader034.vdocuments.mx/reader034/viewer/2022050205/5f585dac4405147d2c42603f/html5/thumbnails/43.jpg)
ML Confidential
![Page 44: The Human Manifoldevents.csml.ucl.ac.uk/userdata/lunch_talks/2013_10_15_tg.pdf · 2013-11-19 · Mapping the Human Manifold Scientific potential - Understanding: Better understand](https://reader034.vdocuments.mx/reader034/viewer/2022050205/5f585dac4405147d2c42603f/html5/thumbnails/44.jpg)
Machine learning on encrypted data?
Yes: Both training and prediction!
How: Somewhat homomorphic encryption (SHE)
But: Restricted class of machine learning algorithms: Must be implemented as bounded-degree polynomials in the inputs
No division, no comparison
Just additions and very few multiplications
Prototype in computational algebra system Magma Linear means classifier
Fisher’s linear discriminant
![Page 45: The Human Manifoldevents.csml.ucl.ac.uk/userdata/lunch_talks/2013_10_15_tg.pdf · 2013-11-19 · Mapping the Human Manifold Scientific potential - Understanding: Better understand](https://reader034.vdocuments.mx/reader034/viewer/2022050205/5f585dac4405147d2c42603f/html5/thumbnails/45.jpg)
Proof-of-Concept: Breast Cancer Data
Graepel, Lauter, and Nährig, “ML Confidential: Machine Learning on Encrypted Data”
in International Conference on Information Security and Cryptology – ICISC 2012
![Page 46: The Human Manifoldevents.csml.ucl.ac.uk/userdata/lunch_talks/2013_10_15_tg.pdf · 2013-11-19 · Mapping the Human Manifold Scientific potential - Understanding: Better understand](https://reader034.vdocuments.mx/reader034/viewer/2022050205/5f585dac4405147d2c42603f/html5/thumbnails/46.jpg)
Conclusions Mapping the manifold of human online behaviour
Science: Understanding human behaviour and large-scale psychometrics
User benefit: Deep personalization and adaptation to user preferences
Balance need for privacy with quality of service
Challenges and future work Define a new predictive and interpretable framework for human behaviour
Which “Likes” do people get exposed to and how? Is there a filter bubble?
Discover causal relationships from observational data
Collaboration with Bing and U Cambridge
![Page 47: The Human Manifoldevents.csml.ucl.ac.uk/userdata/lunch_talks/2013_10_15_tg.pdf · 2013-11-19 · Mapping the Human Manifold Scientific potential - Understanding: Better understand](https://reader034.vdocuments.mx/reader034/viewer/2022050205/5f585dac4405147d2c42603f/html5/thumbnails/47.jpg)
Predictability:
Personalization:
Confidential ML:
![Page 48: The Human Manifoldevents.csml.ucl.ac.uk/userdata/lunch_talks/2013_10_15_tg.pdf · 2013-11-19 · Mapping the Human Manifold Scientific potential - Understanding: Better understand](https://reader034.vdocuments.mx/reader034/viewer/2022050205/5f585dac4405147d2c42603f/html5/thumbnails/48.jpg)
©2013 Microsoft Corporation. All rights reserved.