increasing security sensitivity with social proof: a large-scale experimental confirmation, at ccs...
DESCRIPTION
One of the largest outstanding problems in computer security is the need for higher awareness and use of available security tools. One promising but largely unexplored approach is to use social proof: by showing people that their friends use security features, they may be more inclined to explore those features, too. To explore the efficacy of this approach, we showed 50,000 people who use Facebook one of 8 security announcements—7 variations of social proof and 1 non-social control—to increase the exploration and adoption of three security features: Login Notifications, Login Approvals, and Trusted Contacts. Our results indicated that simply showing people the number of their friends that used security features was most effective, and drove 37% more viewers to explore the promoted security features compared to the non-social announcement (thus, raising awareness). In turn, as social announcements drove more people to explore security features, more people who saw social announcements adopted those features, too. However, among those who explored the promoted features, there was no difference in the adoption rate of those who viewed a social versus a non-social announcement. In a follow up survey, we confirmed that the social announcements raised viewer’s awareness of available security features.TRANSCRIPT
Increasing Security Sensitivity With Social Proof: A Large-Scale Experimental Confirmation
Sauvik Das
Carnegie Mellon University
Adam Kramer
Facebook, Inc.
Laura Dabbish
Carnegie Mellon University
Jason Hong
Carnegie Mellon University
1
Summary
2
3
We showed 50,000 facebook users an announcement urging them to explore security tools. Announcements varied in the presence of, specificity, and framing of social proof.
Overview
Social proof increased awareness.
4Overview
Social proof increased overall adoption but not motivation.
5Overview
Simple social proof, with high specificity and no subjective framing, performed best.
6Overview
Background & Motivation
7
Wait, why is this important?
Security Tools Underutilized
8
Today's user-facing security technology can
prevent many of the security breaches average
people experience.
But people do not use user-facing security
technology, for three reasons:
Background & Motivation: Why is this important?
Security Sensitivity
9
AwarenessDo users know about security threats and tools?
MotivationDo users want to use security threats and security tools?
KnowledgeDo users know how to use security tools?
Background & Motivation: Why is this important?
Key Motivation
10
The need for higher security sensitivity remains a
large outstanding problem in computer
security.
Background & Motivation: Why is this important?
How can we best increase security sensitivity?
11Background & Motivation: Why is this important?
Social Proof
12
We look to others for cues on how to act when we are uncertain. If everyone else is doing it, it must be right!
Background & Motivation: Why is this important?
Social proof is known to influence human behavior.- Milgram, Bickman and Berkovitz found that they could get many pedestrians to stop and stare up at the sky if they had a seed group look up at the sky in the middle of the sidewalk.
- On Facebook, Kramer showed that users are more likely to share emotional content that matches the valence of the emotions shared by their friends.
13Background & Motivation: Why is this important?
Social-proof interventions can nudge human behavior.- Cialdini et al. found that hotels can reduce guest’s use of towels by showing them a message that previous hotel guests were less wasteful.
- On Facebook, Bond et al. found that showing people that their friends voted made them significantly more likely to vote.
14Background & Motivation: Why is this important?
How to increase security sensitivity?
15
Social proof is a key catalyst for security related
behavior change—increasing awareness,
motivation and knowledge (Das, Kim, Dabbish,
Hong, 2014).
Background & Motivation: Why is this important?
Key Observation
16
We may be able to use social proof to increase
security sensitivity.
Background & Motivation: Why is this important?
Background & Motivation Recap
17
Key Observation
18
Key MotivationThe need for higher security sensitivity remains one
large outstanding problems in computer
security.
We may be able to use social proof to increase
security sensitivity.
We may be able to use social proof to help solve one of the large outstanding problems in computer security.
19
Our Contributions
20
Can social proof be used to increase security sensitivity?
21Our Contributions
Does the presentation of the social proof (e.g., its specificity and framing) alter its effect on security sensitivity?
22Our Contributions
Methods
23
Methods: Social Prompt Experiment
24
Controlled, randomized experiment with 50,000
active facebook users.
Part of annual security awareness campaign run by
facebook, promoting the following three voluntary-
use security tools:
Methods: Social Prompt Experiment
Promoted Security Tools
25
Login ApprovalsTwo-factor authentication. Enter in additional random code generated on trusted device for every log in.
Login NotificationsReceive e-mail/SMS notifications on every login attempt.
Trusted ContactsSocial identification. Specify 3-5 “trusted contacts” to vouch for you if you forget password and do not have access to registered e-mail.
Methods: Social Prompt Experiment
Security Awareness Campaign
26
Show people an announcement on their newsfeed.
Call-to-action button Announcement text
Methods: Social Prompt Experiment
Adding Social Proof
27
We modified the text to include social proof.
We created seven variations, varying in the
specificity and framing of the social proof.
Methods: Social Prompt Experiment
Raw Template
28
Very specific (exact number/percent), no subjective
framing.
Methods: Social Prompt Experiment
Only Template
29
Very specific, negative framing, at most 10% of
security tool using friends.
Methods: Social Prompt Experiment
Over Framing
30
Less specific (value rounded down), positive
framing, at least 10% of security tool using friends.
Methods: Social Prompt Experiment
Some framing
31
Least specific, no subjective framing.
Methods: Social Prompt Experiment
Sample picked randomly among:
32
U.S. Facebook Users>= 18 years of age
At least 10 friends who used security toolsHad not themselves used security tools
Logged in at least once in the past month
Methods: Social Prompt Experiment
Sample assignment
33
Each person assigned randomly and evenly to be
shown one of the eight announcements.
n=6,250 shown each announcement
Methods: Social Prompt Experiment
Sample experience
34
The campaign ran for 4 days in November ‘13.
Participants shown their assigned announcement
at each login, but at most three times.
Not shown again if they already clicked the call-to-
action button.
Methods: Social Prompt Experiment
Measures
35
Click-through rate (awareness)
7-day adoptions (motivation)
5-month adoptions (motivation)
Our social interventions did not attempt to
increase knowledge of how to use security tools.
Methods: Social Prompt Experiment
Covariates
36
Demographics Social Network
Behavioral
AgeGenderFriend countAccount length
Mean friend ageFriend age entropyPercent male friendsMean friends’ account lengthFriend country entropyMean friend-of-friend countNumber of feature-using friends
Posts CreatedPosts DeletedComments CreatedComments DeletedFriends AddedFriends RemovedPhotos Added
Methods: Social Prompt Experiment
Results
37
Descriptive Stats
38
46,235 (93%) logged in and saw an announcement.5,971 (13%) clicked on the announcement over all. 1,873 (4%) adopted one of the promoted features in 7 days.4,555 (10%) adopted one of the promoted features in 5 months.
Results
Raw Overview
39
Group N Clicks 7-day adoptions
5-month adoptions
Raw # 5862 846 (14.4%) 280 (4.8%) 623 (10.6%)
Some 5828 835 (14.3%) 243 (4.2%) 602 (10.3%)
Over # 5770 779 (13.5%) 248 (4.3%) 547 (9.5%)
Only # 5668 748 (13.2%) 225 (4.0%) 548 (9.7%)
Over % 5761 724 (12.6%) 223 (3.9%) 557 (9.7%)
Only % 5708 714 (12.5%) 221 (3.9%) 555 (9.7%)
Raw % 5953 730 (12.3%) 225 (3.8%) 573 (9.6%)
Control 5685 595 (10.5%) 208 (3.7%) 550 (9.7%)
Results
Raw Overview
40
Group N Clicks 7-day adoptions
5-month adoptions
Raw # 5862 846 (14.4%) 280 (4.8%) 623 (10.6%)
Some 5828 835 (14.3%) 243 (4.2%) 602 (10.3%)
Over # 5770 779 (13.5%) 248 (4.3%) 547 (9.5%)
Only # 5668 748 (13.2%) 225 (4.0%) 548 (9.7%)
Over % 5761 724 (12.6%) 223 (3.9%) 557 (9.7%)
Only % 5708 714 (12.5%) 221 (3.9%) 555 (9.7%)
Raw % 5953 730 (12.3%) 225 (3.8%) 573 (9.6%)
Control 5685 595 (10.5%) 208 (3.7%) 550 (9.7%)
Results
Clicks
41
Does social proof draw more people to explore announcements, and thereby increase people’s awareness of available security tools?
Results
Logistic Regression
42
Modeled clicks with a logistic regression.
DV: Clicked (yes/no)IV: Which announcement shownControls: Previously listed demographic, social network, and behavioral covariates.
Results
Clicks Model Regression Table
43
Variable Coefficient
Treatment: Raw # 0.36
Treatment: Some 0.35
Treatment: Over # 0.29
Treatment: Only # 0.26
Treatment: Over % 0.21
Treatment: Only % 0.19
Treatment: Raw % 0.17
# security feature using friends 0.09
p < 0.001 for allResults
Clicks Model Regression Table
44
Variable Coefficient
Treatment: Raw # 0.36
Treatment: Some 0.35
Treatment: Over # 0.29
Treatment: Only # 0.26
Treatment: Over % 0.21
Treatment: Only % 0.19
Treatment: Raw % 0.17
# security feature using friends 0.09
p < 0.001 for allResults
Clicks Model Odds Ratios
45
1
1.1
1.2
1.3
1.4
1.5
Raw % Only % Over % Only # Over # Some Raw #
1.431.42
1.341.3
1.231.21
1.19
Odds ratio for clicking on announcement (relative to control)
p < 0.001 for allResults
Clicks Model: Specificity
46
# conditions get 7% more clicks than %
conditions (p=0.0004).
But specificity has a nuanced effect. The two best
performers were very specific (Raw #) and
completely ambiguous (Some).
Results
Clicks Model: Framing
47
Framing of social proof did not have an effect.
Insignificant performance differences between Raw
(13.3%), Over (13.0%), and Only (12.9%) framings
(p=0.54).
Results
Clicks Model Regression Table
48
Variable Coefficient
Treatment: Raw # 0.36
Treatment: Some 0.35
Treatment: Over # 0.29
Treatment: Only # 0.26
Treatment: Over % 0.21
Treatment: Only % 0.19
Treatment: Raw % 0.17
# security feature using friends 0.09
p < 0.001 for allResults
Clicks Finding Summary
49
1. Social proof can help increase awareness of security tools. And, this effect is amplified when people have more security-feature using friends.
2. Framing had no statistically discernible effect, but the performance of the Raw # condition suggests that wordsmithing is unlikely to help.
3. Specificity had a non-linear effect. #s were better than %s, but both very specific and ambiguous social proof attracted clicks.
Results
Adoptions
50
Does social proof motivate more people to adopt available security tools?
Results
Logistic Regression
51
Modeled short term and long term adoptions with a
logistic regression.
DV: 7-day adoptions, 5-month adoptionsIV: Which announcement shownControls: Previously listed demographic, social network, and behavioral covariates. Also, whether or not user clicked on the announcement.
Results
Clicks Model Regression Table
52
Variable 7-day Coefficient 5-mo. Coefficient
Treatment: Raw # -0.01 -0.001
Treatment: Some -0.18 -0.03
Treatment: Over # -0.07 -0.13
Treatment: Only # -0.16 -0.09
Treatment: Over % -0.12 -0.06
Treatment: Only % -0.12 -0.05
Treatment: Raw % -0.15 -0.06
# security feature using friends 0.17 * 0.20 *
* p < 0.05Results
Clicks Model Regression Table
53
Variable 7-day Coefficient 5-mo. Coefficient
Treatment: Raw # -0.01 -0.001
Treatment: Some -0.18 -0.03
Treatment: Over # -0.07 -0.13
Treatment: Only # -0.16 -0.09
Treatment: Over % -0.12 -0.06
Treatment: Only % -0.12 -0.05
Treatment: Raw % -0.15 -0.06
# security feature using friends 0.17 * 0.20 *
* p < 0.05Results
Overall adoptions
54
0
2.2
4.4
6.6
8.8
11
Control Raw % Over % Only % Only # Over # Some Raw #
10.610.39.59.79.79.79.69.7
4.84.24.343.93.93.83.7
7-day overall adoption rate 5-month overall adoption rate
p=0.003Results
Clicks Model Regression Table
55
Variable 7-day Coefficient 5-mo. Coefficient
Treatment: Raw # -0.01 -0.001
Treatment: Some -0.18 -0.03
Treatment: Over # -0.07 -0.13
Treatment: Only # -0.16 -0.09
Treatment: Over % -0.12 -0.06
Treatment: Only % -0.12 -0.05
Treatment: Raw % -0.15 -0.06
# security feature using friends 0.17 * 0.20 *
* p < 0.05Results
Adoptions Finding Summary
56
1. Social proof can increase overall feature adoptions.
2. However, we found no evidence that social proof increases motivation to use security features more than the non-social control.
3. The Raw # condition (High specificity and no subjective framing) again performed best, yielding the highest adoption rate.
Results
Discussion & Implications
57
What does it all mean? What next?
Social proof can increase both awareness and adoption of security tools.
58
Furthermore, this effect increases in strength as more of one’s friends use security tools.
Discussion & Implications
Finding 1
59
Finding 1: Implication
To maximize awareness and adoption, we should iteratively show non-adopters with many security-using friends social proof announcements.
Discussion & Implications
60
Finding 2
The type of social proof we tested did not significantly affect motivation to use security tools.
Discussion & Implications
61
Finding 2: ImplicationThis does not mean that social proof is ineffective or has a negative effect on motivation:
1. Needs to be timely & in context; and,2. Needs to be reinforced at the interface where decisions are being made.
Discussion & Implications
62
Finding 3
The most effective presentation of social proof appears to be the simplest: high specificity and no subjective framing.
Discussion & Implications
63
Finding 3: Implication
No need for wordsmithing. Simply presenting people with social proof that others they know use security tools is the best way to reap the benefits of social-proof based interventions.
Discussion & Implications
Conclusion
64
We provided some of the first empirical evidence that social proof can be used to increase security sensitivity.
65
We believe our work opens up a new line of inquiry for solving the longstanding problem of getting users to care and take agency over their security.
66
3. The most effective presentation of social proof appears to be the simplest: high specificity and no subjective framing.
1. Social proof can increase both awareness and adoption of security tools.2. The type of social proof we tested did not significantly affect motivation to use security tools, but that does not mean that all social proof would be ineffective.
Increasing Security Sensitivity With Social Proof: A Large-Scale Experimental Confirmation
Sauvik Das [[email protected]]
Carnegie Mellon University
Take-aways
67
How to increase security sensitivity?
68
AwarenessSecurity announcements and news.
MotivationMake security tools faster, flashier, cooler.
KnowledgeMake security tools more usable, security education.
Background & Motivation: Why is this important?
Security sensitivity remains lower than ideal.
69Background & Motivation: Why is this important?
Raw Overview
70Results
Group N Clicks 7-day adoptions
5-month adoptions
Raw # 5862 846 (14.4%) 280 (4.8%) 623 (10.6%)
Some 5828 835 (14.3%) 243 (4.2%) 602 (10.3%)
Over # 5770 779 (13.5%) 248 (4.3%) 547 (9.5%)
Only # 5668 748 (13.2%) 225 (4.0%) 548 (9.7%)
Over % 5761 724 (12.6%) 223 (3.9%) 557 (9.7%)
Only % 5708 714 (12.5%) 221 (3.9%) 555 (9.7%)
Raw % 5953 730 (12.3%) 225 (3.8%) 573 (9.6%)
Control 5685 595 (10.5%) 208 (3.7%) 550 (9.7%)
Raw Overview
71Results
Group N Clicks 7-day adoptions
5-month adoptions
Raw # 5862 846 (14.4%) 280 (4.8%) 623 (10.6%)
Some 5828 835 (14.3%) 243 (4.2%) 602 (10.3%)
Over # 5770 779 (13.5%) 248 (4.3%) 547 (9.5%)
Only # 5668 748 (13.2%) 225 (4.0%) 548 (9.7%)
Over % 5761 724 (12.6%) 223 (3.9%) 557 (9.7%)
Only % 5708 714 (12.5%) 221 (3.9%) 555 (9.7%)
Raw % 5953 730 (12.3%) 225 (3.8%) 573 (9.6%)
Control 5685 595 (10.5%) 208 (3.7%) 550 (9.7%)
Adoption Models Odds Ratios
72Results
0
0.2
0.4
0.6
0.8
1
Some Only # Raw % Over % Only % Over # Raw #
1
0.880.950.940.940.91
0.97 0.990.93
0.890.890.860.850.83
7-day odds ratio for adoptions (relative to control)5-month odds ratio for adoptions (relative to control)
all insignificant
Challenges
73
Wait, why is this hard?
Challenges
74
Historically, security tool usage has been kept
confidential and data of its adoption has been
siloed and stripped of its social context.
Challenges: Why is this hard?
We lack a global view: we do not know who
uses what security tools, nor whether any of their
social connections use those tools.
As a result...
75
(1) It is difficult to create interventions that increase
security sensitivity with social proof; and,
(2) It is difficult for security tools to diffuse through
social channels.
Challenges: Why is this hard?
Social Media To The Rescue
76
Social media platforms have the elusive global view:
they know who does and does not use different
security tools, as well as how many of their social
connections use those security tools.
Working with facebook, we put social proof to the
test in the context of increasing security sensitivity.
Our Contributions
We have overlooked a potentially fruitful opportunity to use social factors to increase security sensitivity.
77Challenges: Why is this hard?
Sample experience
78
Participants who clicked the call-to-action button
of any of the announcements were taken to the
same interstitial that explained the promoted
features and allowed them to activate the features.
Methods: Social Prompt Experiment