Amazon MTurk for Security and Privacy Studies

Alan NochensonIST 50110/9/2012

What is Mechanical Turk?

Launched in 2005 Allows requestors to post Human

Interface Tasks (HITs) which are completed by people for a small prices

Security + Privacy + Behavioral Economics

Security: “The state of being free from danger or threat.”

Privacy: “The state or condition of being free from being observed or disturbed by other people.”

Behavioral economics: concerned with decision-making and rationality

Traditional studies in this area

E.g. Grossklags UPSEC ‘08 Recruited participants in from a

university into a lab study Had them play an economic game

(weakest link) in a security context Compared actual behavior to predicted

behavior and found a number of differences

Small scale, time-consuming to organize

Studies using Mechanical Turk

Online surveys and simple task-based surveys Facebook privacy desired settings (Liu et

al.) Targeted ad taglines (Leon et al.) Comparing privacy policy designs (Kelley

et al.)

Studies using Mechanical Turk

More involved uses Phishing susceptibility (Sheng et al.)

Malware installations (Christin et al., Kanich et al.)

Malware installations

Study by Christin et al. aimed to see how much you need to pay people to install an unknown application

Malware installations

70% of participants that ran the program realized the danger

Malware installations

Follow up by Kanich et al. Investigated what vulnerabilities were

active on computers of people that downloaded the program

Found that it costs about $50 to infect 1000 hosts (taking into account payment and vulnerability rates)

Things to keep in mind

Incentives (payment) Validity

Demographics Habitual participants Online effects (Horton et al., Paolacci et

al.) Attrition Cheating Ethics/legality

Questions?