authentication melee: a usability analysis of seven web ... brigham young university byu...

Download Authentication Melee: A Usability Analysis of Seven Web ... Brigham Young University BYU ScholarsArchive

Post on 23-Dec-2020

0 views

Category:

Documents

0 download

Embed Size (px)

TRANSCRIPT

  • Brigham Young University BYU ScholarsArchive

    All Theses and Dissertations

    2015-04-01

    Authentication Melee: A Usability Analysis of Seven Web Authentication Systems Scott Ruoti Brigham Young University - Provo

    Follow this and additional works at: http://scholarsarchive.byu.edu/etd

    Part of the Computer Sciences Commons

    This Thesis is brought to you for free and open access by BYU ScholarsArchive. It has been accepted for inclusion in All Theses and Dissertations by an authorized administrator of BYU ScholarsArchive. For more information, please contact scholarsarchive@byu.edu.

    Recommended Citation Ruoti, Scott, "Authentication Melee: A Usability Analysis of Seven Web Authentication Systems" (2015). All Theses and Dissertations. Paper 4376.

    http://home.byu.edu/home/?utm_source=scholarsarchive.byu.edu%2Fetd%2F4376&utm_medium=PDF&utm_campaign=PDFCoverPages http://home.byu.edu/home/?utm_source=scholarsarchive.byu.edu%2Fetd%2F4376&utm_medium=PDF&utm_campaign=PDFCoverPages http://scholarsarchive.byu.edu?utm_source=scholarsarchive.byu.edu%2Fetd%2F4376&utm_medium=PDF&utm_campaign=PDFCoverPages http://scholarsarchive.byu.edu/etd?utm_source=scholarsarchive.byu.edu%2Fetd%2F4376&utm_medium=PDF&utm_campaign=PDFCoverPages http://scholarsarchive.byu.edu/etd?utm_source=scholarsarchive.byu.edu%2Fetd%2F4376&utm_medium=PDF&utm_campaign=PDFCoverPages http://network.bepress.com/hgg/discipline/142?utm_source=scholarsarchive.byu.edu%2Fetd%2F4376&utm_medium=PDF&utm_campaign=PDFCoverPages http://scholarsarchive.byu.edu/etd/4376?utm_source=scholarsarchive.byu.edu%2Fetd%2F4376&utm_medium=PDF&utm_campaign=PDFCoverPages mailto:scholarsarchive@byu.edu

  • Authentication Melee: A Usability Analysis of Seven Web

    Authentication Systems

    Scott Ruoti

    A thesis submitted to the faculty of Brigham Young University

    in partial fulfillment of the requirements for the degree of

    Master of Science

    Kent Seamons, Chair Charles Knutson

    Dan Olsen

    Department of Computer Science

    Brigham Young University

    February 2015

    Copyright c© 2015 Scott Ruoti

    All Rights Reserved

  • ABSTRACT

    Authentication Melee: A Usability Analysis of Seven Web Authentication Systems

    Scott Ruoti Department of Computer Science, BYU

    Master of Science

    Passwords continue to dominate the authentication landscape in spite of numerous proposals to replace them. Even though usability is a key factor in replacing passwords, very few alternatives have been subjected to formal usability studies and even fewer have been analyzed using a standard metric. We report the results of four within-subjects usability studies for seven web authentication systems. These systems span federated, smartphone, paper tokens, and email-based approaches. Our results indicate that participants prefer single sign-on systems. We utilize the Systems Usability Scale (SUS) as a standard metric for empirical analysis and find that it produces reliable, replicable results. SUS proves to be an accurate measure of baseline usability and we recommend that going forward all new authentication proposals be required to meet a minimum SUS score before being accepted by the security community. Our usability studies also gather insightful information from participants’ qualitative responses: we find that transparency increases usability but also leads to confusion and a lack of trust, participants prefer single sign-on but wish to augment it with site-specific low-entropy passwords, and participants are intrigued by biometrics and phone-based authentication.

    Keywords: Usable Security, Authentication, User Study, System Usability Scale

  • ACKNOWLEDGMENTS

    Thanks go to Brent Roberts for help administering the user studies and providing

    some basic analysis of collected data. A special thanks goes to my wife, Emily Ruoti, for

    helping edit this thesis and the WWW’15 submission based on this thesis, and for all the

    other support she gave me throughout my Master’s program.

  • Table of Contents

    List of Figures vii

    List of Tables ix

    1 Introduction 1

    2 Related Work 3

    3 Authentication Tournament 7

    3.1 System Usability Scale . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

    3.2 Tournament Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

    3.2.1 Federated Single Sign-on . . . . . . . . . . . . . . . . . . . . . . . . . 11

    3.2.2 Email-based Single Sign-on . . . . . . . . . . . . . . . . . . . . . . . . 11

    3.2.3 QR Code-based . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

    4 System Walkthroughs 13

    4.1 Google OAuth 2.0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

    4.2 Facebook Connect . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

    4.3 Mozilla Persona . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

    4.4 Simple Authentication for the Web . . . . . . . . . . . . . . . . . . . . . . . 22

    4.5 Hatchet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

    4.6 WebTicket . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

    4.7 Snap2Pass . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

    iv

  • 5 Methodology 33

    5.1 Study Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

    5.1.1 Quality Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

    5.1.2 Participants Demographics . . . . . . . . . . . . . . . . . . . . . . . . 35

    5.2 Task Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

    5.2.1 Authentication System Implementation . . . . . . . . . . . . . . . . . 37

    5.3 Study Questionnaire . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

    5.4 Survey Development . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

    5.5 Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

    6 Results 40

    6.1 First Study – Federated . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

    6.2 Second Study – Email-based . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

    6.3 Third Study – QR Code-based . . . . . . . . . . . . . . . . . . . . . . . . . . 44

    6.4 Fourth Study – “Championship Round” . . . . . . . . . . . . . . . . . . . . 45

    7 Discussion 46

    7.1 System Usability Scale . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

    7.2 Transparency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

    7.3 Single Sign-on Protocols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

    7.3.1 Additional Low-entropy Passwords . . . . . . . . . . . . . . . . . . . 49

    7.3.2 Reputation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

    7.3.3 Dedicated Identity Providers . . . . . . . . . . . . . . . . . . . . . . . 50

    7.4 The Coolness Factor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

    7.4.1 Biometrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

    7.5 Physical Tokens . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

    7.6 Implementation Lessons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

    8 Conclusion 54

    v

  • References 56

    A Usability Study Survey 61

    A.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

    A.2 Demographics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

    A.3 Tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64

    A.3.1 Task 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64

    A.3.2 Task 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64

    A.3.3 Task 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

    A.3.4 Task 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

    A.3.5 Task 5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66

    A.3.6 Task 6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

    A.4 Questionnaire . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

    A.5 End-of-survey Questionnaire . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

    B Federated Single Sign-on Usability Study – Participant Responses 69

    C Email-based Usability Study – Participant Responses 88

    D QR Code-based Usability Study – Participant Responses 110

    E “Championship Round” Usability Study – Participant Responses 134

    vi

  • List of Figures

    3.1 An adjective-oriented interpretation of SUS scores . . . . . . . . . . . . . . . 9

    3.2 Authentication tournament bracket . . . . . . . . . . . . . . . . . . . . . . . 10

    4.1 Google OAuth 2.0 login button . . . . . . . . . . . . . . . . . . . . . . . . . 14

    4.2 Google OAuth 2.0 login screen . . . . . . . . . . . . . . . . . . . . . . . . . . 14

    4.3 Google OAuth 2.0 user account selection screen . . . . . . . . . . . . . . . . 15

    4.4 Google OAuth 2.0 permission grant screen . . . . . . . . . . . . . . . . . . . 15

    4.5 Facebook Connect login button . . . . .

View more >