privacy enhancing keyboard: design, implementation, and usability...

Research ArticlePrivacy Enhancing Keyboard Design Implementationand Usability Testing

Zhen Ling1 Melanie Borgeest2 Chuta Sano3 Jazmyn Fuller4 Anthony Cuomo5

Sirong Lin3 Wei Yu6 Xinwen Fu7 and Wei Zhao8

1Southeast University Nanjing China2University at Albany SUNY Albany NY 12222 USA3UMass Lowell Lowell MA 01854 USA4The City University of New York Queens NY 11367 USA5Quinnipiac University Hamden CT USA6Towson University Towson MD 21252 USA7University of Central Florida Orlando FL 32816 USA8University of Macau Macau

Correspondence should be addressed to Zhen Ling zhenlingseueducn

Received 4 August 2017 Accepted 16 October 2017 Published 8 November 2017

Academic Editor Zhipeng Cai

Copyright copy 2017 Zhen Ling et al This is an open access article distributed under the Creative Commons Attribution Licensewhich permits unrestricted use distribution and reproduction in any medium provided the original work is properly cited

To protect users from numerous password inference attacks we invent a novel context aware privacy enhancing keyboard (PEK)for Android touch-based devices Usually PEK would show a QWERTY keyboard when users input text like an email or a messageNevertheless whenever users enter a password in the input box on his or her touch-enabled device a keyboard will be shown tothem with the positions of the characters shuffled at random PEK has been released on the Google Play since 2014 However thenumber of installations has not lived up to our expectation For the purpose of usable security and privacy we designed a two-stageusability test and performed two rounds of iterative usability testing in 2016 and 2017 summer with continuous improvements ofPEK The observations from the usability testing are educational (1) convenience plays a critical role when users select an inputmethod (2) people think those attacks that PEK prevents are remote from them

1 Introduction

The touch-screen enabled devices have been a popular targetof network attacks Sensitive information like the passwordsentered on mobile devices can be stolen by attackers byexploiting the soft keyboard For example in residue-basedattacks [1ndash5] the tapped keys can be inferred from theoily or heat residues left on the touch screen the orderof which can also be determined by measuring the heatresidue left on the touched positions In computer vision-based attacks [6ndash12] the interaction between the hand andthe keyboard is exploitedThe handmovement and the fingerposition indicate which keys are being touched [10 11 1314] In sensor-based attacks [15ndash21] with the help of theaccelerometer (acceleration) and gyroscope (orientation) a

malware senses the slight motion of a device when the usertypes different keys

To defeat the attacks introduced above we invent a novelcontext aware privacy enhancing keyboard (PEK) for touch-enabled devices It can be observed that these attacks arepossible in that the keys of the keyboard are at the fixedpositionWith PEK we shuffle the positions of the characterson the keyboard Whenever a user of a touch-enableddevice is to type the password a randomized keyboard isshown to her In other words the user is presented with arandomly shuffled keyboard each time she taps a passwordWe maintain the usability of PEK through a context awarefeature a randomized keyboard shows up only when a usertaps a password or pin PEK shows a normal QWERTYkeyboard or a system default one when a user inputs text likean email or a message

HindawiWireless Communications and Mobile ComputingVolume 2017 Article ID 3928261 15 pageshttpsdoiorg10115520173928261

2 Wireless Communications and Mobile Computing

Nonpassword

Password

Inspect the propertyof the Input box

Construct a randomkeyboard

Display the keyboard

Construct a normalkeyboard

Figure 1 Workflow of PEK constructing a keyboard

We are the first to design a generic randomized keyboardfor the Android system though the idea of randomizingthe key layout is not new [22] One version of PEK isimplemented as a third party keyboard for Android and canreplace the system keyboard once it is installed Thereforeonce PEK is chosen as the default keyboard it can be usedby any app

After our presentation at Black Hat USA [10] we releasedPEK as a free Android app to Google Play in August 2014Until the time of writing it has been downloaded 2352times We have released 7 versions with corrected bugs andimproved interface Of them PEK 10 is based on an Androidcode example PEK 2xx and later versions are based onOpenWnn [23] with fixed bugs The current version of PEKis 3233

For the purpose of usable security and privacy wedesigned an iterative usability test to evaluate the user experi-ence of PEK and to explore the reason for the lukewarmnessof using PEK Each iteration of usability test is a two-stagestudy a pilot study and a main study We randomly selectparticipants to reflect different behaviors of Android usersThe pilot study uses surveys and interviews and involvesa small number of people for us to understand potentialusability issues of PEK We then add features to PEK basedon the results of the pilot study and use the main studythrough a web survey hosted at Amazon Mechanical Turk tounderstand the usability of the improved PEK For surveysperformed throughAmazonMechanical Turk we contact theusers to make sure that they install PEK and complete oursurvey questions We performed two rounds of usability testin 2016 summer and 2017 summer respectively After the tworounds of usability testing and app improvements most usersreport the app is easy to install configure and use The otherobservations from the usability testing are also educational(1) convenience plays a critical role when users select an inputmethod (2) peoplemay think those attacks that PEKpreventsare remote from them and the risk from those attacks is smallThe usability test demonstrates the worrisome phenomenathat many users blindly trust their phones for security orare not much concerned with the possible breaches Thesephenomena demonstrate the human factor that contributesto the vulnerabilities of the cyber space

An early version of the usability testing of PEK ispresented in [24] In the conference version we design a two-stage usability test and perform the first usability testing toevaluate the user experience of PEK In this journal versionwe present the detailed algorithms and techniques used toimplement our privacy enhancing keyboard and the secondusability testing to further evaluate the user experience of

updated PEK Related work and new experimental evaluationare presented in this extended version

The rest of this paper is organized as follows The designand the implementation of the third party keyboard of PEKwill be introduced in Section 2 In Section 3 we present themethodology applied in the usability test and the results fromthe first usability testing The second usability testing andresults are presented in Section 4We review the most relatedwork in Section 5 and conclude this paper in Section 6

2 Privacy Enhancing Keyboard

We introduce the basic idea of the privacy enhancing key-board in this section Then we present the technique detailsof PEK implementation and elaborate how to install andconfigure the PEK for Android

21 Basic Idea To thwart diverse attacks such as residue-based attacks computer vision-based attacks and sensor-based attacks we shuffle the positions of the keys of asoftware on a touch screen to ensure that a randomizedkeyboard is shown to a user every time she enters a passwordTherefore the particular keys will not be easily inferred eventhough there is finger oily or thermal residue left on thescreen Moreover vision-based attacks do not work since atouched position by a finger no longer matches a fixed keyThe vibration and orientation information provided by anaccelerometer does not help establish profiles for particularkeys

Figure 1 shows the basic idea and the workflow of PEKconstructing a keyboard when a user touches an input boxThe first step is to inspect the property of this input box tosee whether it is a nonpassword or password input box AQWERTY keyboard is shown if a nonpassword input box isidentified If a password input box is identified we parse theproperty of the keys from a XML file which stores the layoutof the keyboard and then change the label and value of thekeys so that the positions of the keys are shuffled

There are two versions of PEK that we implemented (1)One version is a third party keyboard that can be imple-mented as an Android App for Google Play and installed toan Android device Once enabled PEK runs as an Androidservice in the background A user can use the Android inputsetting menu to enable PEK while the location of the inputsetting menu could vary from device to device Before PEK30 it is the userrsquos responsibility to find the input settingmenu according to the generic introductions we provided atGoogle Play (2) We are also able to revise the source code ofthe Android system default keyboard and the unlock screen

Wireless Communications and Mobile Computing 3

keypad recompile the entire Android project and flash thesystem into the device Apparently the usability of such animplementation is an issue since most users do not havethe capability to recompile the Android system and flash itinto their devices For completeness we also introduce suchan implementation of the unlock screen keyboard while thefocus of our usability testing will be the third-party keyboardversion of PEK

Wehave two challenges for implementing a useful privacyenhancing keyboard

(i) First how can we generate a randomized keyboardThat is what is our privacy enhancing technology

(ii) Second how do we identify the type of input box inorder to show an appropriate keyboard That is howdo we implement the context aware technology

We answer these two questions in the following subsections

22 Privacy Enhancing Technology A general software key-board contains three components denoted as subkeyboardsThe primary subkeyboard is the QWERTY keyboard whichis the most common keyboard layout The second sub-keyboard is the numerical keyboard that may also containsome symbols The last subkeyboard is a symbol keyboardthat contains special symbols The layout of these threesubkeyboards is stored in a XML file which records thepositions labels (the text to display) and values (such asASCII codes) of the keys This keyboard is often used as thesystem default keyboard The system generates the defaultkeyboard in this way the keys will be read from the XML fileone by one and put at a right position

To randomly shuffle the keyboard we just need to changethe label and value at a particular position on a keyboard thatis defined in a XML file Recall that an XML file defines asequence of position label and value corresponding to all thekeys We do not change the overall size of the keyboard andjust change label and value at position Figure 2 illustrates therandomly shuffled keyboard

23 Context Aware Functionality To correctly identifythe type of the input box we can take advantage ofthe input box properties The Android class ldquoEditor-Infordquo can be used to detect the type of input boxIn our case ldquoTYPE NUMBER VARIATION PASSWORDrdquoldquoTYPE TEXT VARIATION PASSWORDrdquo ldquoTYPE TEXTVARIATION VISIBLE PASSWORDrdquo and ldquoTYPE TEXTVARIATION WEB PASSWORDrdquo are used to identify thepassword input box The first type is the variation ofldquoTYPE CLASS NUMBERrdquo while these late three types arethe variations of ldquoTYPE CLASS TEXTrdquo Once the passwordinput box is triggered by the user a new randomizedkeyboard will be constructed As a result the user presenteda different randomized keyboard every time she presses thepassword input box

24 Randomization of the PIN Keypad Although a thirdparty keyboard can be chosen as the keyboard for the unlockscreen we also design and implement a randomized pin

Figure 2 PEK

Figure 3 Randomized keypad

keypad for the unlock screen The implementation of such akeypad involves the revision of the Android system files Forcompleteness we will present its design and implementationHowever we will skip it for the usability testing since it isimpossible for general users to change the Android sourcecode

241 Algorithm Once the PIN mode is selected as a screenlock scheme the system will provide a soft minikeyboardthat is keypad for the user to input a 4-digit pin There are10 digits on the keypad The key icon is used to show thedigits on the buttons instead of a key label As a result weneed to not only randomize the key code but also change thecorresponding icons for keys at different positions Figure 3illustrates the randomized keypad on a Google Galaxy Nexusphone


Table 1 Input time and success rate

Normalkeyboard

Shuffledkeys

Median input time (seconds) 2235 5859Success rate 9850 9883

242 Implementation As shown in Figure 3 an 11-buttonkeypad would be used if the PIN mode is set up as ascreen unlock scheme This keypad is a specially designedkeyboard for the PIN mode instead of a keyboard for thesystem default input method We revised the overriddenmethod ldquocreateKeyFromXml()rdquo in the code file ldquoPasswor-dEntryKeyboardjavardquo to modify the key properties afterthe key constructor is called However the digit shownon the button in Figure 3 is a key icon Consequentlywe need to modify the key codes and correspondingkey icons rather than key labels We store the valuesof the key icons that is Rdrawablesym keyboard num1Rdrawablesym keyboard num2 and so forth into an arrayWe also use the method ResourcesgetDrawable to derive thespecific key icon and replace the original key icon Finally werecompile the source code of the entire Android project toimplement this functionality

25 Installation and Configuration We implement the PEKand release it on the Google Play Store PEK can be found bysearching for either ldquoPEKrdquo or ldquoprivacy enhanced keyboardrdquoon the Google Play Store The downloading process shouldbe fast and relatively quick At Google Store we give a generalintroduction to how to configure the settings of an Androiddevice and use PEK

26 Evaluation of Input Time of PEK To measure the inputtime of the PEK we recruit 20 students 5 female studentsand 15 male students whose average age is 25 years old Weimplemented a test password input box and generated 30random four-letter passwords The students were requiredto input these 30 passwords using a QWERTY keyboardand a shuffled keyboard and the test app recorded the userinput time Table 1 shows the results of our evaluation andFigure 4 gives a box plot of the input time of the two differentkeyboards The median input time is around 22 secondson the QWERTY keyboard and 59 seconds on the shuffledkeyboard The success rates of users inputting four-letterpasswords on both keyboards are high except for the PEKwith a lowest rate The participants in our experiments thinkPEK is acceptable if it pops up the randomized keyboard onlyfor sensitive information input

3 First Usability Testing

In this section we introduce our two-stage usability studyof PEK the pilot study and the main study The first suchusability testing was performed in 2016 summer Thoughsimilar to the former the latter differs from it in the greaternumber of participants questions and other measurementsGenerally speaking it is not necessary to involve many

Normal keyboard Shuffled keys0

5

10

15

20

25

30

35

40

45

Inpu

t tim

e (se

c)

Figure 4 Input time of two distinct keyboards

participants in either the interview or the focus group studyDuring the process of study we keep a good balance of thequalitative and the quantitative results Besides the traditionalqualitative research such as interview and focus groupwe apply various methods to get quantitative informationabout users from different aspects For example the face-to-face interaction with participants enables us to get detailedinformation about their views on the interview questionswhile a web survey covers a larger number of target users andprovides quantitative and statistic results

31 Pilot Usability Test

311 Methodology There are two sessions in the pilot usabil-ity test serving as the base and precondition to the latermain usability test The first session consists of a presurveywith 10 questions an interview with 5 open ended questionsand a postsurvey with 4 questions Multiple-choice questionsare common in the two surveys with easily interpreted andclassified answers The second session follows two or threedays after the first one which is composed of 10 open endedquestions The interview is designed for the record of theanswers and partly for the timing of participants installingand configuring PEK The pilot study tries to address threemajor issues as follows

(i) PQ1 after the release of PEK some users complain onGoogle Play about the difficulty of the configurationprocess Thus we would like to investigate into thefollowing two questions How easily can users installand configure PEK onto their smart devices Does thecomplicated installation and configuration processdiscourage them from using PEK

(ii) PQ2 due to the lack of interest and awareness ofprotecting privacy people may have little demand forPEK Such inference reminds us of another question


Table 2 Installation and configuration time of PEK

Participants Installation time Configuration time(seconds) (seconds)

Participant 1 2901 4579Participant 2 1500 12500

do all the users care about the security on their smartdevices

(iii) PQ3 once a user makes PEK work she will meetwith a randomized keyboard every time she choosesa password input box which takes more time thantyping in a regular QWERTY keyboard Here comesthe question do all the users agree with the point ofview that it is worth taking extra time to protect theirpasswords andor pins

312 Results for Pilot Usability Test Twomales with Androidmobile smart phones participate in the pilot usability testThey are required to install and configure PEK on theirdevices and we time them We measure how long they spendon finishing the installation and configuration and how longit takes for the randomized keyboard to successfully show upwhen participants try to input a password andor pin

Answers to Question PQ1 Users have no difficulty in findingPEK on Google Play and installation Nevertheless theydo have problems in configuring it Table 2 shows thetime of installation and configuration during the pilot testApparently both spend more time on configuration It is theresearchers who give them additional instructions and helpthem successfully configure PEKThe participants fail to finda PEK application icon and get confused when the random-ized keyboard does not show up when they log in to one oftheir accounts like an email The complicated configurationprocess frustrates the participants and discourage them fromconfiguring PEK

Answers to Question PQ2 Neither of the participants haveany security enhancements on their smart phonesThus theythink it is unnecessary to use PEK since there is no sensitiveinformation on their phones According to Participant 1using applications and services which request important orsensitive information on laptop or desktop instead of smartdevices can be regarded as his only way of the securityprecaution However both the participants admit they areamong target audience of PEK for they are educated aboutmobile security and precautionary measures

Answers to Question PQ3 After two to three days in thesecond session of the test Participant 1 and Participant 2hold different views on whether the extra time they spend isworth protecting privacy Participant 1 predicts that nobodywould prefer a randomized keyboard with no keys in thefixed position than a regular QWERTY keyboard with keysin the same position which is familiar to users Using PEKis a challenge to multitask For instance if a user is on thewalk typing in a randomized keyboard is rather difficult

Using PEK wastes time especially when the mobile phonegoes sleep again and again when users attempt to entertheir password The repeated action of entering passwordand the wasted time frustrates Participant 1 Different fromParticipant 1 Participant 2 holds positive views on the use ofPEK for its practicability and dependability He regards PEKas a hand that covers the password sparing usersrsquo trouble ofcovering with their own hands

Two observations can be made from the pilot usabilitytest

(1) The configuration of PEK is a great challenge for bothparticipants which demands more instructions onthe Google Play Store for users to follow and an iconfor them to click when opening PEK As can be seenfrom the test neither of the participants succeeds inusing PEK without the help of researchers becausethey waste time looking for a nonexistent icon

(2) Since Participant 1 mentions the difficulty of usingPEK when unlocking mobile phones with multipletasks we decide to create a separate button on theprivacy enhanced keyboard disabling PEK quickly Inthisway if a userwould rather use a regularQWERTYkeyboard than a randomized one when unlocking themobile phone the button should help him

32 Main Usability Test

321 Methodology The main usability test composed of aweb survey and a focus group usability test is based upon thefindings in the pilot test The web survey is conducted basedon the Qualtrics platform on Amazon Mechanical TurkParticipants are required to follow directions and answerquestions honestly and correctly with a bonus of one dollarThe focus group usability test involves an interview targetingparticipants who install and configure PEK on their devicesand are required to answer several questions In this test thefollowing four major issues are addressed

(i) MQ1 what are the most frequent activities of thesmart phone users If one of the most frequentactivities they do have anything to do with privacythe users should be included as our target audience

(ii) MQ2 have the smart device users already had anawareness of utilizing default security precautionsSimilar questions are covered in the pilot test suchas whether or not typical smart device users are con-cerned with the security measures on their personaldevices

(iii) MQ3 do users consider that their smart devices areproperly protected from outsider attacks

(iv) MQ4 do any smart device users think about takingmore measures to ensure security of their devices

322 Results for Main Usability Test Web Survey The mainusability test involves 2 participants in the focus groupusability test and 266 participants including 132 females and134males in theweb surveyTheir ages range from 18 to above


096

2055

1677

9761801

1306

763

1326

Automatic screen lock aftera certain amount of timeTurning off Bluetooth andorWi-Fi when not in useTurning off location servicesPasscode

PasswordSoftware updates

Other please specifyFingerprint authentication

Figure 5 Distribution of security precautions

50 years old 136 participants useAndroid devices which PEKis compatible with 123 participants use Apple devices withthe rest 7 participants using other devices The web surveyconsists of 21 questions and 266 responses as well as multiplechoice questions with open ended questions

Answers to Question MQ1 The aim of this question is tofind out whether the most frequent activities performed bymobile smart device users involve their personal sensitiveinformation Mobile banking online shopping and socialnetwork increase the possibility of sensitive personal infor-mation being stolen Figure 6 depicts the statistics from theweb survey Internet use is at the top with 8 54 ofthe web survey participants shop online 57 of them usemobile banking and 71 use social networking sites All thethree activities may contribute to personal information beingleaked and an account being hacked If participants intendto protect their information involved in the activities theyshould be a part of PEKrsquos target audience

Answers to Question MQ2 A user who has no other securityprecautions on her device is not likely to utilize PEK Whatmatters most is not the amount of security precautions butthe usersrsquo awareness of protecting their personal informationfrom the potential attacks Figure 5 illustrates the distributionof security precautions web survey participants implementon their devices At 2055 automatic screen lock after acertain amount of time is the top answer More questionstherefore arise after the results of these particular questionsAre smart device users unconcerned with security Or arethey uninformed of the security problems on the devices andthe potential attacks

013713

786

767

799

770

687595624

496

500

436

401

538

570

309

691306

Accessing emailInternet useGetting directions (GPS)Listening to musicWatching videos or moviesReadingOnline bankingTake photos andor videosOther please specify

Social networkingMake andor receive

Make andor receive

Playing gamesInstant messagingReceiving news alertsOnline shoppingMake and share

Video chat

text messages

payments (Venmo or Paypal)

phone calls

Figure 6 Answers to question MQ1

Answers to Question MQ3 This question is designed tofigure out whether or not the web survey takers are awareof the potential attacks to their own smart devices Basedon the results we can have judgment between two reasonsfor usersrsquo low awareness of security-lack of education aboutattacks and unconcern with security The answers to thequestion vary by the degree to which the web survey takersare concerned with security The top answer at 3659 isldquoprobably yesrdquo followed by ldquomayberdquo at 2927 and ldquoprobablynotrdquo at 2005 It is noteworthy that the rate of the degreeof protection on the mobile devices might not match howwell they are really protected What worries us is exactly thehigh level of certainty they show about protecting their smartdevices Figure 7 demonstrates the distribution of answersto the question of how well protected their smart devicesare

Answers to QuestionMQ4 It surprises us a lot that users showgreat interest and willingness in taking more measures toprotect their devices from attacks Despite that few of themreally implement more security precautions such a resultcould be a good beginning Figure 8 shows the distribution ofthe answers to this question 3767 of the web survey takersanswered ldquoprobably yesrdquo with 3062 of ldquomayberdquo and 1924of ldquodefinitely yesrdquo These groups of people can be potentialPEK users under the premise of ensured user experience andsecurity


1165

3659

2927

2005

244

Definitelyyes yes

Probably Maybe Probablynot

Definitelynot

000

500

1000

1500

2000

2500

3000

3500

4000

()

Figure 7 Distribution of answers to question MQ3

000

500

1000

1500

2000

2500

3000

3500

4000

4500

()

Definitelyyes yes


Definitelynot

1924

3767

3062

1084

163


323 Results for Main Usability Test Focus Group UsabilityTest Besides the web survey mentioned above the focusgroup usability test targets 2 participants They are inter-viewed at the same timewith 19 open ended questions similarto those asked in the web survey Both use mobile Androidsmart phones

(a) What three activities do you primarily do on yourmobile phone Participant Arsquos list of most frequentactivities contains using the alarm reading the newsand listening to music The top three activities Par-ticipant B performs on the mobile smart device aresendingreceiving texts taking photos and usingsocial network applications Participant B is morelikely to be a candidate for PEK than Participant ANone of the activities they listed are frequently chosenby the web survey takers

(b) What kind of security have you implemented on yourmobile phone Both the participants answered ldquonoperdquoto this question Neither has installed any defaultsecurity precautions to their smart devices

(c) Are you satisfied with the level of security on yourmobile phone Both of them give an affirmativeanswer

(d) Would you ever consider adding more security featuresto your mobile phone Surprisingly the two partici-pants are somewhat open to this question We couldinfer that they do not install any security out oflaziness Or they are confident in protecting theirprivate data from leaking when using mobile phones

(e) At this point during the interview we have bothparticipants install and configure PEK

(f) Would you recommend this application to a friendParticipant A is glad to recommend it to friends whoare concerned with security since they often showup in public Participant B thinks this application isa good recommendation to those who need moresecurity

(g) Do either of you have any suggestions about improvingthe application Participant B shows little interest inPEK He says that ldquoit can be used but I will not useitrdquo One suggestion from Participant A is to get rid ofthe large popup of a key when hitting a key He findsit really annoying that the large version of the lettercovers the whole screen leaving little space for otherletters

33 Improvements in PEK 3x We have noticed in the pilotusability test that it is the configuration process that takesparticipants long time during which they fail to find the PEKapplication icon on the smart phonesWe add an icon of PEKto the Android home screen as shown in Figure 9 so that auser can tap it and finish configuration as shown in Figure 10To set PEK as a keyboard a user can click the ldquoOpen AndroidInput Settingsrdquo

Moreover many participants think it is inconvenient touse PEK in specific circumstances since PEK cannot belearned So we take their suggestion to create a new buttonenabling them to turn onoff the randomization of PEK As isshown in Figure 11 we implement a random toggle button onthe keyboard in order that users can choose between a regularkeyboard and a randomized keyboard according to their ownwishes

4 Second Usability Testing

In 2017 summer a second two-stage usability test wasconducted by another researcher who performed interviewsand surveys The format is similar to the format of the firstusability testThefirst test is an interview-based pilot usabilitytest that is done to pinpoint issues Data collected from thepilot test is used to help form a web survey The second testthat is the survey-based main usability testing is conductedafter PEK is improved based upon the pilot study

41 Participants

Pilot Usability Test There are 12 participants 6 males and 4females for the phone based interview Ages range from 17 to54 50 of the participants are iOS users 30 are Androidusers and 20 are both iOS and Android users For thistest a Samsung S8 is provided by the interviewer for them to


Figure 9 Home screen app

Figure 10 PEK setting

complete the task For the clipboard based interview of thistest both participants are female and above the age of 50 Oneof them is an AndroidApple user and the other is a basiccellphone user The clipboard provides written instructionson the installation and configuration of PEK

Main Usability Test The main usability test had 281 par-ticipants There are 163 male and 118 female participantsParticipants range from 18 to 65 years old and are fromvarious backgrounds Figure 13 shows the age distributionAll of them are Android users because it is a requirementfor the web survey also because the PEK is only currentlyavailable for the Android platform

42 Pilot Usability Test The pilot test had all of the par-ticipants interviewed in person The interview task was toinstall and configure PEK on the Samsung S8 with minimal

Figure 11 Toggle button

help from the interviewer Participants were encouraged tothink aloud and ask any questions if needed The goal of theinterview was to find any common problems that arose whenparticipants were using the PEK Halfway through the studythere was a realization that some of the participants were notfamiliar with the Android operating system or smart phoneoperating system in general To compensate for this lack offamiliarity there was a step-by-step print-out of the wholeinstallation and configuration process of the PEK (screen byscreen)The print-out is called clipboard for participants thatdid notwant to or did not knowhow to use the Samsung S8 Inthis interview via the clipboard participants were asked whatsteps they would take to download and configure the PEKsuccessfully There were only two participants for this typeof interview If the participant answered correctly they wereallowed to proceed to the following page The participantswere also encouraged to think aloud and ask questions likethe ones in the S8 interview However if they could not getto a certain point without asking too many questions theclipboard was taken away and the test was followed by theinterviewer asking for feedback on their experience of thePEK itself

Four major issues in the pilot test are addressed and theinstallation as well as configuration time for the updated PEKis evaluated

(i) PQ11015840 have you heard of the PEK application Asshown in Figure 14 most of the participants neverheard of the PEK so an explanation is needed

(ii) PQ21015840 did you view the visuals on the Google PlayStore As shown in Figure 15 the belief of ldquonotbeing able to configure the apprdquo was drawn from theparticipants not paying attention to the visuals

(iii) PQ31015840 on a scale of 1 to 5 how comfortable are youwith operating your device Figure 16 illustrates thedistribution of the answers of the comfortability withusersrsquo own device If participants are not comfortable


Figure 12 PEK configuration app interface

or familiar with operating their own device this couldalso be a reason why they could not set up the PEK

(iv) PQ41015840 do you have security on your phone suchas a pin or password As depicted in Figure 17 ifparticipants are password or pin users they can be keycandidates to utilize the improved PEK

(v) Installation and configuration times as seen in Fig-ure 18 on average it takes everyone interviewed22 seconds to install the app and 118 seconds toconfigure the keyboard Overall it takes participantsapproximately 5 times longer to set up the keyboardcompared to their installation time

43MainUsability Test This test is formulated after commonissues are discovered by the participants in the pilot test Theissues are fixed and then a survey for only Android users ispublished Improvements to the PEK are as follows

(i) Fixing program bugs Apparently nobody wants touse an app that crashes all the time

(ii) Enhancing and adding to settings (on-screen instruc-tions for configuring the PEK) As shown in Figure 12we add the on-screen instructions in the configura-tion app and instruct the users how to configure anduse PEK

The web survey is hosted by Amazon Mechanical TurkThis survey allows the participants to install and configurethe PEK alone while leaving feedback Each participant isallotted 40 minutes to complete the survey Each participantis also compensated for their genuine and honest feedbackNewly formulated questions for the web survey are as follows

(i) Do you know how to use your smartphone If par-ticipants do not feel comfortable with operating theirsmartphone that can be part of the issue as to whythey could not configure the app

23

73

14

53

Age range

18ndash2425ndash3435ndash45

45ndash5555ndash65

Figure 13 Distribution of participant ages

NoYes

30

70

Heard of PEK

Figure 14 Distribution of answers to PQ11015840

20

80

NoSometimes

Viewed visuals on app store



20

2060

Comfortability with own device

345


50

20

30

Security

PinNoneBoth (pin amp fingerprint)


(ii) How often do you enter a password or pin on yourphone a day If the participants enter their passwordsdaily at a high frequency the PEK will be a perfect fitfor them

(iii) Did you follow the on-screen instructions after youinstalled the app to help configure the PEK Withthe new update the user would be forced to view theinstructions on how to set up the keyboard This isbetter than the visuals on the app store because usersare now obligated to look at it This is different fromthe app store previews because users are not forced toview the visuals to install the app

The web survey is broken down into two parts The firstquarter of the survey was strictly demographic questions andthe rest of the survey is about the usersrsquo experience with thePEK In this test the following tenmajor issues are addressed

(i) MQ11015840 do you understand how to use your smart-phone As shown in Figure 19 58 thoroughly

Installation time versus configuration time

Installation time (secs)Configuration time (secs)

10987654321

0

50

100

150

200

(Sec

onds

)

250

300

Figure 18 Installation and configuration time

understood 34 mostly understood 8 somewhatunderstood and less than 1 either somewhat ormostly did not understand Because of these findingsbeing very similar to the comfortability question inthe pilot test it is clear that issues with the PEK hadnothing to do with the usersrsquo understanding of theirown device

(ii) MQ21015840 on a scale of 1 to 10 howwould you rate the easeof installing the PEK app (1 being extremely hard 10being extremely easy) As can be seen in Figure 2049 of the participants rate the ease of installing thePEK 8 or higher Just like the results of the pilot testthe installation is relatively easy

(iii) MQ31015840 on a scale of 1 to 10 howwould you rate the easeof setting up the PEK (before actually using it) (again1 being extremely hard and 10 being extremely easy)As depicted in Figure 21 56 of the participants feelthat the configuration process is relatively good Thecomments for the ratings being an 8 or higher includeldquono problems at allrdquo or ldquonothingrdquo Some of the lowerrated comments about the configuration complainthat there is ldquotoo muchtoo little informationrdquo orwould like that it could ldquoshow more picturesrdquo

(iv) MQ41015840 did you use the on-screen instructions toset up the keyboard Suggestions to have on-screeninstructions from the pilot test took on a liking in themain usability test As seen in Figure 22 close to 90utilized the on-screen help for configuring the app

(v) MQ51015840 were the instructions helpful This questionis displayed if ldquoyesrdquo is selected to MQ41015840 As shownin Figure 23 99 of the participants who use theinstructions think they are either helpful or somewhathelpful Only 3 participants who belong to the 1do not think they are One of them says ldquoI am stillunable to understand how to use this There shouldbe a tutorial or user guide for the same or help toolrdquoand the others left no feedback

(vi) MQ61015840 were you able to configure the keyboardwithout any problems This question is displayed ifldquonordquo is selected for MQ41015840 As shown in Figure 24only 65 are able to successfully accomplish the setupwithout the instructions


1

1

22

95

162

Mostly does not understand

Somewhat does not

Somewhat understands

Mostly understands

Thoroughly understands

Participants understanding of their Android device

200150100500

Figure 19 Distribution of answers to MQ11015840

2 4 5

7

9

11

1315

16

18

Ease of installing the PEK app

123

456

789

10


(vii) MQ71015840 did you go back to follow the instructions forhelp or attempt to solve them yourself This questionis displayed if ldquoyesrdquo is not selected for MQ61015840 Asdepicted in Figure 25 62 are able to set up the PEKon their own successfully while the remaining 38have to turn back to the instructions The main issuefor the ones who have to return to the instructionsis locating the keyboard icon to switch keyboardsoutside of the settings

(viii) MQ81015840 the PEK is useful As depicted in Figure 2688 of the participants fall within the agree rangeSome of their comments also include ldquo[liked] theidea of PEK [and] will definitely use itrdquo ldquonothingwas confusingrdquo and ldquoeffective keyboardrdquo For theparticipants that fall into the 12 their responsesinclude ldquocould not get PEK enable[ed]rdquo and ldquo[thePEK] barely gives any predictions correctlyrdquo

(ix) MQ91015840 would you recommend the PEK to anyoneAs seen in Figure 27 67 of the participants areeither willing or definitely would recommend thePEK to others However the remaining 33 are not

1 11

2

9

10

20

27

19

10

Ease of configuring the PEK

123

456

789

10


guaranteed or will not at all This is a motivation toimprove the app even more

(x) MQ101015840 would you continue using the PEK after thissurvey As seen in Figure 28 almost half of theparticipants would continue using the app after thesurvey Reasons why others would either maybe ornot use it include ldquodifficulty using the keyboard withother languagesrdquo ldquoGoogle Play instructions were not[effective]rdquo and ldquodoes not like the idea of the appcollecting your passwordsrdquo while we explicitly notePEK does not collect any passwords

44 Summary In summation the pilot and main usabilitytest results are extremely valuable The pilot test allows themain issue of configuring the keyboard to be found All theiPhone Android and basic cellphone users are allowed toparticipate in the pilot test because we want to see if thereis a common thought process that is reoccurring across ourparticipants Surely all participants share the same thoughtthat the PEK will automatically be enabled after they turnit on in the language and input settings This makes them abit frustrated and lowers their motivation to continue using


NoYes

13

87

Utilized on-screen instructions


1

27

72

No

YesSomewhat

Were the instructions helpful


the app The main usability test narrows down our audiencestrictly to Android users Since the app is only currentlyavailable on the Google Play Store we want to test theupdated app only on the users that are familiar with thephonersquos system The main improvement of the updated appthat would directly affect consumers is the added on-screenconfiguration instructions While there are other bug fixesand code improvements this fix would directly be associatedwith our pilot test participantsrsquo configuration problem Only40 of our pilot phone based interviewees say that theywouldmight use the app in the future That number drasti-cally increases with the added instruction component to 88in theweb surveyThemajority of personal responses on theirinteraction with the PEK claim to have no issues configuringthe app and think it is easy However because all responsesdo not claim this there is still room for improvement Someof the critiques from the web survey suggested we have amore interactive instruction for configuring the keyboardIdeas of having a showcase view for the PEK setup have beenmentioned to attend to this request Some other thoughtswithin the design team are to make the keyboard availablein multiple languages to diversify the audience and increasefuture downloads

65

27

8

Successful configuration without instruction

YesSomewhatNo


23

62

15

How issues with configuration

Back to instructionsAttempted and

Attempted but wentsucceeded alone

back to instructions

were resolved


Strongly agreeSomewhat agreeSomewhat disagree

23

41

24

8 2 2

The PEK is useful

AgreeNeither agree

Strongly disagreenor disagree



24

43

24

6 3

Would you recommend the PEK

Definitely

ProbablyProbably not Might or

Definitely not

might not


12

41

47

No

YesMaybe

Would you continue to use the PEK afterthe survey


5 Related Work

Various side channel attacks against mobile devices aim toinfer a victimrsquos sensitive information for example passwordsentered on the soft keyboard of the touch-enabled screenThey can be classified into two categories internal andexternal side channel attacks In internal side channel attacksit is assumed that an attacker is able to install a malwarein a victimrsquos device and exploit diverse sensor data insidea device for example front camera and microphone [20]accelerometer [15ndash21] and ambient light sensor [25] In exter-nal side channel attacks an attacker can exploit side channelsoutside a device Three example external side channel attacksare residue-based attacks [1ndash5]Wi-Fi-based attacks [26] andvision-based attacks [6ndash12 27 28]

Intensive research efforts have been made to mitigatethese side channel attacks in the past decades For exampleHirsch [29 30] invented a secure keypad input terminalto randomly display the ten numerical digits 0 through9 McIntyre et al [31] proposed a random PIN pad todisplay a random numerical keypad layout however for

usage purpose it still preserved the numerical order in thehorizontal or vertical direction Moreover they adopted aregular hexagon background pattern for each key whichsignificantly increases the number of possible key locationsHoanca and Mock [32] investigated the arrangements forsixteen characters on a 4 times 4 screen to randomize thedistribution in the vertical horizontal spiraling diagonallyand other directionswhile preserving the lexicographic orderShin [22] first generated a 10-button random keypad byrandomly arranging the numbers and letters together Theuser should remember the mapping relationship betweenthe letters and numbers Then a randomized letter keypadis displayed so that the user can recall the letters corre-sponding to the numbers of her password and input thepassword Lee [33] proposed a method to randomly displayten numerical digits in arrays matrix a wheel format orwith different key background colors background patternsshapes and fonts Kim [34] presented a scheme to first select5 random numbers out of 10 and displayed them in a 12-button keypad layout Then by pressing a ldquonextrdquo buttonthe remaining 5 numbers can be randomly displayed inthe keypad In comparison our randomized keyboard canrandomly arrange the 26-letter keyboard and automaticallyidentify the type of the input box Therefore our privacyenhancing keyboard can provide both privacy protection andusability

Randomized keyboards are often applied in online bank-ing apps However they are application-level randomizedkeyboards that can only be used in a particular applicationThe PEK is a system-level Android keyboard that can beused for any application including screen lock email andbanking Moreover it can sense the property of the inputbox to pop up an appropriate keyboard so as to improve theuser experience More importantly we are the first to designa generic randomized keyboard for Android

6 Conclusion

This paper presents a full-scale usability testing of a genericAndroid privacy enhancing keyboard (PEK) which canprevent various attacks against touch-enabled devices frominferring user pins or passwords We perform an iterativetwo-round two-stage usability test including pilot usabilitytests and main usability tests for improving PEK for broadadoption Based on the findings of the two usability testsin the first usability test we implement new features inthe current PEK After the iterative improvement effortsmost users find our app easy to use and install Howeverthe usability test demonstrates the worrisome phenomenathat many users blindly trust their phones for security orare not much concerned with the possible breaches Thesephenomena demonstrate the human factor that contributesto the vulnerabilities of the cyber space

Disclosure

Any opinions findings conclusions and recommendationsin this paper are those of the authors and do not necessarilyreflect the views of the funding agencies


Conflicts of Interest

There are no conflicts of interest in the manuscript

Acknowledgments

This work was supported in part by National Key RampDProgram of China under Grant 2017YFB1003000 byNational Natural Science Foundation of China underGrants 61502100 61532013 61402104 61572130 6160211161632008 and 61320106007 by US NSF Grants 14610601642124 1547428 and CNS 1350145 by University Systemof Maryland Fund by Ant Financial Research Fund byJiangsu Provincial Natural Science Foundation of Chinaunder Grants BK20150637 and BK20140648 by JiangsuProvincial Key Technology RampD Program under GrantBE2014603 by Jiangsu Provincial Key Laboratory of Net-work and Information Security under Grant BM2003201by Key Laboratory of Computer Network and InformationIntegration of Ministry of Education of China under Grant93K-9 and by Collaborative Innovation Center of NovelSoftware Technology and Industrialization

References

[1] M Zalewski ldquoCracking safes with thermal imagingrdquo 2005httplcamtufcoredumpcxtsafe

[2] A J Aviv K Gibson E Mossop M Blaze and J M SmithldquoSmudge attacks on smartphone touch screensrdquo in Proceedingsof the Workshop on Offensive Technology WOOT 2010

[3] K Mowery S Meiklejohn and S Savage ldquoHeat of the momentcharacterizing the efficacy of thermal camera-based attacksrdquo inProceedings of theWorkshop on Offensive Technologies (WOOT)2011

[4] Y Zhang P Xia J Luo Z Ling B Liu and X Fu ldquoFingerprintattack against touch-enabled devicesrdquo in Proceedings of the2nd ACM Workshop on Security and Privacy in Smartphonesand Mobile Devices (SPSM rsquo12) pp 57ndash68 Raleigh NC USAOctober 2012

[5] Y Abdelrahman M Khamis S Schneegass and F Alt ldquoStaycool understanding thermal attacks on mobile-based userauthenticationrdquo in Proceedings of 35th Annual CHI Conferenceon Human Factors in Computing Systems (CHI) pp 3751ndash3763Denver CO USA May 2017

[6] M Backes M Duermuth and D Unruh ldquoCompromisingreflections - or - how to read lcdmonitors around the cornerrdquo inProceedings of the 29th IEEE Symposium on Security and Privacy(SampP) 2008

[7] M Backes T Chen M D1rmuth H P A Lensch andMWelkldquoTempest in a teapot Compromising reflections revisitedrdquo inProceedings of the 30th IEEE Symposium on Security and Privacy(SampP) 2009

[8] D BalzarottiMCova andGVigna ldquoClearshot eavesdroppingon keyboard input from videordquo in Proceedings of the 29th IEEESymposium on Security and Privacy (SampP) 2008

[9] F Maggi A Volpatto S Gasparini G Boracchi and S ZaneroldquoA fast eavesdropping attack against touchscreensrdquo in Proceed-ings of the 2011 7th International Conference on InformationAssurance and Security IAS 2011 pp 320ndash325MelakaMalaysiaDecember 2011

[10] Q Yue Z Ling X Fu B Liu W Yu and W Zhao ldquoMy googleglass sees your passwordsrdquo in Proceedings of the Black Hat USA2014

[11] Q Yue Z Ling X Fu B Liu K Ren and W Zhao ldquoBlindrecognition of touched keys on mobile devicesrdquo in Proceedingsof the 21st ACM Conference on Computer and CommunicationsSecurity CCS 2014 pp 1403ndash1414 November 2014

[12] J Sun X Jin Y Chen J Zhang R Zhang and Y ZhangldquoVisible Video-assisted keystroke inference from tablet back-side motionrdquo in Proceedings of the 23rd ISOC Network andDistributed System Security Symposium (NDSS) 2016

[13] L Zhang Z Cai and X Wang ldquoFakeMask A Novel PrivacyPreserving Approach for Smartphonesrdquo IEEE Transactions onNetwork and Service Management vol 13 no 2 pp 335ndash3482016

[14] Z Cai Z He X Guan and Y Li ldquoCollective data-sanitizationfor preventing sensitive information inference attacks in socialnetworksrdquo IEEE Transactions on Dependable and Secure Com-puting 2016

[15] L Cai and H Chen ldquoTouchLogger Inferring keystrokes ontouch screen from smartphone motionrdquo in Proceedings of the6th USENIX Workshop on Hot Topics in Security (HotSec) 2011

[16] Z Xu K Bai and S Zhu ldquoTapLogger inferring user inputs onsmartphone touchscreens using on-board motion sensorsrdquo inProceedings of the 5th ACM Conference on Security and Privacyin Wireless and Mobile Networks pp 113ndash124 Tucson ArizUSA April 2012

[17] E Owusu J Han S Das A Perrig and J Zhang ldquoACCessorypassword inference using accelerometers on smartphonesrdquo inProceedings of the Proceeding of the 13th Workshop on MobileComputing Systems and Applications (HotMobile rsquo12) no 9 NewYork NY USA February 2012

[18] E Miluzzo A Varshavsky S Balakrishnan and R R Choud-hury ldquoTapprints your finger taps have fingerprintsrdquo in Pro-ceedings of the 10th International Conference on Mobile SystemsApplications and Services (MobiSys rsquo12) pp 323ndash336 Amble-side UK June 2012

[19] A J Aviv B Sapp M Blaze and J M Smith ldquoPracticality ofaccelerometer side channels on smartphonesrdquo in Proceedingsof the 28th Annual Computer Security Applications Conference(ACSAC rsquo12) pp 41ndash50 ACM Orlando Fla USA December2012

[20] L Simon and R Anderson ldquoPin skimmer Inferring pinsthrough the camera and microphonerdquo in Proceedings of the 4thACM Workshop on Security and Privacy in Smartphones andMobile Devices (SPSM) 2013

[21] D Ping X Sun andBMao ldquoTextlogger inferring longer inputson touch screen using motion sensorsrdquo in Proceedings of the 7thACMConference on Security and Privacy inWireless andMobileNetworks (WiSec) 2015

[22] H-S Shin ldquoDevice and method for inputting password usingrandom keypadrdquo United States Patent No 7698563 2010

[23] OMRONSOFTWARECo LtdOpenwnn 2012 httpssource-forgenetulluctme722-cmci890e9a90d9a7fe5f0243b9392ea-a787d1381e987treepackagesinputmethodsOpenWnn

[24] Z LingM Borgeest C Sano et al ldquoA case study of usable secu-rity Usability testing of android privacy enhancing keyboardrdquoin Proceedings of the 12th International Conference on WirelessAlgorithms Systems and Applications (WASA) 2017

[25] R Spreitzer ldquoPin skimming Exploiting the ambient-light sen-sor in mobile devicesrdquo in Proceedings of the 4th ACMWorkshop


on Security and Privacy in Smartphones and Mobile Devices(SPSM) 2014

[26] M Li Y Meng J Liu et al ldquoWhen CSI meets public WiFiInferring your mobile phone password via WiFi signalsrdquo inProceedings of the 23rd ACM Conference on Computer andCommunications Security CCS 2016 pp 1068ndash1079 October2016

[27] Z Li Q Yue C SanoW Yu andX Fu ldquo3D vision attack againstauthenticationrdquo in Proceedings of the ICC 2017 - 2017 IEEEInternational Conference on Communications pp 1ndash6 ParisFrance May 2017

[28] K Jin S Fang C Peng et al ldquoVivisnoop Someone is snoopingyour typing without seeing itrdquo in Proceedings of the IEEEConference on Communications and Network Security (CNS)2017

[29] S B Hirsch ldquoSecure keyboard input terminalrdquo United StatesPatent No 4333090 1982

[30] S B Hirsch ldquoSecure input systemrdquo United States Patent No4479112 1982

[31] K EMcIntyre J F Sheets D A J Gougeon CWWatson K PMorlang and D Faoro ldquoMethod for secure pin entry on touchscreen displayrdquo United States Patent No 6549194 2003

[32] B Hoanca and K Mock ldquoScreen oriented technique for reduc-ing the incidence of shoulder surfingrdquo in Proceedings of the 2005International Conference on Security andManagement SAMrsquo05pp 334ndash340 June 2005

[33] C Lee ldquoSystem andmethod for secure data entryrdquo United StatesPatent Application Publication 2011

[34] I Kim ldquoKeypad against brute force attacks on smartphonesrdquoIET Information Security vol 6 no 2 pp 71ndash76 2012

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014


Active and Passive Electronic Components

Control Scienceand Engineering

Journal of


International Journal of

RotatingMachinery


Hindawi Publishing Corporation httpwwwhindawicom

Journal of

Volume 201

Submit your manuscripts athttpswwwhindawicom

VLSI Design



Shock and Vibration


Civil EngineeringAdvances in

Acoustics and VibrationAdvances in



Electrical and Computer Engineering

Journal of

Advances inOptoElectronics


Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

SensorsJournal of


Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014


Chemical EngineeringInternational Journal of Antennas and

Propagation




Navigation and Observation



DistributedSensor Networks



Nonpassword

Password

Inspect the propertyof the Input box

Construct a randomkeyboard

Display the keyboard

Construct a normalkeyboard

Figure 1 Workflow of PEK constructing a keyboard

We are the first to design a generic randomized keyboardfor the Android system though the idea of randomizingthe key layout is not new [22] One version of PEK isimplemented as a third party keyboard for Android and canreplace the system keyboard once it is installed Thereforeonce PEK is chosen as the default keyboard it can be usedby any app

After our presentation at Black Hat USA [10] we releasedPEK as a free Android app to Google Play in August 2014Until the time of writing it has been downloaded 2352times We have released 7 versions with corrected bugs andimproved interface Of them PEK 10 is based on an Androidcode example PEK 2xx and later versions are based onOpenWnn [23] with fixed bugs The current version of PEKis 3233

For the purpose of usable security and privacy wedesigned an iterative usability test to evaluate the user experi-ence of PEK and to explore the reason for the lukewarmnessof using PEK Each iteration of usability test is a two-stagestudy a pilot study and a main study We randomly selectparticipants to reflect different behaviors of Android usersThe pilot study uses surveys and interviews and involvesa small number of people for us to understand potentialusability issues of PEK We then add features to PEK basedon the results of the pilot study and use the main studythrough a web survey hosted at Amazon Mechanical Turk tounderstand the usability of the improved PEK For surveysperformed throughAmazonMechanical Turk we contact theusers to make sure that they install PEK and complete oursurvey questions We performed two rounds of usability testin 2016 summer and 2017 summer respectively After the tworounds of usability testing and app improvements most usersreport the app is easy to install configure and use The otherobservations from the usability testing are also educational(1) convenience plays a critical role when users select an inputmethod (2) peoplemay think those attacks that PEKpreventsare remote from them and the risk from those attacks is smallThe usability test demonstrates the worrisome phenomenathat many users blindly trust their phones for security orare not much concerned with the possible breaches Thesephenomena demonstrate the human factor that contributesto the vulnerabilities of the cyber space

An early version of the usability testing of PEK ispresented in [24] In the conference version we design a two-stage usability test and perform the first usability testing toevaluate the user experience of PEK In this journal versionwe present the detailed algorithms and techniques used toimplement our privacy enhancing keyboard and the secondusability testing to further evaluate the user experience of

updated PEK Related work and new experimental evaluationare presented in this extended version

The rest of this paper is organized as follows The designand the implementation of the third party keyboard of PEKwill be introduced in Section 2 In Section 3 we present themethodology applied in the usability test and the results fromthe first usability testing The second usability testing andresults are presented in Section 4We review the most relatedwork in Section 5 and conclude this paper in Section 6

2 Privacy Enhancing Keyboard

We introduce the basic idea of the privacy enhancing key-board in this section Then we present the technique detailsof PEK implementation and elaborate how to install andconfigure the PEK for Android

21 Basic Idea To thwart diverse attacks such as residue-based attacks computer vision-based attacks and sensor-based attacks we shuffle the positions of the keys of asoftware on a touch screen to ensure that a randomizedkeyboard is shown to a user every time she enters a passwordTherefore the particular keys will not be easily inferred eventhough there is finger oily or thermal residue left on thescreen Moreover vision-based attacks do not work since atouched position by a finger no longer matches a fixed keyThe vibration and orientation information provided by anaccelerometer does not help establish profiles for particularkeys

Figure 1 shows the basic idea and the workflow of PEKconstructing a keyboard when a user touches an input boxThe first step is to inspect the property of this input box tosee whether it is a nonpassword or password input box AQWERTY keyboard is shown if a nonpassword input box isidentified If a password input box is identified we parse theproperty of the keys from a XML file which stores the layoutof the keyboard and then change the label and value of thekeys so that the positions of the keys are shuffled

There are two versions of PEK that we implemented (1)One version is a third party keyboard that can be imple-mented as an Android App for Google Play and installed toan Android device Once enabled PEK runs as an Androidservice in the background A user can use the Android inputsetting menu to enable PEK while the location of the inputsetting menu could vary from device to device Before PEK30 it is the userrsquos responsibility to find the input settingmenu according to the generic introductions we provided atGoogle Play (2) We are also able to revise the source code ofthe Android system default keyboard and the unlock screen











Figure 2 PEK






Normalkeyboard

Shuffledkeys








5

10

15

20

25

30

35

40

45

Inpu

t tim

e (se

c)





























096

2055

1677

9761801

1306

763

1326








013713

786

767

799

770

687595624

496

500

436

401

538

570

309

691306



Make andor receive


Video chat

text messages


phone calls





1165

3659

2927

2005

244

Definitelyyes yes


Definitelynot

000

500

1000

1500

2000

2500

3000

3500

4000

()


000

500

1000

1500

2000

2500

3000

3500

4000

4500

()

Definitelyyes yes


Definitelynot

1924

3767

3062

1084

163














41 Participants
























23

73

14

53

Age range


45ndash5555ndash65


NoYes

30

70

Heard of PEK


20

80

NoSometimes




20

2060


345


50

20

30

Security









10987654321

0

50

100

150

200

(Sec

onds

)

250

300









1

1

22

95

162


Somewhat does not


Mostly understands



200150100500


2 4 5

7

9

11

1315

16

18


123

456

789

10





1 11

2

9

10

20

27

19

10


123

456

789

10






NoYes

13

87



1

27

72

No

YesSomewhat




65

27

8


YesSomewhatNo


23

62

15





were resolved



23

41

24

8 2 2

The PEK is useful

AgreeNeither agree




24

43

24

6 3


Definitely


Definitely not

might not


12

41

47

No

YesMaybe



5 Related Work





6 Conclusion


Disclosure





Acknowledgments


References





































RoboticsJournal of





Journal of



RotatingMachinery



Journal of

Volume 201


VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation



















Figure 2 PEK






Normalkeyboard

Shuffledkeys








5

10

15

20

25

30

35

40

45

Inpu

t tim

e (se

c)





























096

2055

1677

9761801

1306

763

1326








013713

786

767

799

770

687595624

496

500

436

401

538

570

309

691306



Make andor receive


Video chat

text messages


phone calls





1165

3659

2927

2005

244

Definitelyyes yes


Definitelynot

000

500

1000

1500

2000

2500

3000

3500

4000

()


000

500

1000

1500

2000

2500

3000

3500

4000

4500

()

Definitelyyes yes


Definitelynot

1924

3767

3062

1084

163














41 Participants
























23

73

14

53

Age range


45ndash5555ndash65


NoYes

30

70

Heard of PEK


20

80

NoSometimes




20

2060


345


50

20

30

Security









10987654321

0

50

100

150

200

(Sec

onds

)

250

300









1

1

22

95

162


Somewhat does not


Mostly understands



200150100500


2 4 5

7

9

11

1315

16

18


123

456

789

10





1 11

2

9

10

20

27

19

10


123

456

789

10






NoYes

13

87



1

27

72

No

YesSomewhat




65

27

8


YesSomewhatNo


23

62

15





were resolved



23

41

24

8 2 2

The PEK is useful

AgreeNeither agree




24

43

24

6 3


Definitely


Definitely not

might not


12

41

47

No

YesMaybe



5 Related Work





6 Conclusion


Disclosure





Acknowledgments


References





































RoboticsJournal of





Journal of



RotatingMachinery



Journal of

Volume 201


VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation











Normalkeyboard

Shuffledkeys








5

10

15

20

25

30

35

40

45

Inpu

t tim

e (se

c)





























096

2055

1677

9761801

1306

763

1326








013713

786

767

799

770

687595624

496

500

436

401

538

570

309

691306



Make andor receive


Video chat

text messages


phone calls





1165

3659

2927

2005

244

Definitelyyes yes


Definitelynot

000

500

1000

1500

2000

2500

3000

3500

4000

()


000

500

1000

1500

2000

2500

3000

3500

4000

4500

()

Definitelyyes yes


Definitelynot

1924

3767

3062

1084

163














41 Participants
























23

73

14

53

Age range


45ndash5555ndash65


NoYes

30

70

Heard of PEK


20

80

NoSometimes




20

2060


345


50

20

30

Security









10987654321

0

50

100

150

200

(Sec

onds

)

250

300









1

1

22

95

162


Somewhat does not


Mostly understands



200150100500


2 4 5

7

9

11

1315

16

18


123

456

789

10





1 11

2

9

10

20

27

19

10


123

456

789

10






NoYes

13

87



1

27

72

No

YesSomewhat




65

27

8


YesSomewhatNo


23

62

15





were resolved



23

41

24

8 2 2

The PEK is useful

AgreeNeither agree




24

43

24

6 3


Definitely


Definitely not

might not


12

41

47

No

YesMaybe



5 Related Work





6 Conclusion


Disclosure





Acknowledgments


References





































RoboticsJournal of





Journal of



RotatingMachinery



Journal of

Volume 201


VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation































096

2055

1677

9761801

1306

763

1326








013713

786

767

799

770

687595624

496

500

436

401

538

570

309

691306



Make andor receive


Video chat

text messages


phone calls





1165

3659

2927

2005

244

Definitelyyes yes


Definitelynot

000

500

1000

1500

2000

2500

3000

3500

4000

()


000

500

1000

1500

2000

2500

3000

3500

4000

4500

()

Definitelyyes yes


Definitelynot

1924

3767

3062

1084

163














41 Participants
























23

73

14

53

Age range


45ndash5555ndash65


NoYes

30

70

Heard of PEK


20

80

NoSometimes




20

2060


345


50

20

30

Security









10987654321

0

50

100

150

200

(Sec

onds

)

250

300









1

1

22

95

162


Somewhat does not


Mostly understands



200150100500


2 4 5

7

9

11

1315

16

18


123

456

789

10





1 11

2

9

10

20

27

19

10


123

456

789

10






NoYes

13

87



1

27

72

No

YesSomewhat




65

27

8


YesSomewhatNo


23

62

15





were resolved



23

41

24

8 2 2

The PEK is useful

AgreeNeither agree




24

43

24

6 3


Definitely


Definitely not

might not


12

41

47

No

YesMaybe



5 Related Work





6 Conclusion


Disclosure





Acknowledgments


References





































RoboticsJournal of





Journal of



RotatingMachinery



Journal of

Volume 201


VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation










1165

3659

2927

2005

244

Definitelyyes yes


Definitelynot

000

500

1000

1500

2000

2500

3000

3500

4000

()


000

500

1000

1500

2000

2500

3000

3500

4000

4500

()

Definitelyyes yes


Definitelynot

1924

3767

3062

1084

163














41 Participants
























23

73

14

53

Age range


45ndash5555ndash65


NoYes

30

70

Heard of PEK


20

80

NoSometimes




20

2060


345


50

20

30

Security









10987654321

0

50

100

150

200

(Sec

onds

)

250

300









1

1

22

95

162


Somewhat does not


Mostly understands



200150100500


2 4 5

7

9

11

1315

16

18


123

456

789

10





1 11

2

9

10

20

27

19

10


123

456

789

10






NoYes

13

87



1

27

72

No

YesSomewhat




65

27

8


YesSomewhatNo


23

62

15





were resolved



23

41

24

8 2 2

The PEK is useful

AgreeNeither agree




24

43

24

6 3


Definitely


Definitely not

might not


12

41

47

No

YesMaybe



5 Related Work





6 Conclusion


Disclosure





Acknowledgments


References





































RoboticsJournal of





Journal of



RotatingMachinery



Journal of

Volume 201


VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation































23

73

14

53

Age range


45ndash5555ndash65


NoYes

30

70

Heard of PEK


20

80

NoSometimes




20

2060


345


50

20

30

Security









10987654321

0

50

100

150

200

(Sec

onds

)

250

300









1

1

22

95

162


Somewhat does not


Mostly understands



200150100500


2 4 5

7

9

11

1315

16

18


123

456

789

10





1 11

2

9

10

20

27

19

10


123

456

789

10






NoYes

13

87



1

27

72

No

YesSomewhat




65

27

8


YesSomewhatNo


23

62

15





were resolved



23

41

24

8 2 2

The PEK is useful

AgreeNeither agree




24

43

24

6 3


Definitely


Definitely not

might not


12

41

47

No

YesMaybe



5 Related Work





6 Conclusion


Disclosure





Acknowledgments


References





































RoboticsJournal of





Journal of



RotatingMachinery



Journal of

Volume 201


VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation










20

2060


345


50

20

30

Security









10987654321

0

50

100

150

200

(Sec

onds

)

250

300









1

1

22

95

162


Somewhat does not


Mostly understands



200150100500


2 4 5

7

9

11

1315

16

18


123

456

789

10





1 11

2

9

10

20

27

19

10


123

456

789

10






NoYes

13

87



1

27

72

No

YesSomewhat




65

27

8


YesSomewhatNo


23

62

15





were resolved



23

41

24

8 2 2

The PEK is useful

AgreeNeither agree




24

43

24

6 3


Definitely


Definitely not

might not


12

41

47

No

YesMaybe



5 Related Work





6 Conclusion


Disclosure





Acknowledgments


References





































RoboticsJournal of





Journal of



RotatingMachinery



Journal of

Volume 201


VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation










1

1

22

95

162


Somewhat does not


Mostly understands



200150100500


2 4 5

7

9

11

1315

16

18


123

456

789

10





1 11

2

9

10

20

27

19

10


123

456

789

10






NoYes

13

87



1

27

72

No

YesSomewhat




65

27

8


YesSomewhatNo


23

62

15





were resolved



23

41

24

8 2 2

The PEK is useful

AgreeNeither agree




24

43

24

6 3


Definitely


Definitely not

might not


12

41

47

No

YesMaybe



5 Related Work





6 Conclusion


Disclosure





Acknowledgments


References





































RoboticsJournal of





Journal of



RotatingMachinery



Journal of

Volume 201


VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation










NoYes

13

87



1

27

72

No

YesSomewhat




65

27

8


YesSomewhatNo


23

62

15





were resolved



23

41

24

8 2 2

The PEK is useful

AgreeNeither agree




24

43

24

6 3


Definitely


Definitely not

might not


12

41

47

No

YesMaybe



5 Related Work





6 Conclusion


Disclosure





Acknowledgments


References





































RoboticsJournal of





Journal of



RotatingMachinery



Journal of

Volume 201


VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation










24

43

24

6 3


Definitely


Definitely not

might not


12

41

47

No

YesMaybe



5 Related Work





6 Conclusion


Disclosure





Acknowledgments


References





































RoboticsJournal of





Journal of



RotatingMachinery



Journal of

Volume 201


VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation












Acknowledgments


References





































RoboticsJournal of





Journal of



RotatingMachinery



Journal of

Volume 201


VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation




















RoboticsJournal of





Journal of



RotatingMachinery



Journal of

Volume 201


VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation









privacy enhancing keyboard: design, implementation, and usability...

Documents