Master Thesis

Computer Science

Thesis no: MCS:2011:21

Sept. 2011

A Comparative Evaluation of Usabilityfor the iPhone and iPad

Muhammad Azam

Luqman Ahmad

School of Computing

Blekinge Institute of Technology

SE-371 79 Karlskrona

Sweden

1

This thesis is submitted to the School of Computing at Blekinge Institute of Technology in partial

ful�llment of the requirements for the degree of Master of Science in Computer Science. The

thesis is equivalent to 20 weeks of full time studies.

Contact Information:

Author(s):

Muhammad Azam

Address: Gyllenstjärnas Väg 18, 20

Post code 37140 Karlskrona,

Sweden

Email: azambth72@gmail.com

Luqman Ahmad

Address: Gyllenstjärnas Väg 18, 20

Post code 37140 Karlskrona,

Sweden

E-mail: luqman.khan83@gmail.com

University advisor:

Dr. Veronica Sundstedt

School of Computing

School of Computing

Blekinge Institute of Technology Internet : www.bth.se/com

SE-371 79 Karlskrona Phone : +46 455 38 50 00

Sweden Fax : +46 455 38 50 57

2

ACKNOWLEDGMENTS

In the name of Allah, the Most Bene�cent, the Most Merciful.

First and foremost we would like to thank Allah the Almighty, Who blessed us courage

and devotion for this study.

We heartily thank our supervisor Dr. Veronica Sundstedt, who encouraged us and

gave guidance throughout this research work. Her constructive suggestions, directions

and invaluable advice made us capable to complete this thesis. We would also like to

express our appreciation to those who participated and dedicated their valuable time for

these studies, without their participation these studies would not have been feasible.

Last but not least, we express our gratitude to our parents and family members, for

providing us untiring support and prayers during this research work.

3

Abstract

Many everyday systems and products seem to be designed with little regard to

usability. This leads to the frustration, wasted time and errors. So the usability of

the product is important for its survival in the market.

In many previous studies the usability evaluation of the iPhone and iPad carried

out individually and very little work has been done on the comparative usability

evaluation. However, there was not any study conducted on the comparative us-

ability evaluation and measuring the performance of the iPhone versus iPad in a

controlled environment.

In this research work, the authors performed the comparative usability evaluation

and measured the performances of the iPhone and iPad on the selected applications

by considering the young users as well as the elderly users. Another objective of this

study is to identify the usability issues in performances of the iPhone and iPad.

A survey and experiment techniques were used to achieve the de�ned objectives.

The survey questionnaire consisted of 42 statements that presented the di�erent

usability aspects. The objectives of the survey study were to validate the identi�ed

issues from the literature study, identify new issues and measure the signi�cant

di�erence in user opinions for the iPhone and iPad. However, the experiment studies

helped to measure the performance signi�cances between the devices against the

three user groups (novice user, experienced user, elderly user) and among the groups

over the devices. Further, objective was to measure the satisfaction level of the

participated users against the iPhone and iPad.

The experiment was performed in a controlled environment. Total six tasks (two

tasks per application) were de�ned and each participant performed the same task on

both devices. Generally the authors found that the participants performed better

on the iPad with lower error rates as compare to the iPhone.

Keywords: Usability, Usability Evaluation, Touch Screen, Smart-phone, iPhone,

iPad, performance.

4

Contents

1 INTRODUCTION 10

1.1 Aim and Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

1.2 Research Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

1.3 Study Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

1.4 Thesis Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

2 BACKGROUND 14

2.1 What is Usability? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

2.2 Usability Measurement . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

2.3 Performance Measuring . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

2.4 Usability Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

2.5 Touch Screen and Smart Phones . . . . . . . . . . . . . . . . . . . . . . . . 17

2.6 The iPhone and the iPad . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

2.7 Features of the iPhone and iPad . . . . . . . . . . . . . . . . . . . . . . . 18

2.7.1 Multi-touching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

2.7.2 Operating System (iOS 4) . . . . . . . . . . . . . . . . . . . . . . . 19

2.7.3 Applications (Apps) Store . . . . . . . . . . . . . . . . . . . . . . . 19

2.8 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

2.9 Issues in the iPhone and iPad . . . . . . . . . . . . . . . . . . . . . . . . . 20

2.9.1 Consistency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

2.9.2 A�ordance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

2.9.3 Small Search Box and Missing Information . . . . . . . . . . . . . . 21

2.9.4 Invisible Control and Missing Instructions . . . . . . . . . . . . . . 22

3 RESEARCH METHODOLOGY 23

3.1 Literature Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

3.2 Survey Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

3.3 Experimental Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

3.3.1 Participant Selection . . . . . . . . . . . . . . . . . . . . . . . . . . 27

3.3.2 Variables Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

3.3.3 Hypotheses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

3.3.4 Task De�nition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

3.3.5 Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

3.3.6 Experimental Material . . . . . . . . . . . . . . . . . . . . . . . . . 32

3.3.7 Post Test Questionnaire . . . . . . . . . . . . . . . . . . . . . . . . 33

3.3.8 Pilot Studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

3.4 Evaluation Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

3.4.1 Mann Whitney Test . . . . . . . . . . . . . . . . . . . . . . . . . . 33

5

3.4.2 Normality Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

3.4.3 Analysis of Variances (ANOVA) . . . . . . . . . . . . . . . . . . . . 34

3.4.4 Unpaired T-test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

3.4.5 Paired T-test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

3.4.6 Wilcoxon Matched Paired Signed Ranks Test . . . . . . . . . . . . 35

3.4.7 Error Bars . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

3.5 Validity Threats . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

3.5.1 Internal Validity Threats . . . . . . . . . . . . . . . . . . . . . . . . 36

3.5.2 External Validity Threats . . . . . . . . . . . . . . . . . . . . . . . 36

3.5.3 Constructive Validity Threats . . . . . . . . . . . . . . . . . . . . . 36

3.5.4 Statistical Conclusion Validity Threats . . . . . . . . . . . . . . . . 36

4 STATISTICAL ANALYSIS, RESULTS ANDDISCUSSIONS 37

4.1 Survey . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

4.2 Demographics Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

4.3 Usability and Preference Testing . . . . . . . . . . . . . . . . . . . . . . . . 37

4.3.1 System Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

4.3.2 Touch Screen Gestures Results . . . . . . . . . . . . . . . . . . . . . 41

4.3.3 Keypad Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

4.3.4 Applications Results . . . . . . . . . . . . . . . . . . . . . . . . . . 46

4.3.5 Participant Comments . . . . . . . . . . . . . . . . . . . . . . . . . 50

4.4 Measuring Usability Attributes . . . . . . . . . . . . . . . . . . . . . . . . 51

4.4.1 Task Time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

4.4.2 Accuracy Measurement . . . . . . . . . . . . . . . . . . . . . . . . 61

4.4.3 Satisfaction Results . . . . . . . . . . . . . . . . . . . . . . . . . . . 64

4.4.4 Participant Comments . . . . . . . . . . . . . . . . . . . . . . . . . 68

5 CONCLUSIONS AND FUTURE WORK 70

5.1 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70

5.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

6 APPENDICES 73

6.1 Appendix A Survey Questionnaire . . . . . . . . . . . . . . . . . . 73

6.2 Appendix B Survey Questionnaire Results . . . . . . . . . . . . . 76

6.3 Appendix C Survey Questionnaire p-values . . . . . . . . . . . . . 78

6.4 Appendix D Post Test Questionnaire . . . . . . . . . . . . . . . . 80

6.5 Appendix E Task Time on the iPhone and iPad . . . . . . . . . . 82

6

List of Figures

1 The Overall Study Process. . . . . . . . . . . . . . . . . . . . . . . . . . . 12

2 Shackel's Usability De�nition [21]. . . . . . . . . . . . . . . . . . . . . . . . 14

3 Nielsen's Usability De�nition [21]. . . . . . . . . . . . . . . . . . . . . . . . 15

4 ISO 9241-11 Usability De�nition [21]. . . . . . . . . . . . . . . . . . . . . . 16

5 Di�erent Consistency Issues in SMS and Mobile Voip Applications. . . . . 20

6 A�ordance Issues in the Weather Applications of the iPhone and iPad. . . 21

7 Small Search Box Issues in the Applications of the iPhone and iPad. . . . . 21

8 Invisible Control and Missing Instructions Issues in the Twitter and Face-

book Application on the iPhone and iPad. . . . . . . . . . . . . . . . . . . 22

9 Presentation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

10 System. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

11 Touch Screen. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

12 Keypad. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

13 Applications. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

14 Participants Selection for the Experiment. . . . . . . . . . . . . . . . . . . 27

15 Laboratory Setup for User Trials. . . . . . . . . . . . . . . . . . . . . . . . 32

16 Participant Group Levels Comparison Across the Device. . . . . . . . . . . 34

17 Device Levels Comparison Across the Participant. . . . . . . . . . . . . . . 35

18 Percentage Graph Results of the Degree of Agreement Across Each System

Statement on the iPhone. St. Represents Strongly. . . . . . . . . . . . . . 41

19 Percentage Results of the Degree of Agreement Across Each System State-

ment of the iPad. St. Represents Strongly. . . . . . . . . . . . . . . . . . . 41

20 Graphs Show Percentage Results of the Degree of Agreement Across the

Each Statement of the Touch Screen Gestures on the iPhone and iPad

Respectively. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

21 Graph Shows Percentage Results of the Degree of Agreement Across the

Each Statement of the Keypad on the iPhone. . . . . . . . . . . . . . . . 45

22 Graph Shows Percentage Results of the Degree of Agreement Across the

Each Statement of the Keypad on the iPad. . . . . . . . . . . . . . . . . . 46

23 Percentage Graph Results of the Degree of Agreement Across Each Appli-

cations Statement on the iPhone. . . . . . . . . . . . . . . . . . . . . . . . 49

24 Percentage Graph Results of the Degree of Agreement Across the Each

Statement of the Applications on the iPad. . . . . . . . . . . . . . . . . . 49

25 Device Levels Comparison Graph of the Performance Measurement of the

Participant Group Levels for the Task Facebook Login. . . . . . . . . . . . 54

26 Device Levels Comparison Graph of the Performance Measurement of the

Participant Group Levels for the Task Send a Message. . . . . . . . . . . . 55

7

27 Device Levels Comparison Graph of the Performance Measurement of the

Participant Group Levels for the Task Location Identi�cation. . . . . . . . 56

28 Device Levels Comparison Graph of the Performance Measurement of the

Participant Group Levels for the Task Location Close View. . . . . . . . . 58

29 Device Levels Comparison Graph of the Performance Measurement of the

Participant Group Levels for the Task New Note. . . . . . . . . . . . . . . 59

30 Device Levels Comparison Graph of the Performance Measurement of the

Participant Group Levels for the Task Evernote Logout. . . . . . . . . . . 60

31 Novice User Mistakes Results Graph on the iPhone and iPad. . . . . . . . 62

32 Experienced User Mistakes Results Graph on the iPhone and iPad. . . . . 63

33 Elderly User Mistakes Results Graph on the iPhone and iPad. . . . . . . . 63

34 The Graph Presents the Median Comparison of the Degree of Agreement

Against Each Question by Three User Groups on the iPhone. . . . . . . . . 67

35 The Graph Presents the Median Comparison of the Degree of Agreement

Against Each Question by Three User Groups on the iPad. . . . . . . . . . 68

8

List of Tables

1 iPhone and iPad Speci�cation Comparison. The Bold Values Show the

Di�erences in Speci�cation. . . . . . . . . . . . . . . . . . . . . . . . . . . 18

2 Usability Issues on the iPhone and iPad. . . . . . . . . . . . . . . . . . . . 22

3 Advantages and Disadvantages of Qualitative Approach [52] . . . . . . . . . 23

4 Results of Surveyed Participants Comments Against the Di�erent Features

of the iPhone and iPad. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

5 Results from the ANOVA Tests for Estimating the Signi�cance Among the

Group Levels and Between the Device Levels Across Each Task. The p-

value Shown the Signi�cance Among the Mean Values of Each Participant

Group Against the Each Task. However, the Bold (p-values) Values Shown

the Signi�cant Di�erences Among the Groups and Between the Devices. . . 51

6 Participant Group Levels Comparison Results. The p-values Showed the

Di�erences Between the Participant Group Levels on the iPhone and iPad.

However, the Bold p-values Shown that there was a Signi�cant Di�erence

Between the Group Levels. . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

7 Device Levels Means Comparisons Results. The p-value Showed the Dif-

ferences Between the Means of the iPhone versus iPad Against each User

Group. However, the Bold p-values Shown the Signi�cant Di�erence. . . . 52

8 Means Results and Di�erences of Each Type of Mistake Made by Each

User Group on the iPhone and iPad. However, the Bold p-values Shown

the Signi�cant Di�erence. . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

9 Post Test Questionnaires Results in the Form of the Median, the Standard

Deviation and the Di�erences Between the Users Opinions Against Each

Question. However, the Bold p-values Shown the Signi�cant Di�erence. Q,

iP and id Represented the Question, iPhone and iPad Respectively. . . . . 64

10 Frequency and Percentage Results of the User Comments on the Di�erent

Aspects of Both the Devices. . . . . . . . . . . . . . . . . . . . . . . . . . . 68

9

Chapter 1

1 INTRODUCTION

Human Computer Interaction (HCI) is concerned with the joint performance of tasks by

the human and machines [1]. It has become valuable due to the fact that it involves the

design, implementation and evaluation of computer interactive systems that human beings

use in di�erent contexts to accomplish their tasks. The main goal of HCI is to produce

computer systems which are easy to use, well functional, e�cient, e�ective, and that are in

accordance with the requirements of the user [2]. The HCI goal would be ensured with the

usability of the product. According to the Webster dictionary, the usability originate from

the word �usable� which means �capable of being used � or �convenient and practicable for

use�.

We are living in a mobile society [3]. New information and communication technologies

(ICTs) have become a part of people's everyday lives. The mobile technologies are devel-

oping very quickly [5] and the trend is changing from conventional desktop computing to

the mobile computing. It facilitates the users to access and use their services anywhere

anytime [4] e.g. doing trading, shopping, banking and bill payment etc [6]. So in the

last few years, the developers redirect their design e�ort from a desktop perspective to a

small screen approach [7]. Under the constraint of small size, the usability of the mobiles

devices and their applications is a key factor for the success of mobile computing [8].

The mobile phone technologies have an impact on all age groups in our society. Studies

illustrate that young people are more active users of technology [9, 10]. They are the

�rst users who adopt the new technology and develop an actual mobile communication

culture [10]. On the other hand, the elderly people are also very keen to use the new

mobile technology but because of the small size, invisibility of text and complexity of the

functions, they avoid to use it [11]. The elderly population is growing rapidly in almost

all the economically developed countries [12]. It is noticed that elderly people take more

interest in games as compared to younger but because of the complexities and di�culties

of the user interfaces they cannot take part more e�ectively [13]. Earlier studies presented

that most of the services are designed by considering the young population and mistakenly

the elderly people are also considered as a part of the same age group [14].

In the mobile phone multi-touch technology, the iPhone, iPod Touch, and iPad are

sophisticated devices, which provide advanced and sensitive user interactions based on

gestures [15]. The iPhone and iPad support millions of the applications that designed and

developed by the Apple and the third party. It has been considered that the Evernote

is used by millions of the people in the world to remember their important notes and

10

synchronize them with their PCs [38]. Meanwhile, in the social network sites the Facebook

is one of the leading applications having more than 500 million active users in the world

[69]. Only in the Sweden more than 4 million people are using the facebook [71]. Similarly,

the Google web application (search engine) is also a famous site having the large number

of users [70]. The simplicity and consistency of the devices motivated the authors to

conduct the comparative usability research studies of the iPhone and iPad.

In order to �nd out whether the iPhone and iPad are performing according to user

expectations or not, the authors performed the survey and empirical usability evaluation

studies. The authors selected three frequently used applications (Facebook, Google Mobile

Application, Evernote) and measured the usability performances of three user groups

(novice user, experienced user, elderly user) as well as the iPhone and iPad. The age

range for the young people is 22-34 and for the elderly people is 60-75.

1.1 Aim and Objectives

The aim of this research work is to evaluate and compare usability performance of the

iPhone and iPad on three selected applications considering the elderly people as well as

the young people. To achieve this goal the following objectives are de�ned:

Identifying usability issues regarding the iPhone and iPad.

Investigating usability performance di�erences of the same applications on the iPhone

and iPad.

Investigating the performance di�erences between di�erent user groups using the

same applications and same devices.

1.2 Research Questions

1. What usability issues exist in the iPhone and iPad?

2. How does the performance di�er while using the same application on the iPhone

and iPad?

3. How does the performance di�er using the same application on the same device by

di�erent user groups?

11

1.3 Study Process

The study process presented a hierarchical �ow that was carried out during the the whole

research work.

Figure 1: The Overall Study Process.

1.4 Thesis Structure

Chapter 1 introduces the thesis and de�nes the research domain. In Section 1.1 aim and

objectives are given. In Section 1.2 research questions are given. In Section 1.3 the overall

research process is presented in the form of diagram .

Chapter 2 provides background knowledge about the usability. In Section 2.1 usability

and its di�erent attributes are discussed that de�ned by di�erent authors. Section 2.2

describes how to measure the usability. Section 2.3 describes the performance measuring.

Section 2.4 describes the usability evaluation. Section 2.5 describes the touch screen and

smart phones. Section 2.6 describes the iPhone and iPad. Section 2.7 is about the main

features of the iPhone and iPad. Section 2.8 presents the related work and Section 2.9

presents the usability issues concerning the iPhone and iPad.

Chapter 3 is about the research methodology adopted in the thesis. Section 3.1 presents

12

a literature review. Section 3.2 presents the survey design . Section 3.3 consists of the

experiment design . Section 3.4 presents the evaluation methods and Section 3.5 presents

the validity threats in research.

Chapter 4 presents the survey and experimental analysis results and discussion. Sec-

tion 4.1 presents the survey. Section 4.2 describes the demographics results of the survey.

Section 4.3 presents the usability and preference testing of the survey data. Section 4.4

describes the participants comments regarding the usability of the iPhone and iPad in the

survey questionnaire. Section 4.5 presents the measurement of usability attributes.

Chapter 5 contains conclusions and future work of the thesis. Section 5.1 presents the

conclusions of the research study and Section 5.2 presents the future work.

Chapter 6 contains the Appendices. In Section 6.1 Appendix A represents the survey

questionnaire. In the Section 6.2 Appendix B survey Questionnaire results are given. In

Section 6.3 Appendix C contains the p-values of the survey results. In the Section 6.4

Appendix D presents the post test questionnaire. Section 6.5 contains the Appendix E

that presents the experimental data i.e task time on both devices.

13

Chapter 2

2 BACKGROUND

2.1 What is Usability?

Usability in itself is not a new topic [17]. It is considered as a core term HCI [18] and has

become an important part in the design of products, software applications and devices.

However the user centered design and goal directed interaction design has increased us-

ability of products [17]. The role of usability is to make the system e�ective and easy to

use [19]. There are many de�nitions of the term usability. According to ISO 9241-11 the

term usability can be de�ned as �The extent to which a product can be used by speci�ed

users to achieve speci�ed goals with e�ectiveness, e�ciency and satisfaction in a speci�ed

context of use� [20].

Usability is the composition of the entities [21] e.g. e�ectiveness, e�ciency, learnability,

memorability etc. Some times these entities are de�ned as dimensions, components, scales

or factors of usability. All these forms have the same meaning, so the commonly used

term for them is usability attributes [21].

Shackel [19] is one of the famous authors who realized the importance of usability and

de�ned it as a relative property of a system. This property makes the relationship between

users and the systems; i.e. the subjective usability measure. The other side relates to

objective measures of interaction i.e. how easily and e�ciently users could achieve their

goals . Shackel de�ned the following attributes to measure the usability.

E�ectiveness: how e�ciently and with minimum errors the user completes his

goal.

Learnability: how much time the user required learning and how long this could

remember it.

Flexibility: adaptation to change in tasks.

Attitude: user satisfaction with the system.

Figure 2: Shackel's Usability De�nition [21].

14

Nielsen [22] is another usability expert, who de�ned usability by considering the following

attributes.

Learnability: the system should be simple and easy to learn.

E�ciency: the system should be capable to perform the users work within speci�ed

time.

Memorability: it should be easy to remember, so that the user can easily use it

after some period of not using the system, without having to learn all again.

Errors: the system should have low error rate and provide help to the user while

he/she make errors.

Satisfaction: the system should be attractive, so that the user feels comfort and

satisfaction while using it.

Figure 3: Nielsen's Usability De�nition [21].

Since from last 15 years, the International Standard Organization (ISO) has developed

di�erent standards for HCI and usability. The functionality of these standards is to

make the system simple and consistent. The ISO 9241-11 decompose usability into three

attributes [23] that are as discussed below.

E�ectiveness: the accuracy and time for speci�ed goal achievement by speci�ed

users.

E�ciency: utilization of resources for completion of speci�ed tasks.

Satisfaction: the user must be feeling comfort while using the system.

The software engineering community has also associated usability with interface design.

The ISO 9126-1 de�ned usability as �quality in use�. This de�nition has the same meaning

as the ISO 9241-11 de�ne usability but it is used in speci�c context. For the measurement

of usability, ISO 9126-1 speci�ed the following usability attributes [24].

Understandability: the software product must have the capability that the user

can easily understand and utilize it for the speci�ed task.

15

Figure 4: ISO 9241-11 Usability De�nition [21].

Learnability: the software system should have the capability that the user should

learn it easily.

Operability: the software system must have the capability that user could easily

operate and control it.

Attractiveness: the software system must have capability to attract the user.

The existing usability studies on mobile applications de�ne the nine usability attributes

such as: learnability, e�ciency, memorability, errors, user satisfaction, e�ectiveness, sim-

plicity, comprehensibility and learning performance. Selection of suitable attributes for

evaluation of mobile applications depends on the nature of the applications and the ob-

jectives of the usability study [25].

2.2 Usability Measurement

Many past studies presented that the usability is dependent on the context of use [18].

This context of use is formed by the user, task, equipment and environment [26]. Most

of the studies are conducted in controlled environment, where the user performed some

prede�ned tasks using the speci�ed system. The usability can be assessed by consider-

ing several measurable parameters. These parameters can be divided into two groups:

subjective preference measures and objective performance measures.

Subjective preference measures are being assessed by considering what the user likes

and dislikes about the system. It can be measured by using for example a questionnaires

[27]. An advantage of a questionnaires is that they can provide the evaluator with feedback

from the user's point of view. Another bene�t of the questionnaires is to provide the

comparable measures across the systems that being evaluated [28].

The objective performance measures relate to the e�ectiveness of system (e.g. time,

errors, and number of activities). Objective performance measures are being assessed by

giving the speci�ed tasks under a speci�ed time limit. Sometimes it can be measured by

counting the number of tasks per unit time [27]. The e�ciency measurement of the system

is also related to the total time on task, usages of the resources and mental resources used

to manipulate the system interfaces.

16

2.3 Performance Measuring

Performance is the degree to which the system accomplishes its designated function under

certain constraints [29]. In this study, the authors followed the usability attributes that

de�ned by ISO 9241-11 i.e. e�ectiveness, e�ciency and satisfaction. To test the usability

attributes following metrics were measured.

Task Time: How long it took a participant to complete a task in seconds [30].

Errors: Errors are an unintentional action, mistakes, slips and omissions, a user makes

while performing a task [30].

Satisfaction: It is the reaction of the participants about the overall performance of the

system.

2.4 Usability Evaluation

Usability evaluation is �the systematic process of collecting data, in ordered to have a

better understanding of the users and how the user groups use the product to perform a

speci�c task under speci�ed conditions� [31]. It has been required in di�erent stages of

the system development process. Its goal is to provide feedback that helps to improve

the quality and functionality of the system [32]. Zhang [33] de�ned three approaches

of usability evaluation methods such as: testing (e.g. Coaching method, performance

measurement, thinking aloud protocol), inspection (e.g. heuristic evaluation) and inquiry

(e.g. interviews, �eld observation, questionnaires). In the usability testing approach,

speci�ed users perform speci�c tasks using the system or prototype [21]. The usability

evaluator examines, how easily and e�ciently the users perform their tasks. The usability

inspection approach involves the usability experts or professional to examine whether the

system or each part of the system follows usability principles. In the usability inquiry

approach the evaluator collect information about user perceptions and understanding of

the system through interviews, surveys and verbal discussions [21].

2.5 Touch Screen and Smart Phones

Touch screen user interfaces have become more attractive in electronic devices [34]. There

are two types of touch screens: capacitive and resistive. Resistive touch screens are

made of a number of metallic and electronic conductive layers, separated with a small

gap. When a user tap on the touching surface, both layers make a connection and cause

the electric current that activate the touching event. Capacitive touch screens work by

sensing conductive object e.g. �nger. They allow the multi-touch functionality that can be

performed by using multiple bare �ngers [35]. The touch screen input method is becoming

17

popular in the smart-phones and other mobiles devices such as PDAs and tablet laptops

[36]. In 1993, IBM presented the �rst smart-phone and its name was Simon. The main

features of Simon were a calendar, a calculator, a world clock, and an email client. It had

a touch screen interface that let the users write text by using an on-screen keyboard or

a stylus (i.e. stick like a pen). There is no proper de�nition of the term smart phone.

However, it is considered as a mobile phone that has a greater computing power than the

normal cell phones and having the features like the PCs. After launching the iPhone, the

user interface design had great impact on the smart-phones with multi-touch technology

[37].

2.6 The iPhone and the iPad

Apple designs, manufactures and markets a range of digital products. These products

contain personal computers, mobiles devices, music players, related software and third-

party applications. In the past few years, the company has launched products such as the

iPhone, iPod, and iPad [38]. The iPhone and iPad are considered as revolutionary Apple

products [39] and both have some similar features and look like the smaller and larger

version of each other. However in some perspectives they have some di�erences such as

RAM, size, resolution and weight [40]. These similarities and di�erences are presented in

the Table 1.

Feature iPhone 4 iPad

Chip A4 A4

OS iOS 4 iOS 4

Processor 1 GHz ARM Cortex-A8 1 GHz ARM Cortex-A8

RAM 512 MB 256 MB

Connection Wi-Fi Wi-Fi

Size 4.5 x 2.31 x0.37 inch 9.56 X 7.47 X 0.52 inch

Resolution 960 X 640 pixel 1024 X 768 Pixel

Weight 137g 730g

Table 1: iPhone and iPad Speci�cation Comparison. The Bold Values Show the Di�er-ences in Speci�cation.

2.7 Features of the iPhone and iPad

The main features of the iPhone and iPad are described below:

2.7.1 Multi-touching

The multi-touch technology is used as an input method for the iPhone and iPad. It is

de�ned as the technology where the user can interact with system with multiple inputs

at a time e.g. two or more �ngers for a single task. The users can use their �ngers for

18

typing email, enlarging text, swapping through photos and for using zooming functions.

All input functions are performed by using the �ngers at touch screen interfaces.

2.7.2 Operating System (iOS 4)

The iOS 4 is an operating system that is being used by Apple for the iPhone 4 and iPad.

It is considered as a simple operating system. The user can perform various tasks (e.g.

writing email and zooming text) simply using the �ngers on the touch screen interface.

The user can use millions of di�erent built-in applications with a few �nger gestures such

as tap, drag, swipe and pinch [38].

2.7.3 Applications (Apps) Store

Apps Store is a database that is especially designed for Apple's products. It contains

business applications, education applications, entertainment applications, social network-

ing applications, news applications and so on. The applications are designed by Apple's

designers and developers as well as by a third party. Some applications are free but for

some applications you will have to pay [38].

2.8 Related Work

Tsung [41] performed an age based usability study on the iPhone. Three age groups

i.e. college-level youth, middle-aged professionals and senior citizens were taken. The

conclusion of this study was that the system should not only consider the �glamorous

animation and fancy utilities� but all human physical conditions e.g. hearing and eye

sight that will make the system more senior friendly.

Chaparro [42] evaluated the keyboard performance of the Netbook and iPad. They

measured the performance on three di�erent style keyboards. The iPad soft touch key-

board (portrait as well as in landscape) and the Acer Netbook physical keyboard. The

attributes and their metrics are given below.

Performance (task time and errors)

Perceived Mental Workload

Satisfaction

The study showed that there was a slight di�erence in the performance of the Netbook

physical keyboard and the iPad landscape and no di�erence in the performance of the

iPad portrait and landscape keyboards. The satisfaction level of the iPad landscape was

slightly higher than the physical keyboard and portrait was slightly lower than the physical

keyboard. The preference level of the landscape iPad keyboard and physical keyboard was

same the while portrait keyboard was less preferred by the participants .

19

In another study, six sta� members at Curtin University were supplied with the iPad

to test usability for 100 days. The iPad was handed over to each sta� member and at the

end of the third month these participants were interviewed. The research showed that

there was no problem for inexperienced users using the iPad. The consistency issues were

noted like the other studies and also some common functionalities were identi�ed [43].

The Nielsen and Norm group (NNGroup) [44] conducted a usability study of the iPad

applications and contents, a few weeks after Apple launched the device. They tested seven

users; all had at least three months iPhone experience and only one of them was an iPad

user. In this study they assessed the interface of applications, websites and speci�c iPad

applications, touch screen, changing orientation, and di�erent gestures. They criticized

and pointed out that the applications are designed without considering the importance of

usability e.g. read tab asymmetry, zoom the map or zoom the page, use of proper mental

models, small target area, accidental tapping and the back button etc. They highlighted

many usability issues e.g. inconsistencies and accidental actions etc.

2.9 Issues in the iPhone and iPad

From the literature [44], the authors found some usability issues that are discussed below:

2.9.1 Consistency

The consistency issue remain almost all the time with the iPhone and iPad applications.

Figure 5 shows the inconsistency in the button, keyboard appearance and writing envi-

ronment of the messages of SMS and MobileVoip application of the iPhone.

Figure 5: Di�erent Consistency Issues in SMS and Mobile Voip Applications.

20

2.9.2 A�ordance

The other issue in the iPhone as well as the iPad application was a�ordance. �Users don't

know that something is touchable unless it looks so� [44]. In the weather application of

the iPhone it is very di�cult to add new a location as the option lacks the a�ordance.

Figure 6 shows this a�ordance problem in the weather application of the iPhone. The �i�

at the right bottom side of the application is used for adding the new location but the

option is not very clear as well as not seems to be tappable.

Figure 6: A�ordance Issues in the Weather Applications of the iPhone and iPad.

2.9.3 Small Search Box and Missing Information

Figure 7 shows the small search box and missing search button at the bottom of the Apps

Store application of iPad. As there is a big screen on the iPad the search box needs to be

more clear and also users expect the search button at the bottom of the application.

Figure 7: Small Search Box Issues in the Applications of the iPhone and iPad.

21

2.9.4 Invisible Control and Missing Instructions

Figure 8 shows the invisible control and missing instructions in the iPhone and iPad

applications. In Figure 8 at the right side when the message is too long it create a scroll

bar but it remain invisible until a someone tap on it. Similarly in twitter there are four

pages that can swipe forward and backward but there are no instructions to move them.

Another miss leading button at the top of the twitter application that seems like a setting

button but in fact it is used for reply option.

Figure 8: Invisible Control and Missing Instructions Issues in the Twitter and FacebookApplication on the iPhone and iPad.

Table given bellows describe the list of other usability issues in the iPhone and iPad.

Usability Issues Detail

Inconsistency in interaction Nothing happens

Enlarging the picture

Hyper-linking to a more detail page about that item

Replace the picture with new picture

Popping up a set of navigation choices

Accidental touch Touch something that one did not mean to

False back button Button that look like back button, but when pressed

it lead to new page

Lack of a�ordance Tappable elements look like it not supposed to touch

and untappable look like tappable

Hyperlinks Missing of important hyperlinks like wired magazine

using back button instead of hyperlinks

Changing orientation Application looks di�erent in portrait and landscape

Gestures Di�cult to remember gestures

Multiple panels Small font size and crowded contents

Auto-correct Should turn into autocomplete

Table 2: Usability Issues on the iPhone and iPad.

22

Chapter 3

3 RESEARCH METHODOLOGY

The research methodology is the systematic way to solve the research problem. Commonly

two main approaches are used for the research studies such as quantitative and qualitative

[45]. However, Creswell [46] has also introduced a third type of the research approach

called a mixed method approach i.e. the combination of quantitative and qualitative

approaches.

The quantitative research approach is based on the measurement of quantity or amount.

It is used when one begins with a hypothesis and tests for con�rmation and dis-con�rmation

of that hypothesis [47]. It is applicable to phenomena that can be expressed in terms of

quantity [48]. Generally, the survey and experiment are used in the quantitative approach

to inquiry the hypothetical issues and participants of the studies [46]. Some advantages

of the quantitative approach [49] are listed below:

1. Data can be gathered from large user groups in a short period of time

2. Large sample data can be generalized to the entire population

3. Quantitative research is repeatable

The qualitative research approach is concerned with subjective assessment of attitudes,

opinions and behaviors [48]. It is an interpretative approach which uses the meaning of

phenomena as understood by the participants of the studies [50]. Generally, open ended

questions are used in qualitative research [51]. Some of the advantages and disadvantages

are listed in Table 3.

Advantages of Qualitative Approach Disadvantages of Qualitative Approach

It goes deep and takes into account all details in Finding cannot be generalized because they

the environment are particular environment dependent

No need to recruit a large number of participants The �nding can be biased according to the

Flexibility in time and place understanding of the researcher

No need to conduct interviews of all participants

at same time

Table 3: Advantages and Disadvantages of Qualitative Approach [52] .

In this research, the authors adopted the mixed research approach. The quantitative

research approach was used to perform the experiment and the qualitative research ap-

proach was used to conduct the literature review and analyze the open ended questions

and participants comments. These research studies were carried out in multiple steps such

as a literature review, survey design, experiment design, pilot tests studies, experiment

23

execution, selection of suitable evaluation methods, analysis of gathered data, results,

conclusions and future work.

3.1 Literature Review

In the �rst step a literature review was done. The objectives of a literature review was

to understand the usability, usability attributes, usability issues in smart phone, and the

usability issues in the iPhone and iPad. Another objective of a literature review was

acquiring the understandability of the di�erent statistical usability evaluation methods.

The authors utilized the all available information resources to carried out this re-

search work. The mentioned keywords are used to �nd out the related data from all

available databases. However some of the authentic knowledge resources were listed as

BTH Library's books, IEEE, ACM digital library, Engineering Village, Google Scholar,

Springer-link, Scopus, and Ebrary.

3.2 Survey Design

To �nd usability issues the authors used the survey technique. The objectives of the

surveys were:

1. To validate the issues regarding the usability of the iPhone and iPad that authors

�nd out during the literature review.

2. To get the input for the experimental design.

3. To �nd out the users point of views about the usability of the iPhone and iPad.

A survey is an e�ective way to gather information from a large number of the population

in a short period of time at relatively low cost. It is considered that the respondents are

less likely try to please the researcher and provide the social acceptable responses. The

survey technique gives time to the respondents to think about the question or statement

before answering and do not bind them to complete it in one setting [53].

The seven point Likert scale was used in a survey questionnaire that helps the users

to express their degree of agreement with a statement. The numbers (1-7) were used for

strongly disagree, disagree, near to disagree, neutral, near to agree, agree and strongly

agree respectively for each statement.

In order to collect feedback from the users about the iPhone and the iPad, the authors

divided the survey into demographics, systems (iPhone, iPad) information, applications

information, touch screen features and keypad functionality. Each part contained several

related statements, as well as a space for own comments. The detailed survey questionnaire

is given in Appendix A. However, here the small piece of each section of questionnaire

was presented and structured as follows.

24

1. Presentation This section presented the users demographics (age, sex, quali�cation,

using experience), purpose of the survey study, explain the questionnaire and provide the

general information. Following is the presentation section of the survey questionnaire.

Survey Questionnaire for Evaluating the Usability of the systems(iPhone/iPad)

Participant Number: ______________ Sex: ___________________Home Country: _____________________ Age: ________________Education level: 2 BSc 2 MSc 2 PhD 2 Other __________________This questionnaire gives you an opportunity to describe your experiences of the system.Your responses will help us to evaluate the usability of the system.To as great degree as possible, consider all the tasks that you have been performing withthe system while you answer these questions. Please read each statement and indicatehow strongly you disagree or agree with the statement by marking (X) in the circle. If astatement does not apply to you, leave it empty or use the word (N/A) at the commentsline.Please write comments to elaborate on your answers (if you have any).As you complete the questionnaire, please do not hesitate to ask any questions.

Thank you.

Figure 9: Presentation.

2. System (iPhone/ iPad) This section presented the overall impression of the de-

vices. The objective of this section was to obtain the feedback about the system design,

graphics, capabilities and somehow about the ease of use. A piece of the system section

of a questionnaire is presented in Figure 9 below.

Figure 10: System.

3. Touch Screen This section presented the usability of the touch screen gestures.

Figure 10 shows a few statements of such section.

25

Figure 11: Touch Screen.

4. Keypad This section presented to gain the user opinions about the performance

of the keypad. The statements contained the di�erent keypad aspects such small size of

keys, landscape style and single character keys. The Figure 10 shows a few statements

regarding the keypad section.

Figure 12: Keypad.

5. Applications This section presented the di�erent usability features related to the

applications simplicity, ease to learn, consistency, visibility of the text and recoverability

of the mistakes. A small piece of this section presented in the Figure 12.

Figure 13: Applications.

26

3.3 Experimental Design

Experimental design is the process of planning a study to meet speci�ed objectives. Proper

planning of an experiment is important in order to ensure that the right type of data,

a su�cient sample size and resources are available to answer the research questions of

interest as clearly and e�ciently as possible [54, 55]. The following aspects provide further

insight into the experimental design.

3.3.1 Participant Selection

The authors selected the three user groups such as young novice users (20), young expe-

rience users (20) and elderly users (20). All the young users were bth students and well

known with the English language. In the elderly users 12 were bth employees; �ve were

the Karlskrona public library employees and three were Ronneby public library employees.

In the beginning, the authors were interested to divide the elderly people into two groups

like the young people (novice, experienced) and also visited the old people apartments in

Karlskrona and Ronneby. However, after meeting with the management of the old people

apartments the authors realized that it is di�cult to collect reliable data from the elderly

people because of language communication problems. So the authors decided to make a

single group of the elderly people and involved all those people who would speak, write,

read and understand the English language. Figure 13 shows the participants selection for

the experiment.

Figure 14: Participants Selection for the Experiment.

Novice Participant The person who was never used the smart phone, touch screen

systems or the internet on small devices.

27

Experienced Participant The authors considered the person as an experienced par-

ticipant who was using the smart phone or the iPhone more than three months.

Elderly Participant The person who was at least using a computer system or smart

phone or having the experience about the touch screen system.

3.3.2 Variables Selection

A variable is any character or attribute that can vary across people or situation or thing

[56]. There are two basic types of variables; independent variable and dependent variable.

The independent variable is one which the experimenter controls and manipulates to see

the e�ect on the treatments (dependent variable). On the other hand, the dependent

variable is the responses of the independent variable [56].

In the experiment, the authors used two types of the variables, independent variable

(factor) i.e. �Participant group� and dependent variable i.e. �Device�. The �Participant

group� variable has three levels such as novice user, experienced user and elderly user.

Similarly, the �Device� variable also has two levels such as the iPhone and iPad.

3.3.3 Hypotheses

The �rst step in the research process is the formulation of hypothesis about a speci�c

issue [57]. It is a precise problem statement that can be directly tested through empirical

investigation [58].

In comparative usability study of the iPhone and iPad across the three groups of par-

ticipants, the authors were interested to measure the performance di�erences between the

device levels as well as the di�erences among the participant group levels. The hypotheses

statements were de�ned as as follows:

H01: There is no di�erence in group levels over the device variable.

Ha1: There is a di�erence in group levels over the device variable.

H02: There is no di�erence in device levels across the group variable

Ha2: There is a di�erence in device levels across the group variable.

Here H0 and Ha represent the null hypotheses and alternative hypotheses respectively.

3.3.4 Task De�nition

For the execution of usability tests, six tasks were de�ned. Each task and its description

were designed on a separate paper. The presentation of the tasks was as follows:

28

Facebook

Task 1: Login to Facebook.

Email address: usbtest2011@gmail.com

Password: usbtest2k11

Description:

Tap on Facebook

Tap on the First Text box �Email� and enter the email address

Tap on the �password� text box and enter the password

Tap on �Go� at the keypad

To use capital text tap on

To use numbers tap on

29

Task 2: Compose a new message on Facebook and send it.

To: azambth72@yahoo.com

TEXT: This is a usability test message for the iPhone

Send

Description:

Tap on the Facebook Home button

Tap on messages

Tap on create message at the top right corner of the Facebook application

Enter email address to whom you want to send a text , i.e. Given above

Tap just below the line and Write a short message

Tap on Send Button at the top of application

Google Map

Task 3: using the Google mobile application �nd the location of Blekinge

Tekniska Högskola and tap on the sign to con�rm the location.

Description:

Tap on Google mobile application

Tap on Apps at middle of bottom

Tap on Maps

Write Blekinge Tekniska Högskola

Tap on search button

Tap on the sign to con�rm the location

30

Task 4: Make a close view of Blekinge Tekniska Högskola building with zoom

in function.

Description:

Tap on Layers at the bottom left corner

Select the satellite from list

Tap on search box

Write Blekinge Tekniska Högskola

Tap on sign for zoom in

Evernote

Task 5: Create a new note in the Evernote application

User name: azamjarral

Password: aS5AK9

Title: First Note

Text: My �rst note in the Evernote application

Save

Description:

Open Evernote application

Tap on Sign in button

Tap on text box username and enter username

Tap on the text box password and enter password

Tap on Sign in

Tap on New Note at the bottom left corner. (For iPhone at the middle of Facebook

application)

Tap on the Save button at the right top corner of the application

31

Task 6: Logout from Evernote

Description:

Open Evernote application

Tap on the setting icon

Find the Logout option

Tap on Logout

3.3.5 Procedure

The authors booked a silent room in BTH that was used as a usability laboratory. To

provide the same environment to all the participants, the authors �xed the devices and

chairs. Before starting the experiment, all the tasks were arranged at a separate table. it

was conducted with a single user at a time. The experimental setup is shown in Figure

14.

Figure 15: Laboratory Setup for User Trials.

3.3.6 Experimental Material

The experimental material contained the iPhone, iPad, documents for the tasks descrip-

tion, demonstration and post test questionnaire.

32

3.3.7 Post Test Questionnaire

To know the participants usability experience abut the systems, a post-test questionnaire

was designed. It contained close ended as well as open ended questions. The close ended

questions were designed by using seven points Likert Scale and open ended questions were

also part of the same questionnaire but they were in the form of participants comments.

The close ended questions limit the respondent to the set of alternatives being o�ered,

however the respondent could easily respond the maximum questions within the limited

time. On the other hand the open ended questions help to explore the respondent and

thus avoiding the bias which may occur in the case of close ended questions [59]. The

detailed questionnaire is given in Appendix D.

3.3.8 Pilot Studies

Three pilot studies were conducted i.e. an experiment, a survey and a post test question-

naire. Before starting the experiment pilot test were conducted to assure that the tasks

were properly de�ned and the participants would have no di�culty in understanding every

step in each task. Three tests were conducted, one for each group and improvements were

made on the basis of their feedback. At the start the authors decided to just mention the

task e.g. login to Facebook, however after the pilot test, the authors realized that it was

not easy for the novice users as well as for elderly user to successfully complete the given

tasks. Then the authors decided to write down the step by step description of each task

with the symbolic representation wherever required. In this way the ambiguities were

removed in the experimental tasks.

Similarly the pilot studies were also conducted for the survey questionnaire and post

test questionnaire. After those studies, the changes were made in both questionnaires

according to the requirement of the users.

3.4 Evaluation Methods

In order to evaluate the survey and experimental data, the authors use multiple statistical

test methods. The detail of each method is given below.

3.4.1 Mann Whitney Test

The Mann Whitney test is a nonparametric test that can be used for comparing two

unequal sample size and unpaired data [60, 61]. The key result is the �p� value that shows

the signi�cant di�erence between both samples [62]. In this research, the survey data was

unpaired, unequal samples size and consist on rating scale that is why the Mann Whitney

test used to measure the signi�cance between the iPhone and iPad.

33

3.4.2 Normality Test

After obtaining the data from the experiment, it was important to test the normality

of the data. The authors read three test techniques: Shapiro Wilk Test, Kolmogorov-

Smirnov (KS) Test, and Histogram. The KS test is used for more than 50 samples, so

the authors did not use this method. On the experimental data, the Shapiro Wilk Test

and Histogram techniques could be applied to check the normality. however the authors

used �Shapiro Wilk test� online software that directly show whether the data is normally

distributed or not [63].

3.4.3 Analysis of Variances (ANOVA)

ANOVA is the most powerful and robust parametric method used to measure the signif-

icance. It has some assumptions such as the homogeneity of variance or normally distri-

bution of the data. The ANOVA is used to measure the signi�cant di�erence between

two or more than two groups [64]. In the experiment, there were three user groups and

two devices. The purpose experiment was to measure the di�erences among the groups

and between the devices. So to analyze the data, the authors used two way ANOVA

with replication. Further to see where the signi�cant di�erences between the devices and

among the groups were, the unpaired t-test and paired t-test were used respectively.

3.4.4 Unpaired T-test

The unpaired t-test is used to see where the signi�cant di�erences among groups across

the each device were. The comparison of the groups was made as follows.

Figure 16: Participant Group Levels Comparison Across the Device.

3.4.5 Paired T-test

The paired t-test used to see where the signi�cant di�erences between the devices across

the each group were. The comparison between devices across groups was as follows.

34

Figure 17: Device Levels Comparison Across the Participant.

3.4.6 Wilcoxon Matched Paired Signed Ranks Test

The Wilcoxon Matched Paired Signed Ranks test is a nonparametric test that is an

alternative to the paired t-test. It should be used if the distribution of di�erences between

pairs may be non-normally distributed [65]. The authors applied this method to �nd out

the signi�cance between the post-test questionnaires of the iPhone and iPad.

3.4.7 Error Bars

The error bars are used to estimate the variation or distribution of a statistical data in

the graphs. They can represent the standard deviation, the standard error, the con�dence

interval, or some other measure of error or uncertainty. Here the authors only discussed the

standard deviation and standard error. The standard deviation is a measure of variation

in data about their mean and standard error is a standard deviation of the estimated

statistical data such as mean, and regression coe�cient etc. The error bars do not let you

decide whether the di�erence between the mean is statistically signi�cant or not when

the two samples error bars do overlap or not [66]. It indicates how closely the population

mean is likely to be estimated by the sample mean.

3.5 Validity Threats

Validity refers to the conceptual and scienti�c soundness of a research study and investi-

gation [67]. There are many potential issues that may a�ect the reliability of the research.

Four types of validity threats discussed in the literature [46] are as follows:

Internal validity threats

External validity threats

Constructive validity threats

Statistical conclusion validity threats

35

3.5.1 Internal Validity Threats

Internal validity threats refer to the research design that causes the interferences of the

researcher's ability to draw the correct inference from the gathered data [46]. They might

be the inadequate procedures, wrong participant's selection, and technical skill of the

researchers, ambiguities in the experiment, post-test questionnaire and survey design and

improper instrumentation of the experiment [67, 68]. To overcome such types of issues

the authors studied the core concepts of survey design, experiment design and ongoing

research studies about usability of the iPhone and iPad. In the survey and post test

questionnaire, the authors obtained the demographics that help to minimize the wrong

selection of the participants. Pilot tests were conducted for the survey questionnaire,

experiments and post-test questionnaire which helped to �nd out the ambiguities in all

of them. Furthermore the authors also used the counterbalancing technique to overcome

the bias.

3.5.2 External Validity Threats

External validity threats relate to the generalizability of the results of a research study [67].

They might be the sample characteristics, stimulus characteristics and settings, reactivity

of experimental arrangements, multiple treatments interference, novelty e�ect, reactivity

of assessment and timing of measurement [67]. To minimize the external validity threats,

the authors requested all of the participants to report their own experiences because the

purpose of the study is to evaluate the performances of devices not to the participants.

However there might be external validity risks, as people really do not report their personal

reality because they want to see themselves in a good light [68].

3.5.3 Constructive Validity Threats

Constructive validity threats relate to the selection of de�nitions and the measures of the

variables in a study [46]. Authentic data sources such as IEEE, ACM Digital Library,

ScienceDirect provide the correct de�nitions and measures of the variables. The authors

try to use the authentic data sources to minimize such types of validity threats.

3.5.4 Statistical Conclusion Validity Threats

Statistical conclusion validity threats relate to the statistical conclusion of a study. These

threats might be the cause of the inadequate selection of a statistical power such as

sampling and suitable methods [67, 68]. To reduce such type of validity threats, the

authors studies di�erent statistical methods in detail.

36

Chapter 4

4 STATISTICAL ANALYSIS, RESULTS AND DISCUSSIONS

This chapter presents the analysis, results and discussions of the results about the collected

data of the studies. There are two main sections of this chapter; the �rst one presents the

results and discussions on the results of the survey data and the second one presents the

results and discussions on the results of the experimental data.

4.1 Survey

In order to evaluate the survey data, the authors clustered the 42 survey statements into

subsections such as system results, touch screen results, keypad results and applications

results. The results of the survey responses are given in the Appendix B and the p values

are given in the Appendix C.

4.2 Demographics Results

This section of the survey was presented the demographics of the respondents. The

authors collected feedback from 14 (8M, 6F, age = 21-35 years) iPhone and 10 (7M, 3F,

age = 23-47 years) iPad users through the survey. 69% of the respondents had been using

the iPhone for more than six months. 8% of the respondents had been using the iPhone

for four to six months and the remaining 23% participants had been using it for the last

one to three months. On the other hand, the iPad responses showed that 20% of the

respondents were using the iPad for more than six months. 30% of the respondents had

been using the iPad for four to six months. 30% of the respondents had been using it for

the last one to three months and 20% of the respondents had been using the iPad for less

than one month.

4.3 Usability and Preference Testing

4.3.1 System Results

The �rst section of the usability testing questionnaire named �system� contained 15 state-

ments. Each statement was related to the overall satisfaction about the iPhone and iPad.

The results and signi�cances between the users opinions against each statement regarding

the iPhone and iPad are:

The system is easy to use for opening applications.7% (1) of the iPhone participants

disagreed with a statement and they considered it di�cult for opening applications. 21%

(3) were near to agree, 14% (2) were agree and 57% (8) were strongly agree that the

37

iPhone was easy to use in opening application. On the other hand, the iPad participant's

responses results presented that 10% (1) was near to agree, 70% (7) were agree and 20% (2)

were strongly agree with the above statement. However the comparative study regarding

this statement presented that there was no signi�cant di�erence (p = 0.183 > 0.05)

between the user views for the iPhone and iPad.

The system is easy to use in closing applications. The iPhone results regarding this

statement showed that 8% (1) of the respondent was disagree and he thought it is not easy

to use in closing applications; 25% (3) participants were neutral; 8% (1) participant was

near to agree, 25% (3) were agree and 33% (4) were strongly agree. The iPad participants

responses results presented as: 11% (1) was disagree, 11% (1) was near to disagree; 11%

(1) was neutral; 44% (4) were agree and 22% (2) were strongly agree. The comparative

study of the user's opinions concerning the iPhone and iPad presented that there was no

signi�cance (p = 0.404 > 0.05).

The system is easy to use in �nding help. The iPhone participant's responses results

concerning this statement were presented as: 14% (2) were disagree, 7% (1) was near to

disagree; 43% (6) were neutral; 14% (2) were near to agree, 7% (1) was agree and 14%

(2) were strongly agree. Similarly, the iPad participants responses results presented as:

10% (1) was disagree, 20% (2) were near to disagree; 40% (4) were neutral; 20% (2) were

near to agree and 10% (1) was agree. The comparative study of the user's views showed

that there was no signi�cant di�erence (p = 0.083 > 0.05) between the iPhone and iPad.

I can easily use the system in the sun light. The iPhone participants responses results

regarding this statement were presented as: 7% (1) was near to disagree; 21% (3) were

neutral; 36 % (5) were near to agree, 21% (3) were agree and 14% (2) were strongly

agree. The iPad responses results presented as: 20% (2) were near to disagree; 40% (4)

were neutral; 30% (3) were near to agree and 10% (1) was agree. The comparative study

concerning this statement showed that there is a signi�cant di�erence (p = 0.011 < 0.05)

in user's views regarding the iPhone and iPad. The statistical result showed that the users

preferred the use of iPhone in sun light than the iPad.

It is easy to �nd a new application in the Apple store. The iPhone responses results

showed that 7% (1) was near to disagree; 14% (2) were neutral; 14% (2) were near to

agree, 7% (1) was agree and 57% (8) were strong agree. Similarly the iPad responses

results regarding this statement showed that 10% (1) was disagree, 10% (1) was near to

disagree; 30% (3) were neutral; 30% (3) were near to agree, 10% (1) was agree and 10%

(1) was strongly agree. Comparative statistical result showed that there was a signi�cant

di�erence (p = 0.018 < 0.05). The users responses result showed that they would easily

use the iPad for �nding the new applications on the Apple store comparing the iPhone.

The layout of the applications on the system screen is clear. The results of the iPhone

respondents showed that 7% (1) was disagree; 21% (3) were neutral; 14% (2) were near to

agree, 21% (3) were agree and 36% (5) were strongly agree. Similarly, the iPad participants

38

responses results were presented as: 10% (1) was neutral; 60% (6) were agree and 30%

(3) were strongly agree across above statement. The comparison result showed that there

was no signi�cance (p = 0.322 > 0.05) in users views for the iPhone and iPad.

The graphics of the system are appealing for all age groups. Responses results regarding

the iPhone presented that 7% (1) was disagree; 7% (1) was neutral; 21% (3) were near

to agree, 29% (4) were agree and 36% (5) were strongly agree. Similarly the iPad results

presented that 11% (1) was near to agree, 56% (5) agree and 33% (3) were strongly agree.

The comparison result showed that there was no signi�cant di�erence (p = 0.341 > 0.05)

between the participants opinions about the iPhone and iPad.

I can easily change the background colour of the interface. The iPhone participant's

responses results presented as: 14% (2) were disagree, 14% (2) were near to disagree;

21% (3) were neutral; 14% (2) were near to agree and 36% (5) were strongly agree. On

the other side, the iPad participant's responses results showed that 10% (1) was near

to disagree; 30% (3) were neutral; 20% (2) were near to agree, 30% (3) were agree and

10% (1) was strongly agree. However the comparative study showed that there was no

signi�cance (p = 0.489 > 0.05) between the participants opinions regarding the iPhone

and iPad.

I think the system is designed for all age groups. The iPhone results showed that 7%

(1) was strongly disagree, 7% (1) was disagree; 21% (3) were neutral; 14% (2) were near

to agree, 21% (3) were agree and 29% (4) were strongly agree. Alike, the iPad results

presented that 20% (2) were near to disagree; 10% (1) was neutral; 10% (1) was near

to agree, 30% (3) were agree and 30% (3) were strongly agree. There was no signi�cant

di�erence(p = 0.467 > 0.05) in the user's opinions for the iPhone and iPad.

I like the �nishing of the interface. The iPhone responses results presented that 7%

(1) was disagree; 7 % (1) was neutral; 21% (3) were near to agree, 21% (3) were agree

and 43% (6) were strongly agree. Similarly the iPad results presented that 20% (2) were

neutral; 20% (2) were near to agree, 20% (2) were agree and 30% (3) were strongly agree.

The comparative study showed that there was no signi�cant di�erence (p = 0.382 > 0.05)

in the participants' views for the iPhone and iPad.

The interface of the system is pleasant. The results of the iPhone responses: 7% (1)

was disagree; 14% (2) were neutral; 21% (3) were near to agree, 29% (4) were agree and

29% (4) were strongly agree. Similarly, the results of the iPad responses: 10% (1) was

neutral; 40% (4) were near to agree, 20% (2) were agree and 30% (3) were strongly agree.

The comparative study showed that there was no signi�cant di�erence(p = 0.467 > 0.05)

between the users outlook for the iPhone and iPad.

It is easy to install the required application. The iPhone responses results showed

that 7% (1) was disagree; 7% (1) was neutral; 14% (2) were near to agree, 29% (4) were

agree and 43% (6) were strongly agree. Similarly, the iPad responses results showed that

11% (1) was near to disagree; 11% (1) was neutral; 33% (3) were near to agree, 22% (2)

39

were agree and 22% (2) were strongly agree. The comparison result showed there was no

signi�cant di�erence (p = 0.174 > 0.05).

It is simple to uninstall the application. The iPhone responses results showed that 7%

(1) was disagree, 7% (1) was near to disagree; 7% (1) was neutral; 7% (1) was near to

agree, 36% (5) were agree and 36% (5) were strongly agree. On the other side, the iPad

responses results showed that 11% (1) was near to disagree; 11% (1) was neutral; 11% (1)

was near to agree, 44% (4) were agree and 22% (2) were strongly agree. The comparative

study showed that there was no signi�cance(p = 0.279 > 0.05) in the iPhone and iPad

users views.

The system has all the functions and capabilities I expect it to have. The iPhone

responses results showed that 29% (4) were disagree, 7% (1) was near to disagree; 7% (1)

was neutral; 29% (4) were near to agree, 21% (3) were agree and 7% (1) was strongly

agree. Similarly, the iPad responses results showed that 30% (3) were disagree, 10% (1)

was near to disagree; 20% (2) were neutral; 20% (2) were near to agree and 20% (2) were

agree. The comparison of the users point of views about the iPhone and iPad was showed

no signi�cance(p = 0.265 > 0.05).

I like using the interface of the system. For such statement, the iPhone responses

results showed that 7% (1) was strongly disagree, 7% (1) was near to disagree; 14% (2)

were neutral; 7% (1) was near to agree, 43% (6) were agree and 21% (3) were strongly

agree. The iPad responses results showed that 13% (1) was near to disagree; 13% (1)

was neutral; 50% (4) were near to agree and 25% (2) was strongly agree. Similarly the

comparative study about the user´s point of views regarding the iPhone and iPad showed

no signi�cance (p = 0.233 > 0.05). Figures 17 and 18 showed the results graphs of the

users opinions across the each statement regarding the iPhone and iPad respectively.

40

Figure 18: Percentage Graph Results of the Degree of Agreement Across Each SystemStatement on the iPhone. St. Represents Strongly.

Figure 19: Percentage Results of the Degree of Agreement Across Each System Statementof the iPad. St. Represents Strongly.

4.3.2 Touch Screen Gestures Results

The second section of the usability testing questionnaire named �Touch Screen Gestures�

contained 16-21 statements. Each statement presented di�erent usability features such as

41

zooming functionality, gestures memorability and learnability. The results obtained from

the user responses for each statement presented as follows.

I can easily use the text �zoom in� and �zoom out� signs (gestures). The results of

the iPad responses presented that 7% (1) was strongly disagree; 7% (1) was neutral; 7%

(1) was near to agree, 14% (2) was agree and 64% (9) were strongly agree. The iPad

results showed that 10% (1) was near to disagree; 10% (1) was neutral; 40% (4) were

near to agree, 20% (2) were agree and 20% (2) were strongly agree. The analysis of

the relationship showed that there was signi�cant di�erence (p = 0.027 < 0.05). For this

statement the users views result showed that they could easily performed the zoom in/out

gestures using the iPad than the iPhone.

I like the gestures for selecting and zooming the text. The results of the iPhone re-

sponses showed that 14% (2) were near to disagree; 7% (1) was near to agree, 50% (7) were

agree and 29% (4) were strongly agree. Similarly, the results of the iPad responses showed

that 20% (2) was near to disagree; 20% (2) were neutral; 10% (1) was near to agree, 40%

(4) were agree and 10% (1) was strongly agree. The comparative study showed that there

was no signi�cance(p = 0.108 > 0.05) in users opinions for the iPhone and iPad.

I think the gestures are easy to learn. The iPhone results showed that 7% (1) was

disagree, 7% (1) was near to disagree; 7% (1) was neutral; 43% (6) were near to agree, 7%

(1) was agree and 29% (4) were strongly agree. Similarly, the iPad results showed that

20% (2) were near to disagree; 10% (1) was neutral; 20% (2) were near to agree, 50% (5)

were agree. There was no signi�cance (p = 0.424 > 0.05) in users opinions for the iPhone

and iPad.

I think the gestures are easy to remember. The iPhone results showed that 7% (1) was

near to disagree; 14% (2) were neutral; 29% (4) were near to agree, 21% (3) were agree

and 29% (4) were strongly agree. On the other side, the iPad results showed that 40% (4)

were neutral; 20% (2) were near to agree, 30% (3) were agree and 10% (1) was strongly

agree. The comparative study showed that there was no signi�cance(p = 0.214 > 0.05)

between users views for the iPhone and iPad.

I can easily change the location of the contents (icons) on the interface. The iPhone

results showed that 7% (1) was disagree, 7% (1) was near to disagree; 7% (1) was neutral;

7% (1) was near to agree, 29% (4) were agree and 43% (6) were strongly agree. On the

other hand, the iPad results showed that 10% (1) was near to disagree; 20% (2) were

neutral; 50% (5) were near to agree and 20% (2) were agree. There was a signi�cance(p =

0.015 < 0.05) in users opinions for the iPhone and iPad. The results showed that the user

could easily change the contents location using the iPad as compared to iPhone.

I can easily select the written text for deletion. The iPhone results presented that 7%

(1) was strongly disagree, 14% (2) were disagree, 14% (2) were near to disagree; 14%

(2) neutral; 21% (3) were near to agree, 14% (2) were agree and 14% (2) were strongly

agree. Similarly, the iPad results showed that 30% (3) were disagree, 20% (2) were near

42

to disagree; 10% (1) was neutral; 30% (3) were near to agree and 10% (1) was agree. The

analysis relationship result showed that there was a no signi�cance (p = 0.198 > 0.05).

In Figure 19 presented the graphs results of the users opinions against each statement on

both the devices.

Figure 20: Graphs Show Percentage Results of the Degree of Agreement Across the EachStatement of the Touch Screen Gestures on the iPhone and iPad Respectively.

4.3.3 Keypad Results

The third section of the survey questionnaire named �keypad� contained eight statements.

Each statement assessed the performance of the keypad of both devices. The results

obtained from the user's responses for the each statement are presented as follows.

I enjoy using the keypad while writing Emails. The results of the iPhone responses

showed that 7% (1) was strongly disagree, 21% (3) were near to disagree; 14% (2) were

neutral; 14% (2) were near to agree, 7% (1) was agree and 36% (5) were strongly agree.

The results of the iPad responses showed that 10% (1) was strongly disagree, 10% (1) was

disagree; 10% (1) was neutral; 20% (2) were near to agree, 40% (4) were agree and 10%

(1) was strongly agree. The comparative study of the users opinion showed that there

was no signi�cance(p = 0.446 > 0.05) for the iPhone and iPad.

I like the keypad in landscape style while writing Emails. The results of the iPhone

responses showed that 14% (2) were disagree; 21% (3) were neutral; 14% (2) were near

to agree, 29% (4) were agree and 21% (3) were strongly agree. Similarly, the iPad results

43

showed that 20% (2) were strongly disagree, 10% (1) was near to disagree; 30% (3) were

neutral; 20% (2) were near to agree and 20% (2) were strongly agree. The comparative

study of the user's responses showed that there was no signi�cance(p = 0.143 > 0.05)

regarding the iPhone and iPad.

It is easy to use the special characters (e.g., `' _ *). The iPhone user's responses

results showed that 7% (1) was strongly disagree, 21% (3) were near to disagree; 7% (1)

was neutral; 14% (2) were near to agree, 29% (4) were agree and 21% (3) were strongly

agree. On the other side, the iPad results showed that 22% (2) were disagree; 22% (2)

were neutral; 11% (1) were near to agree and 44% (4) were strongly agree. There was no

signi�cant di�erence (p = 0.206 > 0.05) between the user's opinions for the iPhone and

iPad.

The arrangement of alphabets on the keypad is in accordance to my choice. The iPhone

results showed that 14% (2) were strongly disagree, 7% (1) was near to disagree; 21% (3)

were near to agree, 14% (2) were agree and 43% (6) were strongly agree. The results of

the iPad showed: 10% (1) was strongly disagree, 10% (1) was disagree, 10% (1) was near

to disagree; 10% (1) was neutral; 30% (3) were near to agree and 30% (3) were agree.

The comparison between the user's point of views for the iPhone and iPad was showed

no signi�cance (p = 0.080 > 0.05).

Small keys can cause typing mistakes. The iPhone results showed that 7% (1) was

strongly disagree, 14% (2) were disagree, 7% (1) was near to disagree; 14% (2) were

neutral; 21% (3) were near to agree, 21% (3) were agree and 14% (2) were strongly

agree. Similarly the iPad results showed that 10% (1) was disagree, 20% (2) were near

to disagree; 50% (5) were neutral; 10% (1) was agree and 10% (1) was strongly agree.

The comparison of the user opinions about the iPhone and iPad showed that there was

no signi�cance (p = 0.183 > 0.05).

I can easily �nd the number (e.g. 1, 2, 3...) on the keypad. The iPhone results showed

that 8% (1) was disagree; 15% (2) were neutral; 15% (2) were near to agree, 8% (1) was

agree and 54% (7) were strongly agree. On the other side, the iPad results showed that

20% (2) were near to disagree; 30% (3) were near to agreed, 30% (3) were agreed and

20% (2) were strongly agreed. The relationship between the iPhone and iPad responses

showed that there was no signi�cance(p = 0.129 > 0.05).

The single character keypad improves my writing performance. The iPhone results

showed that 7% (1) was strongly disagree, 7% (1) was disagree, 14% (2) were near to

disagree; 7% (1) was neutral; 29% (4) were near to agree, 14% (2) were agree and 21%

(3) were strongly agree. Similarly, the iPad results showed that 10% (1) was strongly

disagree, 10% (1) was near to disagree; 40% (4) were neutral; 30% (3) were near to agree

and 10% (1) was agree. The comparative study showed that there was no signi�cance(p =

0.143 > 0.05) between user's opinions for the both devices.

I can easily move the cursor forward and backward in the editor while writing text.

44

The iPhone results showed that 14% (2) were strongly disagree, 7% (1) was disagree, 36%

(5) were near to disagree; 7% (1) was neutral; 14% (2) were near to agree, 14% (2) were

agree and 7% (1) was strongly agree. Alike, the iPad results showed that 22% (2) were

disagree, 22% (2) were near to disagree; 22% (2) were neutral and 33% (3) were near to

agree. The comparative study showed that there was no signi�cance (p = 0.500 > 0.05)

in user's opinions for the both devices. The Figure 20 represented the graphs results of

users opinions against each statement of the Keypad on the iPhone and iPad respectively.

Figure 21: Graph Shows Percentage Results of the Degree of Agreement Across the EachStatement of the Keypad on the iPhone.

45

Figure 22: Graph Shows Percentage Results of the Degree of Agreement Across the EachStatement of the Keypad on the iPad.

4.3.4 Applications Results

The fourth section of the questionnaire named �Applications� contained 13 statements.

The results of each statement for the both devices are presented as follows.

The applications (e.g. Evernote, calendar) are simple. The iPhone results presented

that 29% (4) were near to disagree; 7% (1) was neutral; 29% (4) were near to agree, 14%

(2) were agree and 21% were strongly agree. On the other side, the iPad results showed

that 11% (1) was near to disagree; 44% (4) were near to agree, 33% (3) were agree and

11% (1) was strongly agree. The relationship between the users opinions regarding the

iPhone and iPad was not signi�cant (p = 0.385 > 0.05).

The applications (e.g. Facebook, calendar) work very well in portrait style. The iPhone

results showed that 7% (1) was disagree, 7%