Usability testing and redesign of Odeon
cinemas website
Coursework for Usability, principles and practice
Course: User Experience Design, M.A.
Jaroslav Sumbal – B629626
January 2017
2
3
Abstract
This study tested and evaluated usability of Odeon cinemas website. Usability testing was
done in two rounds; first round tested usability of the original website, as it was in
November to December 2016. Tests were analysed and numerous issues were identified
in the design. These issues were then addressed in a redesign: an interactive prototype
done in Axure RP8. The prototype was tested in the second round. Analysis, both
qualitative and quantitative, confirmed that the redesign is more usable than the original
website and shed light on other areas that should be improved or changed in next iteration
of redesign if there were any.
Main recommendations from the study are to remove unnecessary visual clutter from the
homepage and promote finding films and cinemas and move the featured films element
down in the page. Next, in the cinema page, move show times up so that they appear
above the fold. When user clicks on a film name on cinema page, remember which
cinema it was so that they do not have to search it again on film page. Introduce the
movie modal as explained in the main text.
Axure share link: http://bit.ly/OdeonRedesign
Instructions for viewing prototype: prototype was designed to be viewed on full HD
screen. If viewed on a smaller screen or on a screen with scaling turned on, it may be
necessary to zoom out in the browser. To view page structure, variables and other data
about structure of the prototype, hover over top left corner of the prototype and click the
downward facing chevron that appears. Note that this will move everything to the right.
That is why it is hidden by default.
4
Table of Contents 1 Introduction .................................................................................................................. 7
1.1 The website ........................................................................................................... 7
2 Sample description and user goals ............................................................................... 7
2.1 Users ...................................................................................................................... 7
2.1.1 User goals....................................................................................................... 7
2.1.2 Tested sample................................................................................................. 8
3 Methodology ................................................................................................................ 9
3.1 Tasks...................................................................................................................... 9
3.1.1 Task 1 ............................................................................................................. 9
3.1.2 Task 2 ............................................................................................................. 9
3.1.3 Task 3 ........................................................................................................... 10
3.2 Running and recording sessions .......................................................................... 10
3.3 Evaluation metrics ............................................................................................... 11
4 First round of testing – original website .................................................................... 11
4.1 Pilot ..................................................................................................................... 11
4.2 Spotted patterns ................................................................................................... 12
4.3 Eye tracking......................................................................................................... 18
4.4 Mobile version..................................................................................................... 18
4.5 Performance results ............................................................................................. 20
4.6 Subjective satisfaction results ............................................................................. 20
5 Second round of testing ............................................................................................. 21
5.1 Testing on prototype............................................................................................ 21
5.2 Redesigned pages and spotted behaviour patterns .............................................. 21
5.3 Performance results comparison ......................................................................... 24
5.4 Subjective satisfaction results comparison .......................................................... 26
5
6 Conclusion ................................................................................................................. 27
6.1 Personal notes ...................................................................................................... 27
References .......................................................................................................................... 28
Table of Figures
Figure 1 Homepage ............................................................................................................ 12
Figure 2 Loughborough cinema page ................................................................................ 13
Figure 3 Loughborough seat selection ............................................................................... 14
Figure 4 Ticket selection .................................................................................................... 16
Figure 5 Seat reservation ................................................................................................... 17
Figure 6 Mobile - Find a cinema....................................................................................... 19
Figure 7 Mobile - Find a movie ......................................................................................... 19
Figure 8 Mobile - seat reservation ..................................................................................... 19
Figure 9 Redesigned homepage ......................................................................................... 22
Figure 10 Cinema page redesigned .................................................................................... 22
Figure 11 Movie modal ...................................................................................................... 23
Figure 12 Comparison of mean number of clicks per task across designs ........................ 26
Figure 13 Comparison of median number of clicks per task across designs ..................... 26
Figure 14 Comparison of means in questionnaire answers ............................................... 27
Figure 15 Comparison of medians in questionnaire answers ............................................ 27
Table of tables
Table 1 Participants overview .............................................................................................. 9
Table 2 Wrong clicks in first round ................................................................................... 20
Table 3 Questionnaire results - first round ........................................................................ 21
Table 4 Wrong clicks in redesign ...................................................................................... 25
Table 5 Comparison of questionnaire answers .................................................................. 27
6
1 Introduction
The task at hand was to choose any website, perform usability testing on that website,
analyse test results, redesign the website for better usability and finally test the new
design to see if it was better that the original and suggest changes for future redesign.
1.1 The website
After some searching and consideration, webpage of the Odeon cinemas was chosen for
the project (http://www.odeon.co.uk/). There were several reasons for this choice:
the site does not require any authentication for successful completion of user
goals, which is extremely important since there was no support from the company
for mock accounts or anything similar. There is some functionality hidden behind
authentication, but it is not very important.
it has a very large target user group available for research
there is a lot of room for improvement
2 Sample description and user goals
2.1 Users
Many people go to cinema, some less and some more often. Therefore, it was easy to find
intended users of the website for the testing. University students were asked to participate
in this study. The sampling method consisted of one question: “Have you ever used this
website before? If so, when was the last time?”
Reason for this is that usability of a system is best tested on first time users who did not
have the time to get used to it (Cooper, Reimann, and Cronin, 2014). Most of the
participants have never used the website before. While some have, it was at least a month
ago, so there was a good chance of forgetting how it worked, since they did not use it
frequently. There were no other restrictions except for age (over 18).
2.1.1 User goals
Because the initiative for this project did not come from the company running the
website, there was no access to any of their user research or business goals that might
help with identifying user goals that the website should support. Therefore, user goals
were extracted from the features of the website and using common sense.
8
First goal identified was to look up what films are shown in the cinema for inspiration
when users just want to go out for a film. Second goal was booking a film. This was
further divided into two sub-goals:
Booking a film ticket when you know which cinema you want to visit and want to
see it there.
Booking a film ticket when you know you want to watch the film but don’t mind
the cinema selection so much as an area where the cinema should be.
The first sub-goal mimics the traditional way of buying tickets: you go to the cinema, see
what is there and buy a ticket. To test more variants, it is impossible to reserve seats
online in this cinema. The second sub-goal is made possible by technological
advancement, where you can check which cinemas show a specific film. Online seat
reservations are made in this cinema.
2.1.2 Tested sample
There were two rounds of testing. One participant attended both and others either the first
or the second. Their short description is shown in the TABLE 1. Recent study in OECD
countries measured IT skills of the general population using four skill levels. This study is
huge, but Nielsen (2016) pulled out the important information about IT skills and
explained them in easily digestible manner. However, it was not possible to properly
assess participants’ IT skills, so instead of using these four levels, their IT skills were
determined only by what they said about it and categorised into three levels. These levels
were used: poor, basic and good.
9
Participant
No.
Sex IT skills Used the site Eye tracking Test
round
1 Male Good Never No 1
2 Female Poor Never No 1
3 Female Basic Never No 1, 2
4 Male Basic Never No 1
5 Male Good Never Yes 1
6 Male Good Never Yes 1
7 Female Good Within last month No 2
8 Male Good Within last month No 2
9 Female Good Never No 2
10 Male Good Within last month No 2
11 Male Good Never No 2
Table 1 Participants overview
3 Methodology
Method used to evaluate usability of the
website is moderated usability testing.
One reason for this choice is the time
and budget allowances for this project,
which, considering it is a student work,
were not great.
3.1 Tasks
Three tasks written as scenarios that
were linked into one overall scenario
were devised from identified user goals.
The core of each task remained the same
throughout the testing, but the wording
changed after the pilot test for better
clarity.
While task 2 and 3 may seem similar,
there are great differences in the booking
process when done in a small cinema
like the one in Loughborough compared
to a big London cinema.
3.1.1 Task 1
“You want to go watch a film tonight
that you might like at Odeon cinema in
Loughborough, that plays after 19:00,
maybe a comedy. You're not really
decided which film exactly you want to
watch, so you hope that the cinema page
will help you with that. Find such film
and watch the trailer.”
This task was following the first goal to
see how the site enables users to see
what is shown in the cinema and how
clear is the navigation.
3.1.2 Task 2
“After watching the trailer, you realise
that this film is not for you, so you check
10
out other films after 19:00. Choose one
and book a ticket.”
This task was following the first sub-
goal of the second goal to see how the
site enabled users booking of a film
when they are in the cinema.
3.1.3 Task 3
“Just before checkout, you've
remembered that you're going to London
in 2 days and want to go watch Doctor
Strange with a friend in the evening. So,
you think that 2 films in one week would
be too much, so you don't buy these
tickets but book tickets to see Doctor
Strange with a friend in London in
Odeon cinema of your choosing.” – This
film was not played in every London
cinema, only big ones with seat
reservation. So, the choice was merely
an illusion.
This task was following the second sub-
goal of the second goal to see how the
site enabled users booking of a film in a
specified area with multiple cinemas
present.
While this task is plausible (cancelling
tickets in the beginning), as in it can
happen to users, it would be good to
know if the company wanted to make
this scenario easy. One could argue that
when a user is already on a path to
purchase, there should be as few ways
out as possible to “force” the purchase.
Therefore, the way out of the booking
flow was not redesigned, rather only the
subsequent part after user successfully
abandoned the booking path.
3.2 Running and recording sessions
Most testing sessions were done in a
standard environment on the same laptop
with consistent environment with
exception of the eye tracking sessions.
For better analysis of data, screen was
recorded, along with the face of the
participant using the laptop webcam.
At the start of a session, each participant
was welcomed with short conversation,
notified about the purpose of the
research and recording and was asked to
sign an informed consent form. After
that they were told the website was being
tested and not them no ease the possible
tension and asked to think aloud.
Tasks were printed out on pieces of
paper and handed to participants in order
after the previous task was completed.
Participants were asked to read the task
aloud and during the test encouraged to
look at it again, as often they lost focus
of their task.
After completion of all the tasks, they
were asked to fill in a short questionnaire
about their experience with using the
11
website and then add any comments they
wanted.
Finally, they were thanked for their time
and rewarded with a little sweet treat as a
little surprise as they were not promised
anything.
3.3 Evaluation metrics
Decision was made to take a more
qualitative approach rather than
quantitative to measure the usability of
the website. This means that while
timing was recorded, it was not used as a
metric, because participants were
encouraged to think aloud and
sometimes asked questions about their
actions, which makes time measurements
distorted.
Another reason for this decision is the
number of participants and successfully
collected data points. With only 6
participants and sometimes fewer data
points (read more in section 4.5),
reliability of statistical tests is
questionable.
Therefore, all metrics recorded that
could be potentially used as quantitative
with a bigger sample were treated more
as qualitative. Nielsen (2001) makes a
point about this and talks about using
quantitative measures in usability testing
in much greater detail.
One metric considered was the number
of wrong clicks per task. Wrong click
was defined as follows: a click that
moved a participant further away from
completing the task at hand, or that did
not at all do what the participant
expected, for example clicking on an
item which did not do anything.
Other metrics were gathered by
questionnaire. These measured, on a
scale from 1 to 5
the overall perceived ease of use
as well as per task
how text and headings made
sense to participants
how using the site made
participants feel: sad to happy (or
just negative to positive)
how the site was organised,
whether it made sense
4 First round of testing –
original website
4.1 Pilot
The pilot test proved very useful as it
showed that wording of the tasks needed
to be changed to improve clarity and
participants’ understanding of the tasks.
Because of this, results from the pilot
were not considered greatly but were a
necessary step in the process.
12
4.2 Spotted patterns
To analyse results, notes were taken
from looking at screen recordings. They
were then used to make affinity diagram
to see which problems occurred the most
and should be made priority for redesign.
There is a common problem for the first
and third tasks: the homepage (FIGURE
1) is not well laid out. It does not give
enough visual significance to important
features for successful completion of
these tasks, contains a lot of noise and
confuses users a lot. This was part of the
main focus of the redesign.
Figure 1 Homepage
Most users were confused by the menu,
because it reacted on hover. They
expected to click on it, and when they
moved cursor too fast and clicked,
sometimes it froze and they were stuck.
Another point of confusion was the film
page, which was substituted by movie
modal (more in section 5.2 on page 21).
Confusion arose when participants
clicked on the name of the film in
Loughborough cinema page, and
suddenly they just saw reviews. They did
not notice small dropdown to choose a
cinema. One participant even went back
and did it again as she thought she has
done something wrong. And when they
noticed the dropdown, they did not
understand, why they had to choose the
cinema again. The website, for some
reason, did not remember that they
Reacted on hover
13
clicked on the film when in
Loughborough cinema page and did not
automatically choose it.
Second part of the main focus was the
cinema page, when user navigates to a
concrete cinema to check show times or
other information. FIGURE 2 shows what
user sees above the fold after navigation
to the page of Odeon in Loughborough
(on laptop with 15.6” screen and scaling
turned on). No show times, no relevant
information. Some users even thought
that they did not navigate to the right
page at all. It is necessary to scroll down
to see relevant content, but there is no
indicator of that.
Figure 2 Loughborough cinema page
Next, during booking process for a
smaller cinema such as Loughborough,
there is a confusing screen about seat
selection. It shows seat layout and over
that a message that one can choose seats
when one arrives to the cinema. Most
participants did not understand the
message and tried to reserve a seat and
payed attention to the message only after
unsuccessful attempts.
14
Figure 3 Loughborough seat selection
Seat selection was very confusing for
bigger cinemas too, because users first
had to choose a section of seats (FIGURE
4) and then seats on next page (FIGURE
5). There were two big issues with this.
Firstly, majority of users (except for one)
did not even notice there were different
sections and therefore went with the
preselected and thus on the next page
they were confused when they could not
choose any seat they wanted. Secondly,
on the seat selection page, it was not
clear where the screen was and what
does each colour mean, as the legend
was below the fold and some participants
did not notice it at all. So, they first
chose seats thinking they were on the
other side of the room than they were
and then scrolled down to click continue
without noticing that the screen was on
the other side of the room.
The legend itself was not well designed,
because it was not explained what grey
seat means. It says which seat is
available (blue) and which is occupied
(blue with user icon in it). But it
neglected grey seats. Some participants
thought, that because there were also
user icons on grey seats, that grey were
available too. This comes hand in hand
with the fact that most users did not
notice that there were different sections
of seats and so expected to be able to
choose any seat they wanted.
Participants tried
to click on seats
15
Recommendations for future redesign
are to abandon choosing section of seats
before user sees the seats and move it to
the seat selection page. That way user
would have direct connection between
chosen section and available seats.
Going back to ticket selection on
FIGURE 4 for 3D shows, users did not
appreciate that before they checked the
option to have 3D glasses, they did not
know they cost anything. When £1
appeared after selection, they were
unpleasantly surprised. Another common
comment was that: “Why are there two
checkboxes for the same thing? Why not
just one item with a quantity next to it,
just like tickets?” That is a valid
suggestion and would improve usability
of the site, as the current solution
increases cognitive load of users for no
good reason.
16
Figure 4 Ticket selection
Should be moved
to next page
Prices should be
displayed up front.
17
Figure 5 Seat reservation
This was not visible above
fold, mostly unnoticed
What are grey
seats?
18
4.3 Eye tracking
The original website was tested using eye tracker system available with two participants.
There were unexpected problems. The system was not optimised to work with heavily
dynamic websites. Content on this website was dynamically changing based on user’s
actions, and the eye tracker showed only first screenshot of the page before any
interaction. For this reason, heatmaps are irrelevant, as it is not possible to determine why
would user look at a certain place, because the content has probably changed. Also,
videos with the eye movement behaved in the same way and were therefore completely
unusable for this project.
Another issue stemming from this is that it was not possible to ascertain number of wrong
clicks with these two participants, because it is not possible to determine where they
clicked and whether it was wrong.
4.4 Mobile version
The website is not responsive, but it has an alternate mobile version at
http://mobi.odeon.co.uk/. This version contains less clutter and visual noise than the
desktop version. It allows the completion of all goals defined before (see FIGURE 6 and
FIGURE 7), however, reserving a seat is extremely difficult.
At first, all seats are shown and it is impossible to accurately pick the one user wants with
a finger (see FIGURE 8 MOBILE - SEAT RESERVATION). There is an option to zoom in,
but then it is difficult to drag the seats to reveal the one user wants. Also, during the test
the page froze after zooming in, making any other action impossible. Other than that, less
screen space forces to show only important things and it is in a way easier to accomplish
some goals on this version than desktop version.
Pag
e19
Figure 6 Mobile - Find a cinema
Figure 7 Mobile - Find a movie
Figure 8 Mobile - seat reservation
20
4.5 Performance results
While there were six participants in the
first round, data collected from pilot
testing cannot be used as some
conditions and tasks changed afterwards.
Number of clicks was gathered by
watching back recorded videos. Last two
tests were done using eye tracker system.
Because of major issues mentioned
earlier, it was not possible to get the
number of clicks for these tests. Thus,
the number of data points was halved to
𝑁 = 3, so statistical analysis provided
should be taken with a grain of salt.
Participant No. Task 1 Task 2 Task 3
P2 4 2 6
P3 1 3 4
P4 10 2 4
mean 5 2.33 4.67
median 4 2 4 Table 2 Wrong clicks in first round
4.6 Subjective satisfaction results
At the end of section 3.3 on page 11, outline of questionnaire was mentioned. All
questions used Likert scale style of options, but were optimised for measuring usability.
Classical Likert scales have one disadvantage in general: they influence participants in the
question, as it is not open ended, but yes or no style of question (Cooper, Reimann, and
Cronin, 2014). Therefore, instead of statement: I found the site easy to use; with options
of strongly agree to strongly disagree, there was a statement: Overall, the site allowed me
to complete the tasks; with options on one end saying “hardly” and on the other “easily”.
Data points were collected for all but one participant, where technology failed and data
were lost.
These questions were asked (numbers are used in TABLE 3 and TABLE 5):
1. Overall, the site allowed me to complete the tasks (hardly to easily)
2. Using the site made me feel (sad or negative to happy or positive)
3. Text of headings and buttons (confused me to made sense)
4. The site was organized (badly to well)
5. The site allowed me to complete the first task (hardly to easily)
6. The site allowed me to complete the second task (hardly to easily)
7. The site allowed me to complete the third task (hardly to easily)
21
Participant No. q. 1 q. 2 q. 3 q. 4 q. 5 q. 6 q. 7
P1 4 3 5 5 5 4 5
P2 4 3 4 3 4 5 3
P3 4 4 3 2 2 4 4
P4 4 3 5 5 2 5 5
P5 3 3 4 3 2 4 4
mean 3.8 3.2 4.2 3.6 3 4.4 4.2
median 4 3 4 3 2 4 4 Table 3 Questionnaire results - first round
5 Second round of testing
First and second round of testing were
done more than a month apart, so the last
task had to be slightly adjusted to reflect
changes in what film was currently
trending. Otherwise the tasks and
environment were the same.
5.1 Testing on prototype
Second round of testing was done on a
prototype of the redesign. This factor put
greater importance on the role of the
moderator as not every feature was
implemented or completed.
The prototype fully supported the
completion of all goals, with exception
of implementation of genre filter, which
proved to be a technical challenge in
Axure RP8, a prototyping tool chosen
for the job.
5.2 Redesigned pages and spotted
behaviour patterns
As mentioned earlier, major redesign
was done on the homepage and cinema
page, and some small things were
redesigned in the booking flow.
The redesigned homepage can be seen in
FIGURE 9. Featured films element was
moved down and does not span the
entire width of the page to avoid
overwhelming users. Search bar and
browsing options were given a priority.
Interestingly, only one participant used
the featured films element, others went
straight for search or browsing option. It
is questionable whether it is even
important.
22
Figure 9 Redesigned homepage
Figure 10 Cinema page redesigned
FIGURE 10 shows redesigned cinema
page. Instead of being confused by
meaningless “Hello” taking up most of
the space above the fold, users can see
show times right away. This eliminated
the confusion that some users
experienced with the original design,
when they were not even sure that they
are on the right page. As Krug (2013)
highlights as the most important
usability rule, users did not have to think
where they are or what are they supposed
to do now.
Individual show times are aligned in a
grid to make finding relevant options
easier. To prevent visual clutter by
drawing a complete table, principle from
23
Gestalt theory was used saying that
people complete patterns in their minds.
Movie titles and images are all
concentrated on the left side, because
that is how users, at least in our part of
the world, scan the page. Users scan the
page from left to right and from top to
bottom, creating an F shape. Most
important information should then be in
the top left corner (logo of the website
serving as a link to homepage, and then
the heading of the current website, so
that user always know where they are).
Whether the user will start scanning
more to the right and then down, or first
down and then right depends on the
layout. Design can lead the eyes where it
wants, to certain extent (Florentin,
2015).
One problem was detected in the tests:
difference between times for 2D
showings and 3D is not clear. Some
participants clicked on a time, then in
movie modal change the version and
then noticed that time has changed and
did not understand why it happened.
Then did it all over again and then
noticed. These times should be more
clearly differentiated, maybe by colour
or put them further away from each
other.
Another problem is that participants
expected to find time filter. Despite show
times being laid out in table like fashion
for easy navigation, users wanted to
make their life easier by filtering show
times within specified range. This
feature should be added in the next
iteration.
Figure 11 Movie modal
24
FIGURE 11 shows new page element
introduced with the redesign as a
substitute for movie page, internally
called the movie modal. This modal
shows all details of a movie and enables
users to check their selection or change
it. One usability principle is
“Recognition rather than recall”, as
identified by Jakob Nielsen (1995).
Movie modal embodies this principle by
brining all important choices of movie
selection together in one place.
It can be accessed anywhere on the
website where user can choose to see
details of a film. Options available in
prototype are: click on item in featured
films carousel, click on returned result
from search, click on item when
browsing movies, click on movie
picture, name or show time on cinema
page or looking up movie in the main
navigation.
This element is the gateway to booking a
film and therefore easy discoverability is
very important. The ability to change
choice of cinema, version, date and time
helps users to recover from errors. While
completing third task, one user used
browser navigation to go back several
steps from booking confirmation page
back to Loughborough cinema page.
There she clicked on target film she
wanted to watch in London. Then, when
she realised she is still in Loughborough
cinema, she changed her choice to
London in the dropdown. When she was
asked whether she did this on purpose,
whether she knew she could do this
before she has done it, she said that she
clicked on the movie just randomly and
then noticed she could complete the task
by changing the options.
Very common problem was during
transition from task 2 to task 3. There
users found themselves in a situation
with unfinished booking, needing to
cancel it and start a new one. As
mentioned in section 3.1.3 on page 10,
this flow was not redesigned, but it could
be improved. Internet users today are
used to online shopping, and participants
often thought that those tickets were in
some basket. What often happened was
that users tried to find a way to cancel
the tickets before going back, or change
them. They did not realise that they
could just return to homepage and start
over. Therefore, it would be good to add
the option to cancel current tickets. It
does not have to be obvious, but a simple
underlined link in bottom left corner
somewhere would help ease the tension
in users.
5.3 Performance results comparison
In the second round, data points were
collected for every participant. Mean and
25
median were calculated for this data set
as well (see TABLE 4) and one tailed
student’s t-test assuming equal variance
was applied to see if changes found were
statistically significant (note that with
such small samples, reliability of these
tests is not guaranteed, but it can provide
a picture). Significance value chosen was
standard 𝑝 = 0.05. Hypothesis was
always along the same lines: Users do
fewer wrong clicks in redesign than
original.
Participant No. Task 1 Task 2 Task 3
P3 0 0 1
P7 0 0 2
P8 2 0 2
P9 0 2 4
P10 0 1 1
P11 12 0 2
mean 2.33 0.5 2
median 0 0 2
p-value 0.226233439 0.006037575 0.005805456 Table 4 Wrong clicks in redesign
Tasks 2 and 3 have significantly
decreased the number of wrong clicks,
which in turn leads to less confusion and
faster completion of tasks. Task 1 was
not significantly different because of
participant 11. This value is an outlier
and could be ignored from the data set. If
it were ignored, changes would be
significant with 𝑝 = 0.03, and 𝑚𝑒𝑎𝑛 =
0.40.
Decrease in wrong clicks can be seen in
FIGURE 12 and FIGURE 13. Here, the
outlier value for task 1 was ignored to
provide different look on the data.
Median did not change (because it was
an outlier), but mean was affected
dramatically.
26
Figure 12 Comparison of mean number of clicks per task across designs
Figure 13 Comparison of median number of clicks per task across designs
5.4 Subjective satisfaction results
comparison
Answers from all participants were
successfully recorded in the second
round. However, after applying the same
test for significance, only question 5 got
significantly better results, while
question 1 almost got it. These are
shown in FIGURE 14 and FIGURE 15.
For detailed values see TABLE 5
(question are written in section 4.6 on
page 20).
5
2.33
4.67
0.40 0.5
2
0
1
2
3
4
5
6
Task 1 Task 2 Task 3
Mea
n o
f w
rong c
lick
sComparison of mean number of clicks per task across
designs
original redesign
4
2
4
0 0
2
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
Task 1 Task 2 Task 3
Med
ian o
f w
rong c
lick
s
Comparison of median number of clicks per task
across designs
original redesign
27
Participant No. q. 1 q. 2 q. 3 q. 4 q. 5 q. 6 q. 7
P3 5 3 4 5 5 5 4
P7 5 5 5 5 5 5 5
P8 4 3 4 3 3 4 3
P9 4 3 5 5 5 5 3
P10 4 3 4 4 5 4 2
P11 4 4 3 4 4 5 5
mean 4.33 3.50 4.17 4.33 4.50 4.67 3.67
median 4 3 4 4.5 5 5 3.5
p-value
0.05194
4
0.24585
2
0.47302
5
0.14596
7
0.0280
6
0.21398
7
0.21398
7 Table 5 Comparison of questionnaire answers
Figure 14 Comparison of means in questionnaire
answers
Figure 15 Comparison of medians in questionnaire
answers
6 Conclusion
Both qualitative and quantitative analysis clearly show that the redesign is more usable
than the original design. Throughout the text were mentioned several suggestions on how
the redesign could be improved to increase its usability.
6.1 Personal notes
I learned a lot doing this project. My moderating skills have improved from session to
session. I also got better in prototyping and learned to appreciate more how different
people use the web differently and that one cannot rely purely on best practices. I would
have liked to learn to work more with the Eye tracker, which failed me in this project.
Word count: 4 333 (excluding abstract, tables and references)
0
1
2
3
4
5
Comparison of means
Original Redesign
0
1
2
3
4
5
6
Comparison of medians
Original Redesign
28
References
Cooper, A., Reimann, R. and Cronin, D. (2014) About face: The essentials of interaction
design. United States: John Wiley & Sons.
Florentin, T. (2015) Design for a perfect screen. Available at:
http://lp2.webydo.com/design-for-a-perfect-screen-ebook.html.
Krug, S. (2013) Don’t make me think, revisited: A common sense approach to web
usability. 3rd edn. Boston, MA, United States: Pearson Education (US).
Nielsen, J. (2001) Usability metrics. Available at:
https://www.nngroup.com/articles/usability-metrics/ (Accessed: 30 December 2016).
Nielsen, J. (2016) The distribution of users’ computer skills: Worse than you think.
Available at: https://www.nngroup.com/articles/computer-skill-levels/ (Accessed: 30
December 2016).
Nielsen, J. (1995) 10 Heuristics for user interface design. Available at:
https://www.nngroup.com/articles/ten-usability-heuristics/ (Accessed: 20 January 2017).