asos ux lab test
TRANSCRIPT
9 March 2018
DATEREPORT PREPARED BY VER 1.0spacebetween.co.uk
01233 800 991
ASOS UX Lab Test
1Confidential documentPrepared for ASOS
Table of Contents2
Overview
Executive summary
Session information
An introduction to your lab sessions
About the candidates
Tasks
Task details
Brands used
Analysis
Emotional Response
Eye Tracking
Questionnaire
Task completion time
Observations
Homepage
Category / Search
Product listing
Product details
Basket
Checkout
Issues and Opportunities
Basket Popup
Basket Delivery Cards
Checkout Error Notifications
Product Size Preselection
Payment Card Input
PLP Filters
Downloads
Thank you!
Executive summary 3
Top Learnings
1. Basket delivery option cards (and delivery info
icon) open up a new page leading to user
confusion and the interruption of their journey.
2. Checkout create account step error
notifications obstruct the required input fields
without the ability to easily remove, resulting
in the user becoming frustrated due to the
roadblock.
3. Users expect their size to be pre-selected in
the PDP if they used it to filter results on the
PLP.
4. Users expressed frustration at not being able
to see the applied filters on the PLP page.
5. The payment card number input field can be
quite hard to follow for some users, with users
receiving validation errors while trying to
input.
6. Time to checkout was marginally faster on
competitor brands but this did add additional
understanding of the products.
Next Steps
1. Test - Look to implement a solution to
displaying the delivery option information
without having the user navigate away. An
off-screen modal would be a solution for this.
2. Test - Consider a design solution to make the
checkout account step error notifications less
intrusive (under field, add click to close ability,
move notifications to the sides etc.) in the
context of how the error notifications are used
site-wide or a localized fix.
3. Investigation - The ability to pre-select a users
size on the PDP when they’ve selected their
size previously on the PLP.
4. Test - Consider and design mechanisms that
allow users to see which filters are being
applied, such as using filter cards like Boohoo
or displaying the selection within the filter.
5. Test - Automatically showing the card number
input in blocks of 4 to see if we receive a
reduction in errors.
6. Test - Add in a callout for proceeding to
checkout when somebody adds a product to
the basket, this would move them past the
basket and to the checkout flow. Test to ensure
that basket value doesn’t reduce.
Introduction
This tests has been designed to explore a typical customer
journey on the ASOS website and benchmark it against
two competitor websites, Boohoo and Zara.
Understanding
We’ve sourced candidates based on overlap with your
main target demographic. Our test group consisted of
young professionals between the ages of 20 - 32.
Aim
Our aim is to find useful data and opportunities for
optimisation within your website. We’re looking for areas
of confusion or difficulty that the user has come up
against, as well as usage patterns that can provide useful
insights.
Session information
4
5An introduction to your lab sessions
Introduction
The following pages outline the tasks carried out on the
8th March 2018. The report explains the observations,
highlighting pages visited and any points raised.
We’ll take this data and formulate a hypothesis for future
changes that would be validated using A/B testing.
Towards the end of this document, you will find our
recommendations for tests that are deemed to have the
highest probability of increasing conversion rate.
Lab Setup
All of our candidates completed the test tasks in a random
order to maintain a controlled testing environment.
All tasks have been run in the Space Between lab based at
the Space Between offices and have been completed on a
desktop computer.
All candidates used the following 5 inputs:
● Galvanic skin response (GSR)
● Eye tracking
● Facial recognition
● Screen recording
● Heart rate monitoring
Methodology
Emotions
We use the candidates emotions to report on how they are feeling through the tasks and if there are specific pain points. To do this we use facial recognition to analyse a video feed, we take 30 pictures every second and map various sections of the candidates face and then track the sections over time. An example of this would be ‘brow furrow’ where we track how the candidates eyebrows move.
We then take a profile of the facial expressions registered
and map them to the corresponding emotions.
GSR
We monitor a candidates emotional arousal & stress by
measuring changes in the conductivity of the skin. The
sensor monitors skin conductivity between two reusable
electrodes attached to two fingers of one hand, when
shown a stimulus the sweat glands become more active,
increasing moisture on the skin and allowing the current
to flow more readily by changing the balance of positive
and negative ions in the secreted fluid (increasing skin
conductance).
About the candidates 6
Number of Candidates
100
5
Gender distribution
Female Male
40% (2)
Average technical awareness
100
8/10
Average brand awareness
100
8/10
Average previous customer
100
3/10Homeowner
100%0%
20%
Average household income
£100,000£15,000
£40,000
Introduction
We take the utmost care when choosing the candidates for your sessions, ensuring to get a diverse
range of people from different backgrounds, relevant to your demographic and user personas.
60% (3)
“Task
7
On the following website, please look for a pair of jeans and shoes that you like and would fit, add them to your shopping cart and then checkout when ready.
When prompted to sign in or create an account, please create a new account and pay with the provided credit card information.
Task: Find and purchase jeans and shoes
Candidate task
During the course of the test, the participants were presented with three
fashion ecommerce websites and tasked to look up and purchase a new pair of
jeans and shoes for an upcoming event.
Before and during the task each candidate was told to think aloud throughout
the process.
Candidates were unaware of the brand they were testing for.
Focus
We focused our task and findings on the below points:
● How participants find and pick the products they want
● Which route they take when purchasing multiple products
● Differences in checkout patterns and completion times
● Uncover any pain points / opportunities along the way
Journey
This task was designed with the understanding that candidates would choose a
variety of routes to fulfil the goal providing us the opportunity to measure,
compare and draw insights.
8
Home
Category / Search
Product listing
Product details
Product listing
Product details
Basket
Checkout (Guest account)
Task: Find and purchase jeans and shoes 9
Brands Used
ASOS Boohoo Zara
boohoo.com
asos.com zara.com
Analysis
10
Analysis: Emotional Response 11
Positive Emotional Response
We wanted to understand how participants responded emotionally
throughout their journey on the three websites.
In this section, we are looking for positive valence emotions,
Engagement and Joy, each captured through a different input (GSR
and Eye Tracking).
The two positive valence emotions we look at are Engagement
using GSR and Joy using Eye Tracking.
One important note is that the results for Zara need to be put in the
context of participants being frustrated to the point of hilarity,
especially on the product listing pages. Because of this, the joy and
engagement indicators have been falsely enhanced on Zara.
When looking at the average engagement time percent a
participant felt a positive emotion, we notice that they felt 49%
more engaged with ASOS then they did with Boohoo.
Participants engaged better with the imagery, the message, and
general website look & feel of ASOS, 95% better than Boohoo.
Analysis: Emotional Response 12
Negative Emotional Response
We like to use Emotional Responses as indicators for further
investigation. We ask questions like “Why does Boohoo have
significantly less disgust time than ASOS?” which then lead us to
investigate each instance of Disgust that our candidates have
experienced. Often we find repeatable patterns during the
investigation such as each Candidate registered Disgust when
faced with the basket on ASOS.
We can use the “Aggregated Emotion Time by Brand” graph to
understand the types and amount of emotions felt by Candidates.
Zara had significantly more emotions registered than either of the
other two brands, Zara also had more joy registered than either of
the other two brands, though this was not necessarily a good thing
as our Candidates were, in fact, laughing because they perceived
the website to be poorly put together. These were not entirely
positive experiences for the candidates and as we will dig into the
reasons further on.
We also look for stimuli, like Boohoo, where the candidates showed
abnormally low levels of engagement this is often an indication that
the candidates were entirely ambivalent to the experience.
Analysis: Eye Tracking 13
Product details - Heatmap
Attention analysis
A heatmap is generated by aggregating all participant
eye-tracking data in order to produce clusters that help us
better understand where the users gaze is concentrated.
Looking at the product details page heatmap, we can notice
three main clusters where the user gaze fixates. In the order of
Time to First Fixation (TTFF):
1. The first cluster corresponds to the product media
gallery which is what attracts the participants gaze
first and where they spend the most time.
2. The second cluster is located around the media
gallery thumbnails, with a focus on the video play
button.
3. The third cluster corresponds to the size selector
and “ADD TO BAG” CTA.
Analysis: Eye Tracking 14
Product details - Areas of Interest
Attention analysis
No one looked at the wishlist whilst only one person looked at the
share button.
Candidates were immediately drawn to the product imagery and
spent the majority of their time on this page looking at the imagery,
18.7 seconds on average.
All candidates looked at the colour of the product to validate their
choice and so only spent 0.7 seconds on average looking at it.
⅗ candidates looked at the video button and spent on average 2.3
seconds.
Looking at the number of interactions on both the product image
and thumbnails we can say that if you discount of the product image
clicks due to zooming that the candidates preferred to use the
thumbnails to navigate through the product gallery.
Recommendation
We recommend further investigation into a users journey to
understand if they flip-flop between multiple variations of the same
product to find their perfect colour. If this is, in fact, a contentious
issue for the users we would recommend making the colour choice
field interactive to facilitate easy swapping between the product
variations.
Analysis: Eye Tracking 15
Product details - Areas of Interest Chart
Analysis: Eye Tracking 16
Competitor: Boohoo - Product details page
Candidates didn’t spent as long using the Product Image area as on
ASOS due to the larger Thumbnails presented.
Analysis: Eye Tracking 17
Competitor: Zara - Product details page
With Zara’s floating navigation that overlays the main product
imagery the candidates were overwhelmed and the heat map shows
a lot of attention wasted.
Analysis: Exit Questionnaire 18
Net Promoter Score
Analysis
The Net Promoter score measures customer experience and helps
predict business growth. It is considered the core measurement for
customer experience management programs around the world.
Subtracting the percentage of Detractors from the percentage of
Promoters yields the Net Promoter Score, which can range from a low
of -100 (if every customer is a Detractor) to a high of 100 (if every
customer is a Promoter).
ASOS scored very well with a positive NPS of 40 while customers were
neutral about Boohoo with a score of 0 and rather negative about Zara
which managed the lowest possible NPS score of -100.
Analysis: Exit Questionnaire 19
System Usability Scale
The Systems Usability Scale is the most used questionnaire for
reliably measuring perceived usability.
It has become an industry standard with references to over 600
publications and looks at:
● effectiveness (can users successfully achieve their
objectives)
● efficiency (how much effort and resource is expended in
achieving those objectives)
● satisfaction (was the experience satisfactory)
It is based on a 10 item questionnaire and participants will rank
each question from 1 to 5 based on how much they agree with the
statement they are reading. 5 means they agree completely, 1
means they disagree vehemently.
The average SUS score we want to report to is 68.
ASOS scored 87 which is very good, placing it at the top of the B
bracket and gets an Excellent grade.
Boohoo did Good with a score of 76.5 while users found Zara’s
usability lacking, giving it a Poor grade based on a score of 41.
SUS Rating Brackets
90-100 A
80-89 B
70-79 C
60-69 D
Less than 60 F
Grading SUS Key
92 Best imaginable
85 Excellent
72 Good
52 OK/Fair
38 Poor
25 Worst imaginable
Analysis: Task Completion Time 20
Task Time
Total Time
An important metric to consider is the time it takes participants to
complete certain tasks. This is a good indicator of how easily and
efficiently they can use the system in order to achieve their goals.
The total task completion time on ASOS is marginally lower than
Zara, however a significant 14.5% higher than Boohoo.
Time to Checkout
Looking at the time it took participants to pick their items and get to
the checkout (Time to Checkout) it becomes apparent that this is
the area where Boohoo gains most of its lead. Participants got to
the checkout 22.1% faster on Boohoo than ASOS.
Checkout Time
The time it took participants to go through the checkout process is
relatively close between the three websites, however, ASOS did
come third, being a full 10.7% slower than Zara who has the
quickest checkout process.
Analysis: Task Completion Time 21
Page Time
Breaking down the time participants spent on each page helps us
better understand where the differences come from.
Looking at the Boohoo breakdown, we notice that the category and
basket pages are missing as they seem to have optimized the
journey to remove as many steps as possible.
Boohoo makes the main navigation completely available to the user
right from the homepage. This removes the need for participants to
select a category (Men/Women) in order to be able to navigate to
the product listing pages.
In addition, Boohoo allows the users to proceed to checkout from
any page, without forcing them to go through the basket page first.
5 out of 5 participants chose to use this feature, skipping the basket
page and making the task completion time approx 9% faster.
The other side of this is the additional confidence gained from
looking through these additional pages. We found that overall the
happiness was higher on ASOS so we’d look to A/B test this
solution. As this would be a more complicated implementation time
and the stats aren’t all on side, we’d consider this low priority.
Analysis: Task Completion Time 22
Page Time
We measured the average time participants spent on each page in order to uncover patterns and draw insights.
Observations
23
Journey: Step 1 24
Homepage
Notes
The participants generally found the ASOS Homepage very
appealing and intuitive.
Their visual gaze quickly connected with the faces within the hero
image then easily found the categories in the main navigation,
spending an average of 15.5 seconds on the page.
A participant seemed a bit confused after selecting MEN's and then
not being able to return to the homepage when they clicked on the
ASOS logo.
A participant commented that they loved the homepage image,
found it high quality and energetic, they liked the diversity in it.
Journey: Step 2 25
Category / Search
Notes
Most participants did not scroll through this page but rather used
the product navigation to locate the category they were looking for.
3 out of 5 participants used the category navigation to proceed to
the lister page while 2 out of 5 used the search feature.
While not statistically significant, it’s worth noting that the search
user journey had a 87% faster Time to Checkout time.
Journey: Steps 3 & 5 26
Product listing
Notes
Participants intuitively used the product listing page without any
notable areas of confusion of frustration.
Participants easily found and used the filters in order to narrow
down to the product they wanted. One participant specifically
mentioned the size filtering pattern to be very useful.
Participants appreciated the product listing layout as they felt it was
easy to scan through and liked the fact that the models are in
different, interesting poses.
2 out of 5 participants mentioned the filtering experience would be
improved if they could specifically see what filters have been
applied. One of these participants specifically mentioned they
preferred the pattern used by Boohoo.
Journey: Steps 4 & 6 27
Product details
Notes
Participants did not experience any major areas of frustration or
confusion while using the product details page.
Participants really liked the video functionality which they feel helps
visualize how the clothes actually fit and is an important factor in
their decision to buy. One participant even noted it is her favorite
feature on the site.
A participant was frustrated by the video loop/autoplay feature
which they felt they had to pause manually. Another participant
noted that the product image slideshow arrow navigation will skip
over the video
A participant mentioned he enjoyed the add to cart
micro-interaction animation and eye tracking data would suggest
most participants noticed it.
A participant was expecting the size they used to filter and pick his
jeans to be automatically selected in the product details page.
Another participant mentioned this expectation as well but was less
frustrated about it.
Journey: Step 7 28
Basket
Notes
For the most part, participants found the basket page intuitive and
easy to use, the only exception being the delivery cards and the
information icon.
3 out of 5 participants tried to click on the “FREE* STANDARD
DELIVERY” banner below the bag contents thinking it would select
free shipping. They were surprised and confused when a new page
popped up with all the delivery options information and no obvious
means to go back or the ability to select a delivery type.
Another participant clicked on the delivery info icon next to the
delivery picker drop-down with the unexpected page opening
causing a similar reaction.
Participant felt that "Items reserved for 60 minutes" message
added urgency to the process and make them want to order right
now since the items might not be there later.
A participant noted they loved the ability to change the size straight
in the basket because they would not have to go back to the product
details page if they notice they made a mistake or changed their
mind at the last minute.
Journey: Step 8 29
Checkout
Notes
The participants managed to proceed through the checkout process
without encountering significant blockers or areas of intense
frustration or confusion.
A participant mentioned they found it difficult to locate the control
that would allow them to checkout as a guest (login/signup screen).
Eye tracking analysis on the login/signup screen would suggest
most participants found it difficult to locate the control that would
allow them to checkout as a guest. This may be the desired
behaviour as a brand.
Two participants encountered an error when using the address
locator (based on postcode) because the auto-filled address was
longer than the parcel label limit. Both participants felt defensive as
they did not consider it was their fault or concern.
A participant got frustrated with the length of the checkout process
when prompted to create an account (last step).
Another participant did not want to create an account and was
expecting it to be optional. When trying to place the order, the
account input fields displayed error notifications that could not be
easily removed and obstructed access to the fields. The participant
felt very frustrated and mentioned that this could have potentially
made them abandon the purchase.
Issues and Opportunities
30
Issues and Opportunities 31
Basket Popup
Analysis
Given the option to proceed straight to checkout from any page,
users seem to prefer using it instead of going through the
intermediary basket page.
Boohoo achieves this by using a popup that is displayed when the
users hover over the basket icon. The pop-up is optimised to pass
people to the checkout with the addition of a “CHECKOUT” CTA
which all test participants chose to use.
Not forcing the users to go through the basket page could
potentially reduce average basket value and reduce the ability to
upsell, our prediction is that this would be offset by introducing the
upsells at a different point in the journey.
Recommendation
● Understand technical and CX design context and
limitations.
● Look at analytics to clarify user journeys and establish
the relevant KPIs we would be measuring.
● Design a pattern that allows users to jump to checkout
when they are ready from any page, such as a basket
pop-up pattern like Boohoo, a cart sidebar, a checkout
button placed close to the add to cart button etc.
● Biometric Test (qual).
● A/B Test Design (quant).
Issues and Opportunities 32
Basket Delivery Cards
Analysis
In the aside video we can see one of the candidates trying to select a
delivery option. They click on the “Free* STANDARD DELIVERY”
banner and are confused when they’re taken to a page outlining
delivery options. An alternative user clicked on the more
information icon.
This contributed to a 30% slower task completion time for the
Basket page when compared to Zara and the user displays a
moment of confusion and frustration.
Recommendation
● We’d first want to look at the number of users who click
these elements to understand the possible area of effect
of this.
● Look to test a solution to displaying the delivery option
information without having the user navigate away. An
off-screen modal would be a solution for this.
Issues and Opportunities 33
Checkout Error Notifications
Analysis
Users who prefer to checkout as a guest and not create an account
are likely to not fill in the account inputs, without realising it is
mandatory in order to complete their order.
In this scenario, when the user interacts with the “PLACE ORDER”
CTA it triggers the validation for the account inputs. The validation
from the elements cover the label and any input above forcing the
user to work around the issue in order to proceed. This has the
potential to increase cart abandonment.
Recommendation
● Validate assumption through analytics (if relevant
markers are set up), look at user actions after new
account form errors out, establish relevant KPIs to
measure (abandonment rate, task time for the step etc.)
● Consider design solutions to make the notifications less
intrusive (under the field, add click to close ability, move
notifications to the sides etc.) in the context of how the
error notifications are used site-wide or a localized fix.
● Set up and run an A/B test, compare KPIs
Issues and Opportunities 34
Product Size Preselection
Analysis
Users expect the size selection to be automatically filled on the
product details page if they filtered using their size.
Think aloud technique uncovered significant frustration around this
issue with two of the five participants voicing the opinion. In
addition, GSR, Eye Tracking and Facial Recognition indicate
confusion for most participants when fixating on the size selector.
Recommendation
● In our test cases, most users selected a single size, this
would allow the opportunity to prefill the size when we
move to the product page.
● At times when multiple sizes have been selected, we
could opt for selecting the average size. If this isn’t
possible we’d fall back to not preselecting and allow the
validation to push the user into making a choice.
● Understanding the % of options chosen on average for
size would allow us to estimate the possible significance
of this test
Issues and Opportunities 35
Payment Card Input
Analysis
The user inputs their card details but accidentally inputs 17
numbers instead of 16.
As the numbers are grouped together it’s not instantly obvious to
the user that there is an error and they continue and receive the
validation message. They then delete the entire number and start
again.
Recommendation
● Look through analytics at the number of validation errors
on this page to see if we experience a higher than the
average number of errors on this input.
● Running an A/B test in which the credit card input field is
split into blocks of 4 could improve the scannability of
this input, improving the user’s ability to notice errors
and reducing the number of validation errors received.
● The ease of implementation of a feature like this would
see us consider this low hanging fruit.
Issues and Opportunities 36
Analysis
While all participants found the ASOS product listing page filters
easy to notice and use, they seemed to prefer the filtering pattern
used by Boohoo, mostly because it allows them to see which specific
filters are applied.
One participant noted that they can see which filters are being
applied but it’s frustrating not to know the actual value that was
selected. Did not like to have to click on the filter again to find out.
Another participant noted that she loves the ASOS filters, only
wishes they have the filter tags like Boohoo.
Recommendation
● Review analytics to understand how often people are
interacting with the filters, if people are going back to the
filter this could indicate that this could do with
reworking.
● Consider and design mechanisms that allow users to see
which filters are being applied, such as using filter cards
like Boohoo or displaying the selection within the filter.
● Biometric testing the designs and A/B testing the
solution would then be advised.
PLP Filters
Downloads
37
We prepared a handy pack containing all assets and videos created for the purpose of this assessment.
DOWNLOAD PACK
https://goo.gl/g87QZS
REPORT PREPARED BY
SpaceBetween.co.uk
0123 446 323
If you have any questions, please don’t hesitate to get in touch.
Thank you!38
Luke Frake
+44 (0) 1233 800 991
38