usability study of wal-mart website
Post on 29-Nov-2014
70 Views
Preview:
TRANSCRIPT
1
Walmart.com Usability Study
By Robert Bialecki
Usability Design and Testing
May 5, 2011
2
Table of Contents
Executive Summary…………………………………………….....3
Introduction………………………………………………………..4
Target User Analysis…………………………………………....5-6
Participant Demographics…………………………………………6
Methodology………………………………………………………7
Results…………………………………………………………8-13
Recommendations……………………………………………14-16
Appendix……………………………………………………..17-25
3
Executive Summary
I recently conducted a usability test on www.walmart.com using five participants who had no
prior experience with the website.
For more information about the test, see the Introduction section (p. 4 ) and Methodology section
(p. 7 ).
Based on the comments of the test participants, I have compiled a list of recommendations for
enhancing the usability of www.walmart.com. The recommendations are listed in order of
importance, based on the results of the usability test.
1. Decrease the size of the on-screen pop-up that asks if users want a service plan when
they add a product to their shopping cart.
2. Make the “In-Store Events” link easier to find, possibly in the top navigation bar.
3. Turn the “New Customer?” text in the upper right hand corner of the page into a link.
4. Include an “Add to Cart” button on the pictures and brief descriptions in the search
results for products.
5. Reduce the number of images and ads on the homepage.
6. Turn the “Remove” link into a button, like the “Add to Cart” button.
For more information about the recommendations, see the Recommendations section (p. 14-16).
4
Introduction
Wal-Mart’s website (http://www.walmart.com) was the focus of this usability project. Wal-Mart
is one of the largest retailers in the world. The main purposes of its website are to promote and
inform consumers about its products, to allow consumers the option of buying products online,
and to provide a forum for consumers to offer feedback and have their questions answered.
Because of the wide variety of products and services it has to offer, Wal-Mart’s primary
audience is typical American consumers. Their audience comes in both genders and many
different ethnicities and age groups. However, due to their reputation of having “everyday low
prices,” Wal-Mart is particularly popular among middle and lower class families.
The objectives of this usability test were to evaluate the usefulness and level of difficulty of four
common tasks that users of the website might do. Additionally, test participants pointed out
aspects of the website that made the tasks more difficult to complete.
5
Target User Analysis
Persona
Name: Kathy Johnson
Age: 35
Location: La Crosse, WI
Occupation: Stay-at-home Mom
Status: Married, 2 Children
Kathy Johnson is a 35-year-old stay-at-home mom who lives in La Crosse, Wisconsin. Her
husband, Karl, is an accountant at a local business. The couple has twin 10-year-old boys,
Tanner and Taylor, and the family lives in a modest suburban house.
Kathy has been a loyal Wal-Mart customer her whole life. As a young girl, she loved
accompanying her mother on the weekend shopping trip. Today, she recreates those memories
by taking her twin boys with her every weekend. As a stay-at-home mom, Kathy leads a very
busy life. Between maintaining a tidy household, getting the kids ready for school, and preparing
meals for her family, she also often travels to many nearby cities and towns, since her boys are
on the traveling soccer team. One thing Kathy has always loved about Wal-Mart is that she can
find clothes, groceries, toys, and much more all in one store.
Kathy has access to high-speed Internet because her husband must also work at home sometimes,
and the children need a computer for their homework. Kathy would estimate she spends about 6
hours per week using the Internet. In her leisure time, Kathy maintains a blog and keeps in touch
with friends from high school on Facebook and Twitter. Kathy also loves to shop online, where
she often finds better deals than those found in stores. Although she loves to shop both in-stores
and online, Kathy is very frugal, and Wal-Mart’s “everyday low prices” are what has kept her
coming back after all these years.
Key Characteristics: Conclusion:
Is a busy person She would like to find the information
she’s looking for as quickly as possible,
without having to deal with a lot of
clutter.
Shops Online She would be interested in buying Wal-
Mart products online
Often travels to nearby cities and towns She would like to know if there’s a
6
Wal-Mart in her travel destination.
Has access to high-speed Internet She won’t have trouble downloading
large files.
Uses social networking sites She might like to receive Wal-Mart
updates and information on those sites.
Likes to save money She would be interesting in knowing
about any coupons, discounts, deals,
and promotions that Wal-Mart has to
offer.
Demographics of the Participants in this Usability Study
Gender: 2 male, 3 female
Ages: 60, 55, 41, 31, 24
7
Methodology
For this study, I selected five participants who passed the screening process. Participants needed
to fit the following profile:
Be older than 18 years of age
Have more than one year of experience using the Internet
Be familiar with Wal-Mart
Have no negative opinions about Wal-Mart (the company or its practices)
Have never visited Walmart.com
The participants were asked to complete four common tasks that new users of the website might
do. For the purpose of consistency, each participant in the study took the test in the same
makeshift usability lab. The lab consisted of a room with: a desk, a computer with high-speed
Internet access, a comfortable chair, and adequate lighting.
I monitored the usability test for each participant and took notes on their procedures and thoughts
as they completed each task. The tasks were assigned in a different order for each participant so
that the participants’ gradual experience with the website didn’t skew the results of the study.
Due to resource limitations, the test sessions were not recorded on audio or video.
Participants were asked to complete a pre-test questionnaire, post-task questions after each task,
and a post-test questionnaire.
8
Results
Table 1 lists usability problems that participants encountered during the completion of the
usability test. In addition, the table lists the number of participants who experienced the
problems, participant comments in regard to the problems, and the severity level of the problem.
If 4 or 5 of the participants experienced a particular problem, it’s rated high severity. If 3
participants experienced a particular problem, it’s rated medium severity. If 1 or 2 participants
experienced a problem, it’s rated low severity.
Table 1
Usability Problems
Usability Problem
Number of
Participants who
Experienced
Problem
Participant
Comments Severity
A pop-up ad for
adding a service plan
appeared after
participants added an
item to their cart.
Participants clicked
“Close” rather than
“Continue,” which
required scrolling
down to find.
3 out of 3*
“Do I want to add a
service plan? No.”
(closes pop-up)
“Okay… so did it add
the item to my cart or
not?” “Oh, I guess it
must have because the
„My Cart‟ tab now
says 1.”
High
Participants had
trouble finding In-
Store Events
3 out of 5
“The link is too small
and inconspicuous.”
- Participant 2
“I probably would
have found it [the link
on the home page]
sooner if there
weren‟t so many
pictures distracting
me.”
Medium
“New Customer?” on
the home page is not a
link that allows you to
create an account.
3 out of 5
“I assumed clicking
on „New Customer‟
would take me
directly to the
registration. I thought
„Sign in‟ was for
established
customers.”
Medium
9
There is no “Add to
Cart” button on the
pictures and brief
descriptions in the
search results for
products. Participants
had to click on a
particular item in
order to get the “Add
to Cart” button.
3 out of 5 “Where‟s the „Add to
Cart‟ button?” Medium
There are too many
images and ads on the
homepage.
2 out of 5
“I don‟t like how you
have to scroll down
pretty far to find a lot
of the important
information at the
bottom of the page.”
Low
The “Remove” link is
harder to find than the
“Add to Cart” button.
2 out of 5 “That link doesn‟t
stand out very well.” Low
After entering an
address to get driving
directions to a store,
participants saw a
map but had to scroll
down to get the
written directions.
1 out of 5
“Where‟s the written
instructions? … Oh,
there they are.”
Low
* The service plan pop-up appeared only for users who searched for major appliances and
electronics in Task 3.
10
Tables 2 through 5 show the ratings participants gave for the difficulty level of the task, the
amount of time it took to complete the task, and the likelihood that they would use each
task/feature.
Table 2
Task 1: Create a user account at www.walmart.com.
Difficulty Level of Completing This Task Very Easy
Somewhat
Easy
Neither Easy
nor Difficult
Somewhat
Difficult Very Difficult
Number of
participants
who gave this
rating
2 3
The Amount of Time it Took to Complete This Task
A Lot Shorter
Than
Expected
Somewhat
Shorter Than
Expected
About the
Amount of
Time
Expected
Somewhat
Longer Than
Expected
A Lot Longer
Than
Expected
Number of
participants
who gave this
rating
4 1
The Likelihood That They Would Use This Task
Very Likely Somewhat
Likely
Neither
Likely nor
Unlikely
Somewhat
Unlikely
Very Unlikely
Number of
participants
who gave this
rating
1 2 1 1
11
Table 3 Task 2: Get driving directions from your home to the nearest Wal-Mart store.
Difficulty Level of Completing This Task Very Easy
Somewhat
Easy
Neither Easy
nor Difficult
Somewhat
Difficult Very Difficult
Number of
participants
who gave this
rating
1 1 2 1
The Amount of Time it Took to Complete This Task
A Lot Shorter
Than
Expected
Somewhat
Shorter Than
Expected
About the
Amount of
Time
Expected
Somewhat
Longer Than
Expected
A Lot Longer
Than
Expected
Number of
participants
who gave this
rating
1 2 1 1
The Likelihood That They Would Use This Task
Very Likely Somewhat
Likely
Neither
Likely nor
Unlikely
Somewhat
Unlikely
Very Unlikely
Number of
participants
who gave this
rating
2 2 1
12
Table 4 Task 3: Add a product of your choosing to your online shopping cart and then
remove it from your shopping cart
Difficulty Level of Completing This Task Very Easy
Somewhat
Easy
Neither Easy
nor Difficult
Somewhat
Difficult Very Difficult
Number of
participants
who gave this
rating
1 2 2
The Amount of Time it Took to Complete This Task
A Lot Shorter
Than
Expected
Somewhat
Shorter Than
Expected
About the
Amount of
Time
Expected
Somewhat
Longer Than
Expected
A Lot Longer
Than
Expected
Number of
participants
who gave this
rating
1 2 2
The Likelihood That They Would Use This Task
Very Likely Somewhat
Likely
Neither
Likely nor
Unlikely
Somewhat
Unlikely
Very Unlikely
Number of
participants
who gave this
rating
1 1 2 1
13
Table 5 Task 4: Find a list of events taking place at your nearest Wal-Mart within the
next two months.
Difficulty Level of Completing This Task Very Easy
Somewhat
Easy
Neither Easy
nor Difficult
Somewhat
Difficult Very Difficult
Number of
participants
who gave this
rating
1 2 1 1
The Amount of Time it Took to Complete This Task
A Lot Shorter
Than
Expected
Somewhat
Shorter Than
Expected
About the
Amount of
Time
Expected
Somewhat
Longer Than
Expected
A Lot Longer
Than
Expected
Number of
participants
who gave this
rating
2 2 1
The Likelihood That They Would Use This Task
Very Likely Somewhat
Likely
Neither
Likely nor
Unlikely
Somewhat
Unlikely
Very Unlikely
Number of
participants
who gave this
rating
1 2 2
14
Recommendations
Based on the comments of the test participants, I have compiled a list of recommendations for
enhancing the usability of www.walmart.com. The recommendations are listed in order of
importance, based on the results of the usability test.
1. Decrease the size of the on-screen pop-up that asks if users want a service plan when they
add a product to their shopping cart.
Reason: After selecting “Yes” or “No,” users have to scroll down to see the “Continue”
link. The test participants all clicked on the “Close” button in the upper right hand corner
and weren’t sure if the product was added to their cart or not.
2. Make the “In-Store Events” link easier to find, possibly in the top navigation bar.
Reason: Test participants had a difficult time finding the link, as it required them to
scroll half-way down the homepage and sort through less relevant content.
3. Turn the “New Customer?” text in the upper right hand corner of the page into a link.
Reason: New users currently have to click on “Sign In” to create an account, which
implies that the use already has an account. (See Figure 1)
Figure 1
15
4. Include an “Add to Cart” button on the pictures and brief descriptions in the search
results for products.
Reason: Test participants had to click on a particular item in order to be taken to a screen
with an “Add to Cart” button. Some wondered why the button wasn’t already in the
search results. (See Figure 2) This change would be useful for users who already know
exactly what they’re looking for.
Figure 2
5. Reduce the number of images and ads on the homepage.
Reason: The homepage requires quite a bit of scrolling to get from the top to the bottom
of the page. Since there is relevant content throughout the page, this would make that
content easier to find.
16
6. Turn the “Remove” link into a button, like the “Add to Cart” button.
Reason: Some test participants found the “Remove” link hard to find because the text is
small and it’s surrounded by other links. (See Figure 3)
Figure 3
17
Appendix: Usability Protocol
General Script
The screening process:
“Excuse me, could I have a moment of your time?”
If no, “Okay then, sorry to bother you.”
If yes, “Hi, my name is Robert Bialecki. I‟m a student at the University of Wisconsin-Stout, and
I‟m conducting a study for my Usability Testing class. Would you be interested in participating
in my study?”
If no, “Okay then, sorry to bother you.”
If yes, “Okay, thank you. First, I‟d like to ask you a few questions to see if you would qualify for
the study.”
(Ask the questions from the Screener Form and record the responses)
“That‟s all the questions I have for you. Thank you! If you are selected to participate in this
study, what is the best way for me to contact you to schedule a session?”
Before the test:
“Hello, and welcome to the usability study. Again, my name is Robert Bialecki. If you need a
beverage or a restroom, they‟re right over there. As you know, I‟m conducting this study for my
Usability Testing course at UW-Stout. The purpose of my study is to evaluate the Wal-Mart
website. Specifically, I will ask you to complete a series of common tasks on the Wal-Mart
website that new users might use. This usability test will help me evaluate the ease and
usefulness of the tasks. Before we begin, I‟d like to go over this consent form with you. If you
have any questions or concerns regarding the consent form or the study itself, please let me
know right away.”
(Go over the form with the participant, allowing him/her time to read and sign)
“So do you have any questions or concerns?”
“Okay, everything looks good. I would also like you to fill out this questionnaire before we
begin.”
(Allow participant time to complete the questionnaire.)
“Okay, looks good.”
The test:
“So here‟s how the test will work - I will give you four tasks to complete on the Wal-Mart
website. Each task will require you to start at the home page you see currently displayed on the
screen. I will be beside you as you complete each task, but I will not be able to tell you how to
complete the tasks. I will be taking notes on your procedures, thoughts, and my own observations
as you complete each task. There is no time limit on completion of the tasks, but if at any point
you decide a task is too difficult, you may give up, and we will move on to the next task. I would
also like to encourage you to think out-loud as you complete each task. Say out-loud what‟s on
your mind as you complete each task. The thoughts could be about anything related to the task or
the website. For example, you might say „Okay, so next I‟m going to…‟ or „I think it would be
easier if…‟ or „This is frustrating because…‟ I know the process of thinking out-loud may feel
18
weird at first, but your thoughts will be very beneficial to my study. Do you have any questions at
this point? Okay then, here is your first task…”
After the each task:
“You‟re doing a great job. Again, please remember to think out-loud. Do you have any questions
so far? Okay, now before we move on to the next task, I‟d like you to fill out this short
questionnaire about the task you just completed.”
(Allow time for the participant to read and fill out the questionnaire)
“Great! Thank you! Now let‟s move on to the next task. Please return to the home page. Your
next task is…”
After completing the test:
“Well, that‟s the last task. You‟ve done a very good job, both with the tasks and thinking out-
loud. There‟s one last thing I‟d like you to do. Please fill out this questionnaire about the test
overall.”
(Allow time for the participant to read and fill out the questionnaire)
“Okay, that‟s all I have for you. Do you have any last questions? Again, I‟d like to thank you for
your time and participation in this study. Have a nice day!”
19
Screening Form
Candidate Name: ______________________ Qualifies? □ Yes □ No
Screening Questions:
1. Are you older than 18 years of age?
□ Yes (continue) □ No (terminate)
2. Do you have more than one year of experience using the Internet?
□ Yes (continue) □ No (terminate)
3. Are you familiar with the company, Wal-Mart?
□ Yes (continue) □ No (terminate)
4. Do you have any negative opinions about Wal-Mart (the company or its practices)?
□ Yes (terminate) □ No (continue)
5. Have you ever visited Wal-Mart’s website, Walmart.com?
□ Yes (terminate) □ No (continue)
Cell Phone # _______________________
Home Phone # ______________________
E-mail _____________________________
20
Consent to Participate in Wal-Mart Website Usability Study
Title: Walmart.com Usability Study
Moderator: Robert Bialecki
Description:
This is a usability study of Wal-Mart’s website, Walmart.com. The purpose of this study is to
evaluate the effectiveness of the website in order to help better meet the needs of Wal-Mart’s
consumers. You will be asked to complete several tasks that users of the website would
commonly do, as well as answer some questions that pertain to your experience while interacting
with Walmart.com. You will be observed only by the moderator who will be taking notes;
however, you will not be recorded by video or audio.
Time Commitment:
Your involvement in this study will take no longer than one hour.
Risks:
There are no adverse risks involved in the participation of this study. This study is solely for
educational purposes in the completion of a college course project.
Confidentiality:
Your involvement in this study will be held strictly confidential by the moderator. No potentially
identifiable information will be shared with anyone for any reason in any aspect of this study.
Right to Withdraw:
Participation in this study is strictly voluntary. There will be no consequences should you choose
not to participate in this study. Furthermore, you may withdraw from this study at any time and
for any reason, even if you have already agreed to participate.
Moderator
Robert Bialecki
University or Wisconsin-Stout Student
(608) 783-0529
bialeckir@my.uwstout.edu
Statement of Consent:
By signing this form, you acknowledge that you have read and understood the information above
and that you agree to participate in the Walmart.com Usability Study.
_______________________________________________ __________________
Signature Date
21
Pre-test Questionnaire
1. Have you ever visited the website of any retail store before?
□ Yes
□ No
If you answered “Yes,” please answer the following two questions:
Which site(s) did you visit?
Are there any aspects in particular that you liked or disliked about the site(s)?
2. Have you ever purchased anything online from a retail store before?
□ Yes
□ No
If you answered “Yes,” please answer the following two questions:
Which retailer(s) did you purchase from?
Was your experience a positive one? Why or why not?
3. What sorts of information do you feel should be available on a retailer’s website? Please list
what you feel is most important.
4. What sorts of features do you feel should be available on a retailer’s website? Please list at
what you feel is most important.
22
Tasks
1. Create a user account at www.walmart.com
2. Get driving directions from your home to the nearest Wal-
Mart store.
3. Add a product of your choosing to your online shopping cart
and then remove it from your shopping cart.
4. Find a list of events taking place in the next two months at
your nearest Wal-Mart store.
23
Post-task Questionnaire (same for each task)
1. Rate the difficulty level of completing this task:
Very Easy Somewhat Easy Neither Easy
nor Difficult
Somewhat
Difficult Very Difficult
Please explain your rating:
2. Rate the amount of time it took to you to complete this task:
A Lot Shorter
Than I
Expected
Somewhat
Shorter Than I
Expected
About The
Amount of Time
I Expected
Somewhat
Longer Than I
Expected
A Lot Longer
Than I
Expected
Please explain your rating:
3. Rate the likelihood that you would use this feature/task:
Very Likely Somewhat
Likely
Neither Likely
nor Unlikely
Somewhat
Unlikely Very Unlikely
Please explain your rating:
How might this task be made easier?
24
Post-test Questionnaire
1. Overall, rate the average difficulty level of the tasks:
Very Easy Somewhat Easy Neither Easy
nor Difficult
Somewhat
Difficult Very Difficult
Please explain your rating:
2. Rate your overall satisfaction with the Wal-Mart website:
Very Satisfied Somewhat
Satisfied
Neither
Satisfied nor
Dissatisfied
Somewhat
Dissatisfied
Very
Dissatisfied
Please explain your rating:
3. Would you recommend Walmart.com to friends or relatives if they asked? Why or why not?
25
Checklist
Before participant arrives:
□ Clear the desk of any unnecessary clutter or potential distractions
□ Access the Internet and go to the Walmart.com home page
□ Make sure a pen is available for questionnaire completion
□ Have pen and paper ready for note taking
Welcome:
□ Introduce myself and thank participant for participating
□ Show participant where to find beverages and a bathroom
□ Escort participant to the testing room
Before the test:
□ Explain the purpose of the test.
□ Review the Consent Form with the participant, allow time for him/her to read and sign
□ Ask the participant if he/she has any questions or concerns
□ Give the pre-test questionnaire
Instructions:
□ Explain the procedure of the test
□ Ask participant to think out-loud during test; explain what it is and give examples
After each task:
□ Offer reassurance if participant seems to have had difficulty with task
□ Remind participant to think out-loud, if necessary
□ Give post-task questionnaire
□ Ask participant to return to the Walmart.com home page
End of test:
□ Give post-test questionnaire
□ Thank participant for participating
top related