usability testing checklist

Koombea UsabilityTesting Checklist

Koombea Usability TestingChecklist

Koombea Usability Testing Checklist 01

I. Before the Test

1. Decide What You Will Test

a. Consider which areas to test:

i. User Experience • Do pages refresh automatically? Does the website load within 3 seconds? Is the search function easy to find and use?

ii. Accessibility • Can the text be resized? Are forms accessible? Is there a site map?

iii. Homepage • Does the homepage create a positive first impression? Is there a clear call to action? Is the company location and contact information clearly listed?

iv. Navigation • Is navigation consistent throughout each page? Is it well organized, or does it overwhelm users?

v. Search • Is search available on every page? Is the search box wide enough so users can see what they’ve typed?

vi. Links • Are links easily recognizable? Are there any broken links?

vii. Layout • Is important content displayed first? Are pages cluttered? Is the site responsive, with no horizontal scrolling throughout different screen sizes?

viii. Forms • Are forms simple, with only necessary questions asked? Is there a confirmation page after form submission?

ix. Content • Is content scannable? Is it useful and up to date?

b. Create an objective

i. The objective should clearly alert everyone (i.e. the test facilitator, the project sponsor, the stakeholders, etc.) as to what is being tested and why a usability test is being conducted in the first place. Why are you doing this test? What questions do you want answered?

• Example: “Can users easily navigate to the contact form?”; “How easy is it for users to use our application?”; “Do people know what they’re signing up for?”

vi. Links • Are links easily recognizable? Are there any broken links?

vii. Layout • Is important content displayed first? Are pages cluttered? Is the site responsive, with no horizontal scrolling throughout different screen sizes?

viii. Forms • Are forms simple, with only necessary questions asked? Is there a confirmation page after form submission?

ix. Content • Is content scannable? Is it useful and up to date?

b. Create an objective

i. The objective should clearly alert everyone (i.e. the test facilitator, the project sponsor, the stakeholders, etc.) as to what is being tested and why a usability test is being conducted in the first place. Why are you doing this test? What questions do you want answered?

• Example: “Can users easily navigate to the contact form?”; “How easy is it for users to use our application?”; “Do people know what they’re signing up for?”


c. Form test hypotheses

i. Make predictions about users’ behavior based around the test objective. Then, develop your hypotheses about what users will do. Structure your testing to address those hypotheses.

• Example: “Users will have trouble navigating through to the ‘shopping cart’ page”; “Users don’t understand how to use the application.”

d. Arrange scenarios and tasks for the user to accomplish

i. The scenarios should be based on the hypotheses, and should be broken down into small and meaningful tasks. The tasks should be easy to understand and self-explanatory.

1. Task example: “Imagine that you’re at home and you need to use {Name of the product} to {first use case}. Enter {url} and start working on your goal, just as you usually do. Please share your thoughts with us aloud so we can understand your feelings in each particular moment.”

c. Form test hypotheses

i. Make predictions about users’ behavior based around the test objective. Then, develop your hypotheses about what users will do. Structure your testing to address those hypotheses.

• Example: “Users will have trouble navigating through to the ‘shopping cart’ page”; “Users don’t understand how to use the application.”

d. Arrange scenarios and tasks for the user to accomplish

i. The scenarios should be based on the hypotheses, and should be broken down into small and meaningful tasks. The tasks should be easy to understand and self-explanatory.

1. Task example: “Imagine that you’re at home and you need to use {Name of the product} to {first use case}. Enter {url} and start working on your goal, just as you usually do. Please share your thoughts with us aloud so we can understand your feelings in each particular moment.”



2. Recruit Participants

a. Decide on your testing demographic:

i. Gender

ii. Age

iii. Country

iv. Income

v. Web/app expertise

vi. Other: specify additional requirements (i.e. “small business owner”, “Facebook user with 200+ friends”, “college student”, etc.)

b. Advertise for participants

i. Go to a market research firm or temp agency, and ask them to source people of your target profile. Make sure the market research firm doesn’t provide the name of your company or any other details that may affect the judgement of participants. Here’s a list of the top 10 US market research firms to get you started.

ii. Utilize tools like Usertesting.com to get participants.

iii. Post an ad on Craigslist. Craigslist works for in-person and remote usability testing. In addition to the cost of the posting, be prepared to spend between $50 and $200 as an incentive, depending on the commitment and type of user you need.

ii. Utilize tools like Usertesting.com to get participants.

iii. Post an ad on Craigslist. Craigslist works for in-person and remote usability testing. In addition to the cost of the posting, be prepared to spend between $50 and $200 as an incentive, depending on the commitment and type of user you need.

3. Manage the Logistics

a. Book a test room for the test day with internet access and a table/desk with two chairs

b. Screen participants and schedule them into timeslots

c. 2-3 weeks before test: send email to stakeholders, team members, and participants, inviting them to attend

d. 1 week before test: Send email to participants with all relevant information (1 week before test)

i. Directions, parking instructions, location of test room, name and phone number of a contact in case they’re late or get lost, and a non-disclosure agreement if necessary.



II. Test Day

1. Before Each Session

a. Make participants comfortable in testing environment

i. Know the names of each participant ii. Greet participants with a warm welcome, and explain the legal forms that must be signed.

iii. Provide copies of handouts to each participant 1. Recording consent form 2. Task and scenario forms 3. Copy of the test script

iv. If you are providing incentives, make sure they are ready

b. Make sure the recording equipment works, as well as the internet connection and any other technology needed for the tests

i. Check recording software ii. Check if your ethernet cable is connected and wifi is set up as a back up option

iii. Check mouse and keyboard

iv. Connect your laptop to a power source (if applicable)

v. Turn off any unnecessary software

vi. Open the web browser (i.e. Google Chrome, Firefox, Safari, etc.) to a neutral page

iv. Connect your laptop to a power source (if applicable)

v. Turn off any unnecessary software

vi. Open the web browser (i.e. Google Chrome, Firefox, Safari, etc.) to a neutral page


2. During Each Session

a. Explain the reason for the testing session

b. Explain the “Think out loud” protocol

i. With this protocol, you ask participants to complete tasks while continuously ‘thinking out loud’—basically, keeping a running monologue and talking through their thought process. Tell them to point out what they like, don’t like, and where they see possible improvements. Reassure participants that you’re not testing them, you’re testing the application or website.

c. Start the screen recorder (Screenflow is a great tool for this)

d. Present tasks one at a time

i. Make sure to only give participants one task at a time. Throwing several tasks at them maybe intimidate them and alter their approach to the test, comprising results. Once you present the task, allow users to accomplish the task in their own way, without intervening.

e. Once all tasks are completed ask participants about their questions and hand them a post-test questionnaire

i. Ask for overall impressions of the app or site, so you can judge if expectations have been met.

ii. Ask for suggestions. This shows that you value their thoughts and opinions, and it gives you additional insights about how you can improve the user experience.

d. Present tasks one at a time

i. Make sure to only give participants one task at a time. Throwing several tasks at them maybe intimidate them and alter their approach to the test, comprising results. Once you present the task, allow users to accomplish the task in their own way, without intervening.

e. Once all tasks are completed ask participants about their questions and hand them a post-test questionnaire

i. Ask for overall impressions of the app or site, so you can judge if expectations have been met.

ii. Ask for suggestions. This shows that you value their thoughts and opinions, and it gives you additional insights about how you can improve the user experience.


3. After Each Session

a. Stop the screen recorder

b. Save the recording

c. Make sure all documents are signed

d. Create a backup copy of the recording

e. Take notes and analyze the recording as soon as possible

usability testing checklist

Data & Analytics