usability testing kathryn summers ©2010
Post on 18-Sep-2014
809 views
DESCRIPTION
TRANSCRIPT
![Page 1: Usability Testing Kathryn Summers ©2010](https://reader035.vdocuments.mx/reader035/viewer/2022081602/541b03bd7bef0a107d8b6c32/html5/thumbnails/1.jpg)
Usability TestingUsability Testing
Kathryn Summers©2010
Copyright 2003 Kathryn Summers
![Page 2: Usability Testing Kathryn Summers ©2010](https://reader035.vdocuments.mx/reader035/viewer/2022081602/541b03bd7bef0a107d8b6c32/html5/thumbnails/2.jpg)
Usability TestingPreparation
RecruitingScreeningTasks and Scenarios
TestingFacilitationObservation
AnalysisPresentation
![Page 3: Usability Testing Kathryn Summers ©2010](https://reader035.vdocuments.mx/reader035/viewer/2022081602/541b03bd7bef0a107d8b6c32/html5/thumbnails/3.jpg)
Benefits to User-Centered Products (2/09)
Users find them more useful (can do more tasks that the user cares about)
Users can do things more efficientlyUsers learn how to use them more easily
and use more oftenUsers like them more (less frustrating,
more enjoyable
![Page 4: Usability Testing Kathryn Summers ©2010](https://reader035.vdocuments.mx/reader035/viewer/2022081602/541b03bd7bef0a107d8b6c32/html5/thumbnails/4.jpg)
Costs of Poor Design
Copyright 2003 Kathryn Summers
Costs to clientincreased support costsreduced salesexpensive redesign
Costs to customer/userreduced employee productivityincreased employee frustrationmore frequent mistakes
![Page 5: Usability Testing Kathryn Summers ©2010](https://reader035.vdocuments.mx/reader035/viewer/2022081602/541b03bd7bef0a107d8b6c32/html5/thumbnails/5.jpg)
Some general usability principles
Copyright 2003 Kathryn Summers
Simple and natural language (user’s language and labels)
visual consistency (graphics, navigation, layout, labeling)
appropriate feedbackerror prevention, error recoveryhelp information
![Page 6: Usability Testing Kathryn Summers ©2010](https://reader035.vdocuments.mx/reader035/viewer/2022081602/541b03bd7bef0a107d8b6c32/html5/thumbnails/6.jpg)
And information design principles
Copyright 2003 Kathryn Summers
Information available right when and where it’s needed
Organization and presentation of information reflects user’s needs and goals
Visual design reinforces task flows and logical information groupings
![Page 7: Usability Testing Kathryn Summers ©2010](https://reader035.vdocuments.mx/reader035/viewer/2022081602/541b03bd7bef0a107d8b6c32/html5/thumbnails/7.jpg)
Types of tests: Evaluative Testing
Copyright 2003 Kathryn Summers
Confirm that you’ve met a benchmark for usability before release
Confirm that product is more usable than a prior version or a competing product
Evaluative testing will focus on tasks and may often involve quantitative measurements
![Page 8: Usability Testing Kathryn Summers ©2010](https://reader035.vdocuments.mx/reader035/viewer/2022081602/541b03bd7bef0a107d8b6c32/html5/thumbnails/8.jpg)
Types of tests: Exploratory testsTesting prototypes, or comparing prototypes
Copyright 2003 Kathryn Summers
Focus on exploring relationship between system image and user’s mental modelRepresenting classes of objectsRepresenting relationships between objectsAllowing user to manipulate objects
Test navigation, help access, subject matter organization
![Page 9: Usability Testing Kathryn Summers ©2010](https://reader035.vdocuments.mx/reader035/viewer/2022081602/541b03bd7bef0a107d8b6c32/html5/thumbnails/9.jpg)
Values of UsabilityReal world/real user perspectiveTends to find serious functional issues
![Page 10: Usability Testing Kathryn Summers ©2010](https://reader035.vdocuments.mx/reader035/viewer/2022081602/541b03bd7bef0a107d8b6c32/html5/thumbnails/10.jpg)
Values of Eye-trackingAllows testers to see if users read or not –
amount of time spent in area of screen vs scanning
Has value in decisions on layout/ navigation/labeling
![Page 11: Usability Testing Kathryn Summers ©2010](https://reader035.vdocuments.mx/reader035/viewer/2022081602/541b03bd7bef0a107d8b6c32/html5/thumbnails/11.jpg)
Effective user testing
Copyright 2003 Kathryn Summers
Testing the right peopleTesting the right tasks
User testing vs. market research vs. “testing the software”
Watching with attention
![Page 12: Usability Testing Kathryn Summers ©2010](https://reader035.vdocuments.mx/reader035/viewer/2022081602/541b03bd7bef0a107d8b6c32/html5/thumbnails/12.jpg)
Copyright 2003 Kathryn Summers
What users expect from a designHow they respond
Most understandable ways to present content: How the website should be structuredExact function,
arrangement of function, and system
Does the website actually work for the people who are supposedly going to use it? Why or why not?
Determine who is coming online in your targetDemographicPsychographicAttitudes and preferences
Determine what information will provide value to your targetDistinct information needs
and preferences by target sub-segment
Information gaps and priorities
Tools and resources
Market ResearchUser Research
![Page 13: Usability Testing Kathryn Summers ©2010](https://reader035.vdocuments.mx/reader035/viewer/2022081602/541b03bd7bef0a107d8b6c32/html5/thumbnails/13.jpg)
Preparation: Recruiting Participants
Identify your target user groups (demographics, background knowledge, experience, relationship to site/tool, goals)
Decide which groups you want, and how many users you’ll recruit
(decide what characteristics users will share, what characteristics will vary)
Draft a test participant screener (what are make/break characteristics)
Test the screener
![Page 14: Usability Testing Kathryn Summers ©2010](https://reader035.vdocuments.mx/reader035/viewer/2022081602/541b03bd7bef0a107d8b6c32/html5/thumbnails/14.jpg)
Recruiting Participants Decide on number of participants Over recruit (dropout/no show rate) Determine compensation
(amount, cash vs. gift, depends on time spent, organization type, project type)
Acquire consent
![Page 15: Usability Testing Kathryn Summers ©2010](https://reader035.vdocuments.mx/reader035/viewer/2022081602/541b03bd7bef0a107d8b6c32/html5/thumbnails/15.jpg)
Preparation: Developing the Test Identify test objectives/create list of problems Translate objective/problem list into tasks Prioritize tasks (frequency, importance), decide
which tasks to test, Develop scenarios for tasks Write test script (scenarios) Identify resources participants will need for the
tasksHardware, software, data files, instructions, internet connection, time
Determine what data/measurements to collect (metrics)
Conduct a pilot test
![Page 16: Usability Testing Kathryn Summers ©2010](https://reader035.vdocuments.mx/reader035/viewer/2022081602/541b03bd7bef0a107d8b6c32/html5/thumbnails/16.jpg)
Tasks vs. Scenarios Tasks are series of actions made to
achieve goal – ie., “request article from Interlibrary Loan using online form.”
Scenarios are tasks put into a short narrative – ie., “your instructor recommends an article and gives you a citation for “Usability Tests made Simple,” from The Journal of Usability, 2 (34), 11-23. You realize that Cook Library doesn’t have that journal, and you would like to request it from Interlibrary Loan”.
They are meant to take some of the artificiality out of the task.
![Page 17: Usability Testing Kathryn Summers ©2010](https://reader035.vdocuments.mx/reader035/viewer/2022081602/541b03bd7bef0a107d8b6c32/html5/thumbnails/17.jpg)
Tasks vs. ScenariosScenarios should: Allow user to SHOW not TELL actions Include one and only one task Have defined set of pathways to complete
task Use natural language (user’s, not
system’s) Be short – avoid excessive background,
irrelevant details Be clear and provide enough instruction
detail to complete the task (test scenarios with pilot users)
![Page 18: Usability Testing Kathryn Summers ©2010](https://reader035.vdocuments.mx/reader035/viewer/2022081602/541b03bd7bef0a107d8b6c32/html5/thumbnails/18.jpg)
Tasks vs. ScenariosScenarios should NOT: Walk user through process
(e.g. You have the afore-mentioned article citation. Go to the Interlibrary Loan link located in the list of services on the left. Complete the online form including author, article title, journal title, volume, issue and page number)
Scenario should test user’s ability to complete the task in a natural way without step-by-step instructions.
![Page 19: Usability Testing Kathryn Summers ©2010](https://reader035.vdocuments.mx/reader035/viewer/2022081602/541b03bd7bef0a107d8b6c32/html5/thumbnails/19.jpg)
Preparation: Writing scriptWhy a script?
Ensures consistencyScript should include:
Explanation of testConsent (if not done beforehand)Think aloud protocolTasks (description if written, scenarios if
verbal)Questionnaire/demographic questionsFollow up if appropriate
![Page 20: Usability Testing Kathryn Summers ©2010](https://reader035.vdocuments.mx/reader035/viewer/2022081602/541b03bd7bef0a107d8b6c32/html5/thumbnails/20.jpg)
Preparation: Establishing rolesFacilitator
ScriptObserver(s)
Actions Metrics CommentInferences
![Page 21: Usability Testing Kathryn Summers ©2010](https://reader035.vdocuments.mx/reader035/viewer/2022081602/541b03bd7bef0a107d8b6c32/html5/thumbnails/21.jpg)
Observers can capture metrics in metrics-based testing
Copyright 2003 Kathryn Summers
time on taskerror ratesuccessful completion ratenumber of times (or how long) help system
is accessed time spent recovering from errorsnumber of commands/features used
![Page 22: Usability Testing Kathryn Summers ©2010](https://reader035.vdocuments.mx/reader035/viewer/2022081602/541b03bd7bef0a107d8b6c32/html5/thumbnails/22.jpg)
Preparation: Pilot testTest script to see if scenarios/tasks are
workingTest equipmentPractice roles for moderator and observers
![Page 23: Usability Testing Kathryn Summers ©2010](https://reader035.vdocuments.mx/reader035/viewer/2022081602/541b03bd7bef0a107d8b6c32/html5/thumbnails/23.jpg)
Preparation: MaterialsParticipant ListConsent formsCompensation/acknowledgementsScriptsTask scenarios (written/part of script)Observation forms
![Page 24: Usability Testing Kathryn Summers ©2010](https://reader035.vdocuments.mx/reader035/viewer/2022081602/541b03bd7bef0a107d8b6c32/html5/thumbnails/24.jpg)
Facilitating the test Explain the test Help the test participant feel comfortable
(emphasis on testing SYSTEM not USER) Model/Encourage a think-aloud protocol Listen and observe – avoid “rescuing” Probe (avoid using leading questions) Respect user’s expertise
![Page 25: Usability Testing Kathryn Summers ©2010](https://reader035.vdocuments.mx/reader035/viewer/2022081602/541b03bd7bef0a107d8b6c32/html5/thumbnails/25.jpg)
Becoming a better moderator
Copyright 2003 Kathryn Summers
Become a “people” personBe calm and flexiblePay close attentionWatch details but within larger context
Cohesive picture of each testPatterns between tests
Don’t just rely on memoryCommunicate results effectivelyStay organizedPractice!!!
![Page 26: Usability Testing Kathryn Summers ©2010](https://reader035.vdocuments.mx/reader035/viewer/2022081602/541b03bd7bef0a107d8b6c32/html5/thumbnails/26.jpg)
Watching with attention
Copyright 2003 Kathryn Summers
Try not to test your own forms/designsTry to watch with a completely open mindWatch carefully—pay attention to body
language, facial expressions, sounds, where the user is looking, and what they do
Pay most attention to behavior, then to explanations, least attention to preferences or predictions about what other users want
![Page 27: Usability Testing Kathryn Summers ©2010](https://reader035.vdocuments.mx/reader035/viewer/2022081602/541b03bd7bef0a107d8b6c32/html5/thumbnails/27.jpg)
Most of all…
Copyright 2003 Kathryn Summers
Understand that the user never makes “mistakes”—the user only makes efforts to solve the problem
Your goal is to understand what problem-solving effort the user is making
![Page 28: Usability Testing Kathryn Summers ©2010](https://reader035.vdocuments.mx/reader035/viewer/2022081602/541b03bd7bef0a107d8b6c32/html5/thumbnails/28.jpg)
Analysis during the test: Keep a problem list
Copyright 2003 Kathryn Summers
Keep a free-hand list during the test—note “critical incidents” that you may ask follow-up questions about at the end of the session
Use the “highlighter” methodHave a printout of interface screens on hand; circle problem areas as you observe them during the tests
Watch the tapes, categorizing related problems after the test
Capture metrics if desired
![Page 29: Usability Testing Kathryn Summers ©2010](https://reader035.vdocuments.mx/reader035/viewer/2022081602/541b03bd7bef0a107d8b6c32/html5/thumbnails/29.jpg)
ReferencesGoldberg, J. H., Stimson, M. J., Lewenstein, M., Scott, N., Wichansky, A. M. (2002).
Eye tracking in web search tasks: Design implications. In Proceedings of the 2002 symposium on Eye tracking research & applications (pp. 51-58). Retrieved March 28, 2009 from ACM Digital Library.
Hackos, J. T. & Redish, J. C. (1998). User and task analysis for interface design. New York: John Wiley & Sons.
Jeffries, R., Miller, J. R., Wharton, C., & Uyeda, K. M. (1991). User interface evaluation in the real world: Comparison of four techniques. In Proceedings of the SIGCHI conference on human factors in computing systems: Reaching through technology (pp. 119-124). Retrieved March 28, 2009 from ACM Digital Library.
Molich, R., Ede, M. R., Kaasgaard, K., Karyukin, B., (2004). Comparative usability evaluation. Behavior & Information technology, 23 (1), 65. 74. Retrieved March 28 from Academic Search Premier database.
Rubin, J. (1994). Handbook of usability testing: How to plan, design, and conduct effective tests. New York: John Wiley & Sons.
Snyder, C. (2003). Paper prototyping: The fast and easy way to design and refine user interfaces. San Francisco: Morgan Kaufman.
![Page 30: Usability Testing Kathryn Summers ©2010](https://reader035.vdocuments.mx/reader035/viewer/2022081602/541b03bd7bef0a107d8b6c32/html5/thumbnails/30.jpg)
Limitations of usability testing
Copyright 2003 Kathryn Summers
Testing situations are always artificial (maybe in a lab, assigned tasks, paid participation)
Test participants are rarely perfectly representative users
The test design can never fully duplicate natural user behaviorResults depend heavily on tasks—we see
what we think to look for
![Page 31: Usability Testing Kathryn Summers ©2010](https://reader035.vdocuments.mx/reader035/viewer/2022081602/541b03bd7bef0a107d8b6c32/html5/thumbnails/31.jpg)
Finding participants with lower literacy skills
Copyright 2003 Kathryn Summers
Try to recruit participants that Have a strong motivation to care about the
interface (e.g., users with a particular health condition)
Read at 8th grade level or below (REALM 60 or below)
Have been observationally screened for a minimum level of computer ability Open browserNavigate to website using address bar OR
searchScrollUse linksUse mouse
![Page 32: Usability Testing Kathryn Summers ©2010](https://reader035.vdocuments.mx/reader035/viewer/2022081602/541b03bd7bef0a107d8b6c32/html5/thumbnails/32.jpg)
Where to recruit users with lower literacy skills
Copyright 2003 Kathryn Summers
Recruit fromHealth clinicsLiteracy centersPublic libraries
Plan for a two-step recruiting approachAdminister the REALM; observe computer useSchedule selected users for testing
REALM (Rapid Estimate of Adult Literacy in Medicine) (Davis, et al., Family Medicine 1993 25.6: 391-5.)