baobab health 2015 usability inspections
TRANSCRIPT
Usability Inspections: Heuristic Evaluation
Harry Hochheiser Biomedical Informatics
Harry Hochheiser, [email protected] Baobab Health, February 2015
Harry Hochheiser, [email protected] Baobab Health, February 2015
Goal of Usability Assessments
• Can users complete tasks?
• Appropriateness of mental models
• Comparative efficiency
• Subjective satisfaction
• How do we assess?
Harry Hochheiser, [email protected] Baobab Health, February 2015
No bright lines
DesignFunctional
system Paper Prototype Release
Usability Inspections
Usability Studies
Empirical User Studies, Case Studies, Longitudinal Studies, Acceptance Tests
Low cost, low validity Higher cost, validity
Harry Hochheiser, [email protected] Baobab Health, February 2015
Usability Inspections
• “clean-room” static examination of usability • Methodically scrutinize interfaces in search of potential
problems
• Pros: – Inexpensive – no users, relatively easy – Identify major issues at a relatively early stage
• Cons: – May miss problems: generally find < 50% – All results are hypothetical – don’t know which problems might
really lead to errors
Harry Hochheiser, [email protected] Baobab Health, February 2015
Common Questions across inspection types
● Who inspects? ●Background ●How many?
● Which tasks? ● How to interpret findings?
●Problems are not “real” problems experience by users ●How important are they?
Harry Hochheiser, [email protected] Baobab Health, February 2015
Broad classes of inspections
Heuristic Evaluations: How well does an interface conform to guidelines for interface design?
Walkthrough: Analytic examination of interface and interaction requirements, usually informed by some model of the user
Many variants...
Harry Hochheiser, [email protected] Baobab Health, February 2015
Who inspects? Heuristic inspection
Usability experts Domain experts Combination? (Double experts) Users should participate as users when possible
3-5 experts? (Nielsen)
Or more...
Work alone, or in teams..
Walkthroughs May require more cognitive background
Domain expert feedback helpful
Conducted by a team?
Harry Hochheiser, [email protected] Baobab Health, February 2015
Tasks
Heuristic inspections Set tasks Open-ended exploration
Walkthroughs Generally, specific tasks
Harry Hochheiser, [email protected] Baobab Health, February 2015
Nielsen's Heuristics Nielsen, 1994 - http://www.nngroup.com/articles/ten-usability-heuristics/
• Visibility of system status • Match between system and real world • User control and freedom • Consistency and standards • Error prevention • Recognition rather than recall • Flexibility and efficiency of use • Aesthetic and minimalist design • Help users recognize, diagnose, and recover from errors • Help and documentation
Harry Hochheiser, [email protected] Baobab Health, February 2015
Results Nielsen and Molich 1990 , Nielsen 1993
• Few false positives • Find < 50% of errors • “In general, we would expect aggregates of five
evaluators to find about two thirds of the usability problems which is really quite good for an informal and inexpensive technique like heuristic evaluation.”
Harry Hochheiser, [email protected] Baobab Health, February 2015
Results Nielsen and Molich 1990
Source of the “you only need 5 evaluators” rule… Hotly contested…
Harry Hochheiser, [email protected] Baobab Health, February 2015
Which Evaluators? Nielsen, 1992
• “Double Experts” - Domain and Usability expertise – considered best
• Not always available – use some of each, or teams
Harry Hochheiser, [email protected] Baobab Health, February 2015
Heuristic Evaluation ProcedureEvaluators work alone
(except for when they work in teams)
Optional observer can help explain confusing issues and to record issues.
Go through interface several times List specific tasks
Note discrepancies between interface and heuristic Note any concerns - even those that don’t exactly fit heuristics
Individual evaluators meet to aggregate results agree on interpretation and prioritization
Harry Hochheiser, [email protected] Baobab Health, February 2015
Nielsen's Heuristics Nielsen, 1994 - http://www.nngroup.com/articles/ten-usability-heuristics/
• Visibility of system status •“The system should always keep users informed about what is going on, through appropriate feedback within reasonable time”
• Match between system and real world “The system should speak the users’ language, with words, phrases, and
concepts familiar to the user, rather than system-oriented terms. Follow real-
world conventions, making information appear in a natural and logical order.”
• User control and freedom
“Users often choose system functions by mistake and will need a clearly marked
"emergency exit" to leave the unwanted state without having to go through an
extended dialogue. Support undo and redo.”
Harry Hochheiser, [email protected] Baobab Health, February 2015
Nielsen's Heuristics Nielsen, 1994 - http://www.nngroup.com/articles/ten-usability-heuristics/
• Consistency and standards “Users should not have to wonder whether different words, situations,
or actions mean the same thing. Follow platform conventions.”
• Error prevention “Even better than good error messages is a careful design which prevents a
problem from occurring in the first place. Either eliminate error-prone
conditions or check for them and present users with a confirmation option
before they commit to the action.”
• Recognition rather than recall “Minimize the user's memory load by making objects, actions, and options visible. The user should not have to remember information from one part of the dialogue to another. Instructions for use of the system should be visible or easily retrievable whenever appropriate.”
Harry Hochheiser, [email protected] Baobab Health, February 2015
Nielsen's Heuristics Nielsen, 1994 - http://www.nngroup.com/articles/ten-usability-heuristics/
• Flexibility and efficiency of use “Accelerators -- unseen by the novice user -- may often speed up the interaction for the expert user such that the system can cater to both inexperienced and experienced users. Allow users to tailor frequent actions.”
• Aesthetic and minimalist design • “Dialogues should not contain information which is irrelevant or rarely needed. Every extra unit of information in a dialogue competes with the relevant units of information and diminishes their relative visibility.”
Harry Hochheiser, [email protected] Baobab Health, February 2015
Nielsen's Heuristics Nielsen, 1994 - http://www.nngroup.com/articles/ten-usability-heuristics/
• Help users recognize, diagnose, and recover from errors “Error messages should be expressed in plain language (no codes), precisely indicate the problem, and constructively suggest a solution.”
• Help and documentation “Even though it is better if the system can be used without documentation, it may be necessary to provide help and documentation. Any such information should be easy to search, focused on the user's task, list concrete steps to be carried out, and not be too large.”
Harry Hochheiser, [email protected] Baobab Health, February 2015
How to Interpret?
Use severity judgments to prioritize fixes Frequency of problem Impact of problem Persistence – will users be repeatedly bothered? Multiple independent raters increase reliability
Bigger questions – does this design work at all? As with usability studies, try to generalize
Don't solve lots of small problems if the design is inherently problematic
Harry Hochheiser, [email protected] Baobab Health, February 2015
Severity Ratings
0. No problem 1. Cosmetic 2. Minor 3. Major 4. Catastrophe
Measure both impact and frequency?
Harry Hochheiser, [email protected] Baobab Health, February 2015
Reporting heuristic violations http://ocw.mit.edu/courses/electrical-engineering-and-computer-science/ 6-831-user-interface-design-and-implementation-spring-2011/lecture-notes/MIT6_831S11_lec23.pdf
• Goal - communicate problems • Identify successes • Be constructive • Be specific
Harry Hochheiser, [email protected] Baobab Health, February 2015
Heuristic Evaluation reports http://ocw.mit.edu/courses/electrical-engineering-and-computer-science/ 6-831-user-interface-design-and-implementation-spring-2011/lecture-notes/MIT6_831S11_lec23.pdf
• Problem • Heuristic • Description • Severity • Recommendations • Screenshot
Harry Hochheiser, [email protected] Baobab Health, February 2015
Example: NIH Undiagnosed Disease Program UDPICs LIMS tool
Harry Hochheiser, [email protected] Baobab Health, February 2015
Example: NIH Undiagnosed Disease Program UDPICs LIMS tool
Heuristic Visibility of system status; Consistency and standards; Recognition rather than recall
Description The “gear” icon is often used to indicate a menu of systems settings and preference. Although this is not a standard, it is a widely-used convention. UDPICS uses this gear icon to access a menu that contains a variety of options - some of which are settings and others providing additional system functionality Figure 2). This may confuse users who select the menu looking only for settings selections, or for those who look elsewhere for options found in this menu. The use of the gear item also raises concerns for administrative users, who might be confused by the gear icon on the one hand and the “settings and preferences” menu on the left-hand navigation bar.
Severity Minor
Suggested Revisions
One possible solution would be to remove all non-settings items from this menu, adding additional choices to the otherwise underutilized menu bar. Even if the left-hand navigation bar is available only to administrators, placing all settings menu in a single place and adjusting the available options based on user privileges might be easier for users.
Harry Hochheiser, [email protected] Baobab Health, February 2015
Making use of heuristic evaluation results?
● Have member of development team present during inspection?
Can be done with usability studies and other inspections as well They should observe, not analyze
● Debriefing session to explore fixes Consider redesigns where appropriate.
Harry Hochheiser, [email protected] Baobab Health, February 2015
Other Inspection Methods http://www.useit.com/papers/heuristic/inspection_summary.html
Heuristic estimation – compare interfaces
Feature Inspection – look for long sequences of operations, unnatural sequences, those requiring extensive skill...
Consistency inspection – compare designs for consistency
Standards Inspection
Harry Hochheiser, [email protected] Baobab Health, February 2015
Which Heuristics?
• Multiple versions of Nielsen’s list • Other versions
– Special cases?
• More heuristics – more focus – ..but less study
• Too many heuristics – cumbersome, complicated
Harry Hochheiser, [email protected] Baobab Health, February 2015
Web Usability Heuristicshttp://www.andybudd.com/archives/2007/01/heuristics_for_modern_web_application_development/index.htm
• Design for user expectations • Clarity • Minimize Unnecessary Complexity and Cognitive Load • Efficiency and Task Completion • Provide Users with Context • Consistency and Standards • Prevent Errors • Help users, notice, understand, and recover from errors • Promote a pleasurable and positive web experience
Harry Hochheiser, [email protected] Baobab Health, February 2015
Usability Evaluation of an Electronic Medication Administration Record (eMAR) Application Guo, et al. 2011
• eMAR tasks – Access to drug information, policies, and procedures related to
meds – Context-specific patient clinical results – Facilitate communication between nurses and pharmacists – Verification of the five “rights” (patient, drug, dose, route, and
time)
• Heuristic Evaluation for usability • Using Zhang’s heuristics and severity ratings
Harry Hochheiser, [email protected] Baobab Health, February 2015
Heuristic evaluation of eMAR Guo, et al. 2011
• Tasks – Login – Order and modify medication – Verify medication orders – Access drug references – Administer medication – Edit eMAR – Generate reports and review
• Procedure – Train evaluators – Conduct evaluation – Calculate agreement – Build consensus – Review with staff – conduct tasks
Harry Hochheiser, [email protected] Baobab Health, February 2015
Heuristic evaluation of eMAR Guo, et al. 2011
• Observations – classified by heuristic, – grouped into usability problems – Organized by task
Harry Hochheiser, [email protected] Baobab Health, February 2015
Heuristic evaluation of eMAR Guo, et al. 2011
Harry Hochheiser, [email protected] Baobab Health, February 2015
Heuristic evaluation of eMAR Guo, et al. 2011
Harry Hochheiser, [email protected] Baobab Health, February 2015
Agreement?
• What if multiple reviewers disagree? • Can calculate inter-rater reliability
• Cohen’s kappa • Raw agreement
• Goal here is to find potential problems - • being thorough is more important than agreeing