baobab health, cognitive walkthrough
TRANSCRIPT
Harry Hochheiser Biomedical Informatics
Harry Hochheiser, [email protected] Baobab Health, February 2015
UI Inspections 2: Cognitive Walkthroughs
Harry Hochheiser, [email protected] Baobab Health, February 2015
Types of Inspections
• Heuristic Evaluations: How well does an interface conform to guidelines for interface design?
• Walkthroughs: Analytic examination of interface and interaction requirements, usually informed by some model of the user
• Many variants...
Harry Hochheiser, [email protected] Baobab Health, February 2015
Heuristic Evaluation Review
• OpenMRS
• Login, registration, etc.
• How well did it work?
Harry Hochheiser, [email protected] Baobab Health, February 2015
Limits of Heuristic Evaluations
• Might miss task orientation
• Workflow?
• Which steps are critical?
• Errors found in non-crucial steps are not necessarily as problematic
Harry Hochheiser, [email protected] Baobab Health, February 2015
Cognitive Walkthrough(Wharton, et al, 1994, Spencer 2000)
• Evaluate software for learning by exploration • Preferred mode of learning for many users
• Conducted with respect to one or more specific tasks
• Consider, in sequence, user actions needed to complete the task
• Tell a story about interactions
• Ask what user would be trying to do and what interface affords
• Successful interfaces will lead user to correct the appropriate action • and provide clear feedback that progress is being made
Harry Hochheiser, [email protected] Baobab Health, February 2015
Scope/Limitations
• Focus on ease of learning might bias results
• Narrowly-focused method
Harry Hochheiser, [email protected] Baobab Health, February 2015
Inputs to the walkthrough• Users?
• Be specific – background, experience, knowledge
• Task(s)
• Reasonable but representative set of benchmark tasks
• Actions
• What is the correct action sequence for each task and how is it described?
• describe at same level as a good tutorial
• Interface Definition
• how is interface defined? Provide detail relevant for presumed user and context –
• don't bother with information that can be assumed.
Harry Hochheiser, [email protected] Baobab Health, February 2015
The walkthrough(Wharton, et al. 1994)
• Examine each action in solution path
• Attempt to tell a credible story as to why the expected users would choose the correct action
• Based on user's background and goals
• Critical features – those that link task description and correct action
Harry Hochheiser, [email protected] Baobab Health, February 2015
Four questions to ask(Wharton, et al. 1994)
For each action:
• Will the users try to achieve the right effect?
• Will the user notice that the correct action is available?
• Will the user associate the correct action with the effect?
• If the correct action is performed, will the user see that progress is being made toward solution of the task?
Harry Hochheiser, [email protected] Baobab Health, February 2015
Using the questions
• If all four questions can be answered yes – success
• Any single “no” – failure story
Harry Hochheiser, [email protected] Baobab Health, February 2015
Streamlined walkthrough questions(Spencer, 2000)
• Will the user know what to do at this step?
• If the user does the right thing, will they know that they've done the right thing, and are making progress toward their goal?
• Note similarity to heuristics
• Visual system status
• Help users recognize, diagnose, and recover from errors
Harry Hochheiser, [email protected] Baobab Health, February 2015
Information to capture(Wharton, et al. 1994)• User knowledge requirements
• Assumptions about user population
• Notes about side issues
• Design changes?
• Three displays
• Key points of group story
• Information about each class of user
• Side issues and design changes
Harry Hochheiser, [email protected] Baobab Health, February 2015
Example: Forwarding phone calls(Wharton, et al. 1994)Action Sequence
• Pick up receiver → dial tone
• Press #2 → bip bip bip
• Hang up Receiver
• Pick up Receiver → dial tone
• Press #2 → dial tone
• Press forwarding extension → bip bip bip
• Hang up receiver
Harry Hochheiser, [email protected] Baobab Health, February 2015
Action 1: Pick up receiver(Wharton, et al. 1994)
• Response from phone: dial tone
• Will the users try to achieve the right effect?
• Will the user notice that the correct action is available?
• Will the user associate the correct action with the effect?
• If the correct action is performed, will the user see that progress is being made toward solution of the task?
Harry Hochheiser, [email protected] Baobab Health, February 2015
Action 2: Press #2(Wharton, et al. 1994)• Command to cancel forwarding: bip bip bip
• Failure Story
• Criteria: will the user be trying to achieve the right thing?
• Problem – the result why would you be trying to cancel forwarding?
• Criteria: Will user know that the action will have the right effect?
• Problem - might not recognize that correct action is “#2”
• Criteria: if the correct action is taken, will the user see that things are going ok?
• Problem -Will they recognize “bip bip bip” at first?
Harry Hochheiser, [email protected] Baobab Health, February 2015
Procedural Issues
• Always stay on track – proceed as if correct action had occurred
• If necessary, assume a “fix” - some additional user knowledge – to complete tasks correctly
Harry Hochheiser, [email protected] Baobab Health, February 2015
Cognitive Analysis of DecisionSupport for Antibiotic Ordering in a Neonatal Intensive Care UnitSheehan, et al. 2013, Applied Clinical Informatics
• CPOE in NICU
• Usability Assessment of cognitive impact
• Use Norman's Theory of Action to guide analysis
• Goal, plan execution response, interpretation/evaluation, new goals
• Gulfs of executions and evaluation
• Characterize gulfs..
Harry Hochheiser, [email protected] Baobab Health, February 2015
Cognitive Analysis….Sheehan, et al. 2013, Applied Clinical Informatics
• Cognitive distance
• “degree and magnitude of mental transformations required to bridge the gulfs of execution and evaluation”
• Semantic Distance
• “relationship between what the user wants to communicate and the meaning of the corresponding expression in the interface language”
• Articulatory Distance
• “the relationship between the meanings of the expression and their physical form.”
• Issue Distance
• “cognitive effort required when a shift in goal is necessary"
• Use these distances to characterize issues
Harry Hochheiser, [email protected] Baobab Health, February 2015
Cognitive Analysis….Sheehan, et al. 2013, Applied Clinical InformaticsFigure repeated from Rizzo, et al. 1997
Harry Hochheiser, [email protected] Baobab Health, February 2015
Cognitive Analysis….Sheehan, et al. 2013, Applied Clinical Informatics
• Two scenarios
• Developed with focus groups
• “Scenario A involved an infant with late-onset sepsis who required treatment with vancomycin and gentamicin. This infant had an elevated serum creatinine of 1.7, which reflects a dramatic decrease in kidney function. Because these antibiotics are primarily eliminated by the kidney, a creatinine level this high would trigger a specific alert that warns the prescriber that standard antibiotic dosing regimens should not be used for this infant. The first task in this scenario was to enter an order for vancomycin. The second was to enter an order for gentamicin.”
• “Scenario B involved an infant with early onset-sepsis. In this scenario, the infant required admission orders including gentamicin and ampicillin. The task was to enter the admission orders using the NICU general admission order set.”
Harry Hochheiser, [email protected] Baobab Health, February 2015
Cognitive Analysis….Sheehan, et al. 2013, Applied Clinical Informatics
• Use Nielsen's heuristics to classify problems..
• Subgoals,etc.
Harry Hochheiser, [email protected] Baobab Health, February 2015
Cognitive Analysis….Sheehan, et al. 2013, Applied Clinical Informatics
• Usability Problems
Harry Hochheiser, [email protected] Baobab Health, February 2015
Cognitive Analysis….Sheehan, et al. 2013, Applied Clinical Informatics
Harry Hochheiser, [email protected] Baobab Health, February 2015
Cognitive Analysis….Sheehan, et al. 2013, Applied Clinical Informatics
Articulatory Distance
Harry Hochheiser, [email protected] Baobab Health, February 2015
Cognitive Analysis….Sheehan, et al. 2013, Applied Clinical Informatics
Semantic Distance
Harry Hochheiser, [email protected] Baobab Health, February 2015
Cognitive Analysis….Sheehan, et al. 2013, Applied Clinical Informatics
• Summary of all problems