baobab health, cognitive walkthrough

26
Harry Hochheiser Biomedical Informatics [email protected] Harry Hochheiser, [email protected] Baobab Health, February 2015 UI Inspections 2: Cognitive Walkthroughs

Upload: harry-hochheiser

Post on 19-Jul-2015

90 views

Category:

Education


0 download

TRANSCRIPT

Harry Hochheiser Biomedical Informatics

[email protected]

Harry Hochheiser, [email protected] Baobab Health, February 2015

UI Inspections 2: Cognitive Walkthroughs

Harry Hochheiser, [email protected] Baobab Health, February 2015

Types of Inspections

• Heuristic Evaluations: How well does an interface conform to guidelines for interface design?

• Walkthroughs: Analytic examination of interface and interaction requirements, usually informed by some model of the user

• Many variants...

Harry Hochheiser, [email protected] Baobab Health, February 2015

Heuristic Evaluation Review

• OpenMRS

• Login, registration, etc.

• How well did it work?

Harry Hochheiser, [email protected] Baobab Health, February 2015

Limits of Heuristic Evaluations

• Might miss task orientation

• Workflow?

• Which steps are critical?

• Errors found in non-crucial steps are not necessarily as problematic

Harry Hochheiser, [email protected] Baobab Health, February 2015

Cognitive Walkthrough(Wharton, et al, 1994, Spencer 2000)

• Evaluate software for learning by exploration • Preferred mode of learning for many users

• Conducted with respect to one or more specific tasks

• Consider, in sequence, user actions needed to complete the task

• Tell a story about interactions

• Ask what user would be trying to do and what interface affords

• Successful interfaces will lead user to correct the appropriate action • and provide clear feedback that progress is being made

Harry Hochheiser, [email protected] Baobab Health, February 2015

Scope/Limitations

• Focus on ease of learning might bias results

• Narrowly-focused method

Harry Hochheiser, [email protected] Baobab Health, February 2015

Inputs to the walkthrough• Users?

• Be specific – background, experience, knowledge

• Task(s)

• Reasonable but representative set of benchmark tasks

• Actions

• What is the correct action sequence for each task and how is it described?

• describe at same level as a good tutorial

• Interface Definition

• how is interface defined? Provide detail relevant for presumed user and context –

• don't bother with information that can be assumed.

Harry Hochheiser, [email protected] Baobab Health, February 2015

The walkthrough(Wharton, et al. 1994)

• Examine each action in solution path

• Attempt to tell a credible story as to why the expected users would choose the correct action

• Based on user's background and goals

• Critical features – those that link task description and correct action

Harry Hochheiser, [email protected] Baobab Health, February 2015

Four questions to ask(Wharton, et al. 1994)

For each action:

• Will the users try to achieve the right effect?

• Will the user notice that the correct action is available?

• Will the user associate the correct action with the effect?

• If the correct action is performed, will the user see that progress is being made toward solution of the task?

Harry Hochheiser, [email protected] Baobab Health, February 2015

Using the questions

• If all four questions can be answered yes – success

• Any single “no” – failure story

Harry Hochheiser, [email protected] Baobab Health, February 2015

Streamlined walkthrough questions(Spencer, 2000)

• Will the user know what to do at this step?

• If the user does the right thing, will they know that they've done the right thing, and are making progress toward their goal?

• Note similarity to heuristics

• Visual system status

• Help users recognize, diagnose, and recover from errors

Harry Hochheiser, [email protected] Baobab Health, February 2015

Information to capture(Wharton, et al. 1994)• User knowledge requirements

• Assumptions about user population

• Notes about side issues

• Design changes?

• Three displays

• Key points of group story

• Information about each class of user

• Side issues and design changes

Harry Hochheiser, [email protected] Baobab Health, February 2015

Example: Forwarding phone calls(Wharton, et al. 1994)Action Sequence

• Pick up receiver → dial tone

• Press #2 → bip bip bip

• Hang up Receiver

• Pick up Receiver → dial tone

• Press #2 → dial tone

• Press forwarding extension → bip bip bip

• Hang up receiver

Harry Hochheiser, [email protected] Baobab Health, February 2015

Action 1: Pick up receiver(Wharton, et al. 1994)

• Response from phone: dial tone

• Will the users try to achieve the right effect?

• Will the user notice that the correct action is available?

• Will the user associate the correct action with the effect?

• If the correct action is performed, will the user see that progress is being made toward solution of the task?

Harry Hochheiser, [email protected] Baobab Health, February 2015

Action 2: Press #2(Wharton, et al. 1994)• Command to cancel forwarding: bip bip bip

• Failure Story

• Criteria: will the user be trying to achieve the right thing?

• Problem – the result why would you be trying to cancel forwarding?

• Criteria: Will user know that the action will have the right effect?

• Problem - might not recognize that correct action is “#2”

• Criteria: if the correct action is taken, will the user see that things are going ok?

• Problem -Will they recognize “bip bip bip” at first?

Harry Hochheiser, [email protected] Baobab Health, February 2015

Procedural Issues

• Always stay on track – proceed as if correct action had occurred

• If necessary, assume a “fix” - some additional user knowledge – to complete tasks correctly

Harry Hochheiser, [email protected] Baobab Health, February 2015

Cognitive Analysis of DecisionSupport for Antibiotic Ordering in a Neonatal Intensive Care UnitSheehan, et al. 2013, Applied Clinical Informatics

• CPOE in NICU

• Usability Assessment of cognitive impact

• Use Norman's Theory of Action to guide analysis

• Goal, plan execution response, interpretation/evaluation, new goals

• Gulfs of executions and evaluation

• Characterize gulfs..

Harry Hochheiser, [email protected] Baobab Health, February 2015

Cognitive Analysis….Sheehan, et al. 2013, Applied Clinical Informatics

• Cognitive distance

• “degree and magnitude of mental transformations required to bridge the gulfs of execution and evaluation”

• Semantic Distance

• “relationship between what the user wants to communicate and the meaning of the corresponding expression in the interface language”

• Articulatory Distance

• “the relationship between the meanings of the expression and their physical form.”

• Issue Distance

• “cognitive effort required when a shift in goal is necessary"

• Use these distances to characterize issues

Harry Hochheiser, [email protected] Baobab Health, February 2015

Cognitive Analysis….Sheehan, et al. 2013, Applied Clinical InformaticsFigure repeated from Rizzo, et al. 1997

Harry Hochheiser, [email protected] Baobab Health, February 2015

Cognitive Analysis….Sheehan, et al. 2013, Applied Clinical Informatics

• Two scenarios

• Developed with focus groups

• “Scenario A involved an infant with late-onset sepsis who required treatment with vancomycin and gentamicin. This infant had an elevated serum creatinine of 1.7, which reflects a dramatic decrease in kidney function. Because these antibiotics are primarily eliminated by the kidney, a creatinine level this high would trigger a specific alert that warns the prescriber that standard antibiotic dosing regimens should not be used for this infant. The first task in this scenario was to enter an order for vancomycin. The second was to enter an order for gentamicin.”

• “Scenario B involved an infant with early onset-sepsis. In this scenario, the infant required admission orders including gentamicin and ampicillin. The task was to enter the admission orders using the NICU general admission order set.”

Harry Hochheiser, [email protected] Baobab Health, February 2015

Cognitive Analysis….Sheehan, et al. 2013, Applied Clinical Informatics

• Use Nielsen's heuristics to classify problems..

• Subgoals,etc.

Harry Hochheiser, [email protected] Baobab Health, February 2015

Cognitive Analysis….Sheehan, et al. 2013, Applied Clinical Informatics

• Usability Problems

Harry Hochheiser, [email protected] Baobab Health, February 2015

Cognitive Analysis….Sheehan, et al. 2013, Applied Clinical Informatics

Harry Hochheiser, [email protected] Baobab Health, February 2015

Cognitive Analysis….Sheehan, et al. 2013, Applied Clinical Informatics

Articulatory Distance

Harry Hochheiser, [email protected] Baobab Health, February 2015

Cognitive Analysis….Sheehan, et al. 2013, Applied Clinical Informatics

Semantic Distance

Harry Hochheiser, [email protected] Baobab Health, February 2015

Cognitive Analysis….Sheehan, et al. 2013, Applied Clinical Informatics

• Summary of all problems