notes on user observations for baobab health trust, march 2014
DESCRIPTION
Notes on conducting qualitative observation for user requirements, presented to Baobab Health Trust, Lilongwe, Malawi, March 2014TRANSCRIPT
Boabab Health, March 2014 Harry Hochheiser, [email protected]
Human-Computer Interaction - Observations
Harry Hochheiser University of Pittsburgh Department of Biomedical Informatics [email protected] !+1 410 648 9300
Attribution-ShareAlike CC BY-SA
Baobab Health, March 2014Harry Hochheiser, [email protected]
Research Setup
● User(s) with clinical problems
● Current approach not optimal
● How do we know what to build?
Baobab Health, March 2014Harry Hochheiser, [email protected]
Important Claim #1
● Understanding work in context – goals, motivations, priorities, behavior, difficulties, etc. is necessary for building better systems
!● Most go beyond just talking about computer systems to
address bigger picture questions
!
● What if we build a pharmacy prescription tracking system…
● ..but the pharmacy never has any medication…
Baobab Health, March 2014Harry Hochheiser, [email protected]
Important Claim #2 ● Successful implementations may require work redesign
● Translating the same old methods and procedures to computers may not help much
● Must go beyond what we are doing?
● Instead, ask why?
!
● In-depth qualitative research needed to inform these efforts
● Not numbers. Rather, goals, methods, activities..
Baobab Health, March 2014Harry Hochheiser, [email protected]
Important Claim #3 ● In-depth qualitative research needed to inform these
efforts
● Not numbers. Rather, goals, methods, activities..
● Watch, listen , interpret
Baobab Health, March 2014Harry Hochheiser, [email protected]
What are the factors that influence usability?
Is a given tool usable for all users?
For some users?
!
Is an interface full of text usable… for someone who can’t read?
Baobab Health, March 2014Harry Hochheiser, [email protected]
Determinants of Usability
Context: social, organizational, etc.
Tool
Task Data analysis? Writing? Graphing?
usability= f(user,tool, task, context,system)
System: desktop, mobile, tablet, etc.
Baobab Health, March 2014Harry Hochheiser, [email protected]
Systems Engineering for Patient Safety
Lowry, et al. Integrating Electronic Health Records into Clinical Workflow: An Application of Human Factors Modeling Methods to Ambulatory Care National Institute of Standards and Technology NISTIR 7988 http://dx.doi.org/10.6028/NIST.IR.7988
Baobab Health, March 2014Harry Hochheiser, [email protected]
Broad Goal
• Use study of users to understand how to build usable systems
• What is their work?
• What struggles are they facing?
• What information do they need?
• Why do they do what they do?
• How can IT help? Where can’t it help?
• What will be required for an IT tool to work?
Baobab Health, March 2014Harry Hochheiser, [email protected]
Approaches• Engage with users to see how work is done
• Interview
• Observe
• Collaborate
• Build models describing work
• Validate models with users
• Use models to design appropriate interactions.
Baobab Health, March 2014Harry Hochheiser, [email protected]
Some Methodologies
Participatory Design
Contextual Design
Scenario-Based Design
Cooperative Inquiry
Participatory Action
Research
Methodologies
Baobab Health, March 2014Harry Hochheiser, [email protected]
Key Questions & Tradeoffs
● Who to involve?
● When to involve users?
● How to collect information?
● How to interpret?
● How to inform design?
● How to evaluate success?
!● Usual tradeoffs apply: Never enough time or money
Baobab Health, March 2014Harry Hochheiser, [email protected]
Stakeholder AnalysisRosson & Carroll 2002
● Identify stakeholder groups
● Background
● Expectations
● Needs
● Preferences
● Concerns
● Values
!● An important, but often overlooked step
Baobab Health, March 2014Harry Hochheiser, [email protected]
Stakeholders
● Anyone who has an interest in the outcome of a system
● Work, play, or some other aspect of life
● Customer – those who pay for the work
● User – those who work with the system
● Others – perhaps those who are described by data in the system
Baobab Health, March 2014Harry Hochheiser, [email protected]
Stakeholders - Challenges
● Defined by roles, not by person
● Billing clerk for the hospital system is likely also a health-care consumer
● Must identify people who can speak to different roles?
● How can we meaningfully integrate understanding of needs of diverse users?
● Patients, practitioners, financial people, bureaucrats?
Baobab Health, March 2014Harry Hochheiser, [email protected]
A Spectrum of Possibilities for Engaging Stakeholders
Focus Groups
Traditional Written Requirements
Interviews
ObservationContextual InterviewsEthnography/ Participatory Design
Low Cost, Low Fidelity
High Cost, High Fidelity
Surveys
Diaries/ActivityRecording
Baobab Health, March 2014Harry Hochheiser, [email protected]
Observation: When• Understanding work in complex environments
• Often used when interviews with users are not possible
• busy clinical environments
• Even if possible, interviews might not be optimal
• Ecological validity: observations must be as close a match to reality as is practical.
• Least demanding on users and context
Baobab Health, March 2014Harry Hochheiser, [email protected]
Observation• Watch users as they complete tasks
• Who is involved
• What they are trying to accomplish
• How they meet their goals
• Look for
• struggles
• breakdowns
• workarounds
• contextual factors
• improvisation
Baobab Health, March 2014Harry Hochheiser, [email protected]
Methodology• Take notes - lots
• Pictures
• Diagrams
• Audio recordings (if privacy concerns allow)
• No interruption
• possibly debrief with user afterwards
• Multiple methodologies - triangulation
• Multiple observers - validation
Baobab Health, March 2014Harry Hochheiser, [email protected]
Guidelines• Get out of the way
• particularly in busy clinics
• Respect participant privacy
• no recording or photos in clinic
• Consider observing from multiple perspectives
• clinician and patient
Baobab Health, March 2014Harry Hochheiser, [email protected]
After the observation• Debrief
• Summarize notes
• Transcribe any recordings
• Analyze
• build models (more on this later)
• identify new questions - what did you not ask
• ask these at subsequent interviews
• Do this as soon as possible!
Baobab Health, March 2014Harry Hochheiser, [email protected]
Iterative process
Data Collection
Analysis New Questions
Models/ Theories Convergence
Baobab Health, March 2014Harry Hochheiser, [email protected]
Drawbacks of observations• Receptive only
• Potential for misinterpretation
• Observation of work as it is done
• not as management or others think it is done.
• Solutions:
• multiple observers
• Debrief/review with participants post fact
• “I think this is what I saw…”
Baobab Health, March 2014Harry Hochheiser, [email protected]
Iterative process
Data Collection
Analysis New Questions
Models/ Theories Convergence
Validation with Users
Baobab Health, March 2014Harry Hochheiser, [email protected]
“All models are wrong…”•Many possible ways to try to model user work
•Remember the goal:
• to do develop rich and valid understanding of work processes
• to use this understanding to drive design of appropriate software
• The particular methods may matter less than the process
• iteration
• validation
Baobab Health, March 2014Harry Hochheiser, [email protected]
Key Points• Understanding how work is done is vital for building tools
to support that work.
• Try to understand goals
• Redesign work if necessary
• Observations can provide useful data for driving these processes
• Need appropriate participants, triangulation, and validation.
Baobab Health, March 2014Harry Hochheiser, [email protected]
Tomorrow: Bwaila ANC• Meet at Bwaila at 8:15
• Observe ANC clinic
• Return to Baobab: 9:45
• Discuss/review observations: 10:15
• Contextual inquiry presentation: 10:45
• Lunch: 11:45
• Arrive at Bwaila for contextual interviews with ANC clinicians and mock patients: 1:15