thomas loveday - nsw clinical excellence commission
TRANSCRIPT
![Page 1: Thomas Loveday - NSW Clinical Excellence Commission](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5873cfae1a28ab9d168b5fc3/html5/thumbnails/1.jpg)
Considering the ‘Human Factor’ in Medical Device Procurement
Thomas LovedaySnr Manager, Organisational Development and Human Factors
October 2016
![Page 2: Thomas Loveday - NSW Clinical Excellence Commission](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5873cfae1a28ab9d168b5fc3/html5/thumbnails/2.jpg)
Presentation OverviewThis presentation will:• A brief review of why ‘usability’ is a core concern
in medical device procurement• How Human Factors provides a framework for
evaluating usability• Designing for usability OR evaluating usability• The usability evaluation processes/tools being
developed within HealthShare NSW
![Page 3: Thomas Loveday - NSW Clinical Excellence Commission](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5873cfae1a28ab9d168b5fc3/html5/thumbnails/3.jpg)
Clinical Engagement in Procurement• Healthcare staff are now formally involved in:
– Specification writing
– Ongoing contract management
– Desk top evaluation products
– Local site trials
3
Includes one usability item
![Page 4: Thomas Loveday - NSW Clinical Excellence Commission](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5873cfae1a28ab9d168b5fc3/html5/thumbnails/4.jpg)
Why is usability critical?
4
![Page 5: Thomas Loveday - NSW Clinical Excellence Commission](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5873cfae1a28ab9d168b5fc3/html5/thumbnails/5.jpg)
What is Human Factors?• Human Factors is the practice of studying and
designing systems so that they fit the peoplewithin them. It is also known as:
• User-Centred Design (UCD)
• Ergonomics (physical, cognitive, organisational)
![Page 6: Thomas Loveday - NSW Clinical Excellence Commission](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5873cfae1a28ab9d168b5fc3/html5/thumbnails/6.jpg)
The Human Factor in Health• Doctors
• Nurses
• Pharmacists
• Clerical staff
• Cleaners
• Patients
![Page 7: Thomas Loveday - NSW Clinical Excellence Commission](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5873cfae1a28ab9d168b5fc3/html5/thumbnails/7.jpg)
Human Factors• Physiological characteristics
– including vision, strength, flexibility, height
• Psychological characteristics– Including perception, memory, intuition, stress,
fatigue, values
• Proprioceptive characteristics– Including hand-eye coordination, spatial awareness,
balance
![Page 8: Thomas Loveday - NSW Clinical Excellence Commission](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5873cfae1a28ab9d168b5fc3/html5/thumbnails/8.jpg)
System Factors• Communication and interaction between
individuals, teams and services
• The formal structure and the informal culture of the organisation
• Interactions with equipment and physical environment
![Page 9: Thomas Loveday - NSW Clinical Excellence Commission](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5873cfae1a28ab9d168b5fc3/html5/thumbnails/9.jpg)
Change the System or the Human?• Hard (impossible) to change many human factors
– Universal; and/or
– Fixed (i.e. height, perception, memory)
• We can procure devices that accommodate the “human factor” and reduce the incidence of use error.
![Page 10: Thomas Loveday - NSW Clinical Excellence Commission](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5873cfae1a28ab9d168b5fc3/html5/thumbnails/10.jpg)
Comprehensive Device Usability Evaluation Process
10
(The Goal)
![Page 11: Thomas Loveday - NSW Clinical Excellence Commission](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5873cfae1a28ab9d168b5fc3/html5/thumbnails/11.jpg)
HF Specifications in Contracts• Request compliance with:
– AAMI/ANSI HE75:2009;
– ISO/IEC 62366-1:2015;
– ANSI/AAMI/ISO 14971:2007; AND/OR
– ISO 60601-1-8:2006.
• Additional HF specifications identified by:– HF Specialist
– Human Factors Advisory Committee
![Page 12: Thomas Loveday - NSW Clinical Excellence Commission](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5873cfae1a28ab9d168b5fc3/html5/thumbnails/12.jpg)
Heuristic Evaluation• A type of usability inspection method, where
3+ evaluators examine a device for consistency with usability principles
• Designed for interfaces, but has been adapted for devices (Zhang et al, 2004)
![Page 13: Thomas Loveday - NSW Clinical Excellence Commission](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5873cfae1a28ab9d168b5fc3/html5/thumbnails/13.jpg)
Zhang et al (2004) Heuristics1. Consistency and standards – Users should be informed about what is going on
with the system
2. Visibility of system state. Users should be informed about what is going on with the system.
3. Match between system and world – Clear conceptual link between controls and outcomes
4. Minimalist – Any extraneous information is a distraction and a slow-down.
5. Minimize memory load – Users should not have to memorize a lot of information to carry out tasks.
6. Informative feedback – Users should be given prompt and informative feedback about their actions.
7. Flexibility and efficiency – Allow users to create shortcuts to accelerate their performance (where appropriate).
![Page 14: Thomas Loveday - NSW Clinical Excellence Commission](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5873cfae1a28ab9d168b5fc3/html5/thumbnails/14.jpg)
Zhang et al (2004) Heuristics8. Good error messages – Device messages/alerts should be informative enough that
users can understand the nature of errors, learn from errors, and recover from errors.
9. Prevent errors – It is always better to design interfaces that prevent errors from happening.
10. Clear closure – Every task has a beginning and an end.
11. Reversible actions – Users should be allowed to recover from errors.
12. Use users’ language – The language should be always presented in a form understandable by the intended users.
13. Users in control – Do not give users that impression that they are controlled by the systems.
14. Help and documentation – Always provide help when needed.
![Page 15: Thomas Loveday - NSW Clinical Excellence Commission](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5873cfae1a28ab9d168b5fc3/html5/thumbnails/15.jpg)
Method• Evaluator completes key device tasks several
times to identify key user interactions
– For ‘complex’ devices ask an expert to help
• Compares each ‘interaction’ against usability principles (if they apply)
![Page 16: Thomas Loveday - NSW Clinical Excellence Commission](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5873cfae1a28ab9d168b5fc3/html5/thumbnails/16.jpg)
Heuristic Evaluation Cheap
Fast
Requires minimal training
Not specific
Subjective
No user input/involvement
Effectiveness dependent on HF expertise
![Page 17: Thomas Loveday - NSW Clinical Excellence Commission](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5873cfae1a28ab9d168b5fc3/html5/thumbnails/17.jpg)
Simulation-based usability testing
• Control risk;• Observe rare events;• Provide experimental control;• Incorporate qualitative AND
quantitative metrics.
17
Usability Testing can identifying impediments to “ease of use”
and “goal completion” by testing it on real users.
![Page 18: Thomas Loveday - NSW Clinical Excellence Commission](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5873cfae1a28ab9d168b5fc3/html5/thumbnails/18.jpg)
What happens in a usability test?
Courtesy of Dr Cate McIntosh
![Page 19: Thomas Loveday - NSW Clinical Excellence Commission](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5873cfae1a28ab9d168b5fc3/html5/thumbnails/19.jpg)
What are the results of a test?• Identify impediments to ease of use based on:
– Errors
– Time to completion
– Verbal statements during task
– Where users got “stuck”
– Etc
• There metrics provide a basis for comparing devices (comparative testing)
![Page 20: Thomas Loveday - NSW Clinical Excellence Commission](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5873cfae1a28ab9d168b5fc3/html5/thumbnails/20.jpg)
Usability testing Otherwise difficult to predict what a first-time user will experience
Provides more detailed information
As few as five ‘users’ can reveal >90% of design issues
Scenario development requires information gathering
Comparative testing is resource intensive
(2 facilitators x 5 sessions x No. of devices in a simulation suite)
![Page 21: Thomas Loveday - NSW Clinical Excellence Commission](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5873cfae1a28ab9d168b5fc3/html5/thumbnails/21.jpg)
Local site trials
Relatively resource light Real-world evidence May not encounter rare critical events Present risk to patients
Requires rigour and metrics for comparison of devices
![Page 22: Thomas Loveday - NSW Clinical Excellence Commission](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5873cfae1a28ab9d168b5fc3/html5/thumbnails/22.jpg)
Which devices?High-risk categories?
• Infusion Pumps
• PCA Pumps
• Syringe drivers
• Defibrillators
• Beds
![Page 23: Thomas Loveday - NSW Clinical Excellence Commission](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5873cfae1a28ab9d168b5fc3/html5/thumbnails/23.jpg)
The NSW Health Approach
23
(the reality)
![Page 24: Thomas Loveday - NSW Clinical Excellence Commission](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5873cfae1a28ab9d168b5fc3/html5/thumbnails/24.jpg)
Raj Ratwani quote“Summative testing requirements occur at the end of development of the product and any design flaws that are identified are unlikely to be addressed since the product has already been fully developed.”Testimony of Raj M. Ratwani, PhD to U.S. Senate Committee on Health, Education, Labor and Pensions
24
![Page 25: Thomas Loveday - NSW Clinical Excellence Commission](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5873cfae1a28ab9d168b5fc3/html5/thumbnails/25.jpg)
Vendor self-assessment
Vendor asked to identify user-centred design (UCD) methods used during development of device
2.1With which of the following Human Factors standard(s) does the device design and/or evaluation comply?
2.2Has the device or a similar device (see Note 2) ever undergone formal Human Factors assessment as part of the design process?
2.2.1Did the Human Factors Assessment include a review of prior assessments and studies of similar devices to identify and manage potential usability risks?
2.2.2Did the Human Factors Assessment include a 'Heuristic Evaluation' (i.e. a panel of experts reviews the device and judges its compliance with recognised usability principles)?
2.2.3Did the Human Factors Assessment include 'Usability Testing?'
![Page 26: Thomas Loveday - NSW Clinical Excellence Commission](https://reader034.vdocuments.mx/reader034/viewer/2022052418/5873cfae1a28ab9d168b5fc3/html5/thumbnails/26.jpg)
Vendor self-assessment Feasible and sustainable
Increase pressure on manufacturers to adopt UCD practices
UCD processes may be superficial
May not capture unique local critical tasks.
26