Design and Development of Medical Devices through a Human Factors and

Usability Lens

Andy Schaudt, M.S., M.B.A.Director of Usability ServicesNational Center for Human Factors in HealthcareMedStar Institute for Innovation

October 8, 2013

How on earth do we design technologies/systems around drivers?

Do we give them bigger mirrors for putting on makeup?

Do we provide them a better steering wheel so they can drive with their knee?

Drivers are already at a disadvantage because they are human, let’s not make it more difficult/unsafe for them…

Human Error in Healthcare

“More than 1 million patients are injured and approximately 180,000 die each year” due to medical error – roughly equivalent to three commercial airliners crashed every two days. (Andrews LB, Stocking C, Krizek T, et al. 1997)

8th leading cause of death in the United States (U.S.), ahead of motor vehicle accidents, breast cancer and AIDS. (Leape 1994)

About half of adverse events were judged preventable with ordinary standards of care. (Vincent, Neale, Woloshynowych, 2001)

$17.1 Billion annual cost (Van Den Bos, et al. 2011)

Might be 10x more than previous research has shown (Claussen, et al. 2011)

Human Resources?

Engineering People’s Brains?

What is Human Factors Engineering?

“The scientific discipline concerning understanding of interactions among humans and other elements of a system, and applying theory, principles, data, and other methods to design in order to optimize human well-being and overall system performance.” As defined by the Human Factors and Ergonomics Society.

Human Factors

Psychology

Industrial Engineering

Cognitive Science

Anthropometry Biomechanics

Human Factors and Usability Benefits in Healthcare

• Can save lives• Increase safety by reducing human error• Improve system/device design• Improve reputation• Increases user acceptance• Facilitates a stronger regulatory submission• Believe it or not, it reduces overall costs…

Systems Approach

• Truly integrated systems result in higher performance, higher reliability, improved safety– Humans are one component in a larger system– Focus on the interaction or interface between people and

the equipment and environment– Fit the tools and environment to the person; not the

person to the tools and environment

Intended Users?Healthcare Providers? Patients and Family Members?

Intended Environments?

Shift in Healthcare

• Expansion in ambulatory care• Technology allowing for in-home care/monitoring• Conversion in legacy systems to more mobile-based

technology• Bottom line – users and environments are expanding,

and users’ roles are changing• Incorporating human factors and usability early and

throughout device design is crucial

Primary Problem

• IOM Report in 2000– Govt – 50% less errors in 5 years

• 13 years later…– Essentially NO CHANGE

• Why?– Focus still on individual performance– Solutions inconsistent with safety

science

Leape LL, Berwick DM. Five years after To Err Is Human: what have we learned? JAMA. May 18 2005;293(19)Wachter RM. The end of the beginning: Patient Safety Five Years After 'To Err Is Human'. Health Aff. 2004(11)Wachter RM. Patient Safety At Ten: Unmistakable Progress, Troubling Gaps. Health Aff. 2010 (29:1)Landrigan, Parry, et al. Temporal Trends in Rates of Patient Harm Resulting from Medical Care. NEJM 363(22): 2010 Shekelle, Pronovost, et al. Advancing the science of patient safety. Ann Int Med 154(10): 2011Longo, Hewett, Ge, Schubert. The long road to patient safety: a status report on patient safety systems. JAMA, 294(22): 2005.

Error• Human error is:

“A generic term to encompass all those occasions in which a planned sequence of mental or physical activities fails to achieve its intended outcome, and when these failures cannot be attributed to some chance agency.” – (Reason, 1990)

• Use error (most relevant for medical device manufacturers):“act or omission of an act that has a different result

than intended by the manufacturer or expected by the operator.” – (IEC, 2004)

Knowledge-Based

Rule-Based

Skill-Based

Improvisation in unfamiliar environmentsNo routines or rules available to help handle

Protocolized behaviorProcess, Procedure

Automated RoutinesRequire little conscious attention

Figure adapted from: Embrey D. Understanding Human Behaviour and Error, Human Reliability AssociatesBased on Rasmussen’s SRK Model of cognitive control, adapted to explain error by Reason (1990, 2008)

SLIPS & LAPSES

1. MISAPPLY GOOD RULE

2. NOT APPLY GOOD RULE

3. APPLY BAD RULE

TRIAL & ERROR

What about

Violations?

Human-machine System

Task

Lars-Ola Bligård (2007) [adapted from Sanders and McCormick, 1993; Danielsson, 2001]

Where Does the Breakdown Occur?• Use errors occur because of a mismatch between the system user,

equipment, task and environment (FDA, 1999). • Kaye and Crowley (2000) described six reasons why use-related hazards

occur:1. Devices are used in ways that were not anticipated;2. Devices are used in ways that were anticipated, but inadequately controlled for;3. Device use requires physical, perceptual, or cognitive abilities that exceed those

of the user;4. Device use is inconsistent with user’s expectations or intuition about device

operation;5. The use environment affects device operation and this effect is not understood

by the user; or6. The user’s physical, perceptual, or cognitive capacities are exceeded when using

the device in a particular environment.

Defibrillator ExampleCardiac arrest1. nurse with patient2. charges unit…3. clears patient…4. presses “on” button5. Machine powers down

– 2-3 minute delay in shock

Hoyer, Christensen, et al. Annals of Emergency Medicine 2008; 52(5): 512-514.Fairbanks and Wears. Annals of Emergency Medicine 2008; 52(5): 519-521.

Error is Inevitable

• If error is inevitable… How to improve safety?– Reduce the occurrence of human error

• With better design• NOT training and policy

– Mitigate the effects of inevitable error• With better design

– Better feedback– Forcing functions

“Every system is designed to achieve exactly the results it gets.”

- Don Berwick, former IHI president, former director of CMS

Another Challenge!

“The single greatest impediment to error prevention in the medical industry is that we punish people for making mistakes.”

--Lucian Leape, Testimony to congress

Medical Devices and Software• More than 1/3 of medical device incidents involve

user error• More than 1/2 of device recalls for design problems

involve the user interface• Website/user interface design normally costs

upwards of 70% of total development costs• More than 1/2 of device recalls for design problems

involve the user interface

~B.R. Phillips – Mayo Clinic

Let’s Recap…So Far

• You can’t eliminate human error• Healthcare has a “name, blame,

shame, and train” culture• There are numerous

environments and users to design a single system for… and it’s changing

• Is the road ahead futile for medical device designers?

Usability

• ISO 9241-11:1998• The effectiveness, efficiency, and satisfaction with which

specified users achieve specified goals in particular environments

• The components are explained as follows:– Effectiveness: The accuracy and completeness with which specified

users can achieve specified goals in particular environments.– Efficiency: The resources expended in relation to the accuracy and

completeness of goals achieved.– Satisfaction: The comfort and the acceptability of the work system

to its users and other people affected by its use.

Usability engineering process (IEC 60601-1-6)

Usability – Early and Often!

Pre-MarketPost-Market

Design Requirements

Development Deployment

Cost of Intervention in Product Lifecycle

# Design Alternatives

$ of Design Changes

Implement usability early and reduce # of interventions and their associated costs!

Adapted from Bias & Mayhew, 1994, p. 80

Tailor Your Usability Approach

Formative Recommendations• Standards

– E.G. ANSI/AAMI HE75– ISO/IEC 62366:2007

• Card Sorting• Physical Ergonomics• Storyboard• Wireframe• Task Analysis

– Hierarchical task analysis– Cognitive task analysis

• Risk Analysis – FMEA

– Task Analysis; Failure Modes and their Effects; rate Severity, Occurrence, and Detection; calculate Risk Priority Numbers.

– When there isn’t a strong “design control” and your system relies on the user, ask yourself why?

American National Standards Institute & Association for the Advancement of Medical Instrumentation (2009). HE75: Human Factors engineering – Design of medical devices. Retrieved on July 25, 2013 from http://www.aami.org/publications/standards/he75.html

Formative Recommendations, cont..• Heuristic Evaluation– assess compliance of a

device or system with regard to human factors design principles and a predetermined set of design heuristics

• Heuristics– Consistency & Standards– Visibility of System State– Match Between System &

World– Minimalist– Minimize Memory Load– Informative Feedback– Flexibility & Efficiency– Good Error Messages– Prevent Errors– Clear Closure– Others… Heuristics - Nielsen, 1995

Formative Recommendations, cont..• Formative User Testing– Exploratory and is intended to find broad categories of use

challenges– 4-5 users per group find ~80% of the problems (Virzi, 1992)– Think aloud protocol– Design to collect performance measures (e.g. errors,

violations, time-on-task) AND subjective feedback (e.g. opinion ratings, design recommendations)

Virzi, R.. Refining the Test Phase of Usability Evaluation: How Many Subjects is Enough? Human Factors, 1992, 34(4), 457 468

Summative Recommendations• Summative Evaluation

– Summative usability evaluation is the final validation intended to demonstrate that all risks have been identified and adequately addressed and that the impact of residual risks is minimized or acceptable. Usability evaluation of a complete, or near-complete, design under realistic conditions.

• Field User Events• Post-market Usability

Evaluations• Trade-off Studies

Regulatory Landscape

• Understand where your medical device fits here• Medical Device?– Class I General Controls– Class II General Controls and Special Controls– Class III General Controls and Premarket Approval

• Combination Product?• Mobile Medical App?

Medical Device• FDA Draft Human Factors Guidance - “The intent is to improve

the quality of the device user interface such that errors that occur during use of the device are either eliminated or reduced.”– Identify anticipated use-related hazards (derived analytically, see

Section 6) and unanticipated use-related hazards (derived through formative evaluations, see Section 7), and determine how hazardous use situations occur;

– Develop and apply strategies to mitigate or control use-related hazards (see Section 8); and

– Demonstrate safe and effective device use through human factors validation testing (see Section 10).

U.S. Department of Health and Human Services Food & Drug Administration Center for Drug Evaluation and Research (2011). Draft Guidance for Applying Human Factors and Usability Engineering to Optimize Medical Device Design. Report No. 1757. Retrieved on September 29, 2013 from: http://www.fda.gov/downloads/MedicalDevices/DeviceRegulationandGuidance/GuidanceDocuments/UCM259760.pdf

Combination Product• Examples: drug/device, biologic/device,

drug/biologic, or drug/device/biologic• No HF/usability guidance available (expected 2014)• Human factors evaluation of training program

structure and materials is common– Important to show the incorporation of user feedback in

the design evolution of the device/instrument AND the training program

Mobile Medical App

• Final guidance available (not HF/Usability focused)• Intended environment policies for/against?• Multi-use device, not dedicated

chance of

unintended user

interactions

Cost Arguments and Evidence for Human Factors Integration, Issue 1 / October 2006 / HFI DTC

In Summary

• Error in healthcare is substantial• Medical device manufacturers can focus on reducing

“use errors” and mitigating their effects through human factors/usability

• Incorporate user feedback/testing early and often throughout the iterative design process– Result in a safer product; benefits cost, time, and scope

• Keep a record of the user-focused design efforts and tell your usability story in your regulatory submission

Andy Schaudt, M.S., M.B.A.Director of Usability Services

National Center for Human Factors in HealthcareMedStar Institute for InnovationAndy.Schaudt@MedicalHFE.org

www.MedicalHumanFactors.net202-244-9844