braddon sbme risk assessment presentation

Bring your medical device to market faster 1

Linda Braddon, Ph.D.

Risk Assessment for

Medical Devices


My Perspective

� Work with start up medical device companies

� Goal: Making great ideas into profitable products

� We see the same mistakes over and over again


Why do a Risk Assessment?

� You have to do it….

– It is one of the first things FDA will ask for!

– Required by law (21 CFR Section 820)

– Appears on regulatory submission checklists

� Will help define testing that should be done to prove

the safety of your device

� Helps to eliminate costs associated with recalls

� Offers a measure of protection from liability

� It is the right thing to do…


Risk Assessment is:

� Definition: An iterative process of assessing a

product’s benefit / risk balance

� Goal: Developing and implementing tools to

minimize risk while preserving its benefit

� Timing: Should be done through product’s lifecycle


Useful Resources

� ISO 14971 – Application of risk management to medical devices

� FDA Guidance – Incorporating human factors into risk management

� FDA Guidance – Premarketing Risk Assessment

� The focus of this talk is aimed at outlining the specifics for ISO 14971 compliance


Management Responsibility

� Provide evidence of commitment to risk management

– Adequate resources

– Qualified personnel (need appropriate records for proof)

� Have a policy for determining risk

� Review effectiveness of risk management policy

� Determine how acceptable risk will be determined

� Periodic review


Risk Management: The Steps

� Make a plan

� Implement plan

� Report results


� Implement plan

– Identify

– Evaluate

– Control

– Re-evaluate

– Communicate


Make a plan…


Why have a plan?

� Organized approach is necessary for success

� Plan provides roadmap

� Plan encourages objectivity*

� Prevents important elements from being forgotten

– Ex: Lifecycle of device will alter risk and risk evaluations

� Verification of activities is required and does not often happen if not in a plan.

� Needs to have a formal way to integrate production and post-production information into risk management


Risk Management Plan

� Activities must be planned

� Plan must be part of RM file

� Plan includes:

– Planned activities

– Assignment of responsibilities

– Requirements for review of RM activities

– Criteria for risk acceptability

– Verification activities

– Activities related to collection and review of production and

post-production information


Risk Management Team -

Qualified Personell

� Expertise in following areas:

– How is device constructed?

– How does device work?

– How is device produced?

– How is device used clinically?

� *Experts should be trained on how to apply the RM process.

– Records of qualification to provide objective evidence needs

to be collected.

– Records of resumes, training, etc does not have to be kept in

the RM file but should be pointed to…


Risk Management through

Lifecycle

� RM does not stop when medical device goes into production.

� During prototyping, evaluating risk levels is often a rough estimation of risk.

� Use of real device by real users WILL HIGHLIGHT new and different levels of risk not considered in earlier phases.


Start a Risk Management File…


Risk Management File

� Provides traceability for each identified hazard to:

– Risk analysis

– Risk Evaluation

– Implementation and verification of risk control measures

– Assessment of acceptability of residual risk

� Risk management file does not need to physically contain all the records and documents but should at least contain references to their location.

� File can be in any form (notebook, secure files on network, etc…)


Risk Analysis Process

� Description of medical device that was analyzed

� ID of person(s) and organizations who performed analysis

� Scope

– Broad: … for the development of a new medical device …

– Narrow: … evaluation of X change to an existing device…

� Date(s) assessment was performed


Analsyis Techniques

� Preliminary Hazards Analysis (PHA)

– Early in development

– Often little information or history

– Generic hazardous situations

� Fault Tree Analysis (FTA)

– Top down view

– “What is the undesirable event?”

� Failure Mode and Effect Analysis (FMEA)

– How could each component fail? How does a combination of

components fail?

� Hazard and Operability Study (HAZOP)

� Hazard Analysis and Critical Control Point (HACCP)


First questions to ask…


Key questions to ask about

device:

� What is intended use?

– Manufacturer shall document the intended use and reasonably

foreseeable misuse.

– Identify and document characteristics

– When possible, define limits

� What are the device characteristics?

This is where design inputs do double duty for

risk assessment and design control

� ID of hazards: How can it fail, break or hurt someone?

� Estimation of risk for hazards


How to start the analysis:

Questions to Ask (ISO 14971 Annex C)

� Intended use?

� Will the device be implanted?

� Will device be in contact with patient or other people?

� Is energy delivered or extracted from the patient?

� Is a substance delivered or extracted from the patient?

� Are biological materials processed by device for subsequent re-use?

� How is the device sterilized?

� Will device be routinely cleaned?

� Will medical device modify patient enviromentment?


More questions…

� Are measurements taken?

� Is device interpretative?

� Does device work with other devices?

� Are there unwanted outputs of energy or substances?

� Susceptible to environmental influence (ex: vibrations, spills)

� Does device influence environment? (radiation, etc…)

� Does device have consumables?

� Is maintanence or calibration necessary?

� Is there software?


Still more questions….

� Is there a restricted shelf life?

� Are there delayed or long term use effects?

� Will it be subject to mechanical forces?

� What determines lifetime of device (batteries?)

� Is the device intended for single use?

� How should the device be decommissioned or disposed of?

� Does installation or use require special training?

� How will the info for safe use be provided?

� Will a new manufacturing process need to be established?


Questions, questions, questions

� Is successful use dependent on human factors?

� Can the user interface contribute to user error?

� Can distractions cause use errors?

� Are there connecting parts or accessories? (External fixation)

� Is there a control interface?

� Does device display information?

� Is device controlled by a menu?

� Will device be used by a person with special needs?

� Can user interface be used to initiate user actions?


Are we done with questions?

� Is there an alarm system?

� Can the device be deliberately misused?

� Does the device hold data critical to patient care?

� Is device mobile or portable?


Scoring the risk…


Tools for Evaluating Risk

� Fault Tree Analysis (FTA)

� Failure Mode Effects Analysis (FMEA)

� Failure Mode Effects and Criticality Analysis (FMECA)


Fault Tree Analysis

� First identify a failure or safety hazard

� Next identify all possible ways to create hazard

� For example:

– Hazard = Electrical Shock

– How could this happen

• Transformer failure

• Safety Ground Failure


Fault Tree Analysis


What is an FMEA?

� Failure Mode Effects Analysis

� Goal: Eliminate failures before they happen

� Does not require complicated statistical analysis

� Does require human resource and time

� Focused on:

– Preventing defects

– Enhancing Safety

– Increasing Customer Satisfaction


FMEA / FMECA

� FMECA

– Same as FMEA with addition of criticality and often used

interchangably

� Assumes defect at component level

� Assess the effect

� Identifies Potential Solutions


FMEA – When should this be performed?

� When to perform

– Beginning of design effort

– Part of EACH design review

– Through the life cycle of device

� Types

– Design FMEA: Focuses on what could go wrong with a product in both

manufacturing operation and in service as a result of a weakness in the

design

– Process FMEA: Concentrates on reasons for potential failure during

manufacturing and in service

– System FMEA: Looks for potential problems and bottlenecks in larger

processes, such as production lines


FMEA Teams

� Team leader: Guides team through process, keeps notes, organizes

� Often 4 to 6 team members

– Design / Process engineer

– End users

– Sales / Marketing

– People familiar with product / process AND unfamiliar

individuals with fresh perspectives


Steps of FMEA

� Review design

� Brainstorm failure modes

� List effects of failure (device and patient)

� Assign a severity, occurrence and detection rating

� Calculate the Risk Priority Number (RPN)

� Prioritize failure modes based on RPN

� Take action to eliminate or reduce high risk failures

� Calculate new RPN

� Repeat corrective actions if necessary


Estimating Risk

� Severity of occurrence

� Probability or likelihood of occurrence

� Detection of occurrence


Severity: Safety vs Functional

RATING Patient Safety Severity Evaluation Criteria

1No Health Hazard:

� No physical effect of the physiological complaints anticipated

2

Limited Health Hazard:

� Extended Procedure Time

� Physiological complaints

� Temporary minor physical effect

3

Moderate Health Hazard:

� Additional procedure required

� Temporary but significant physical effects

� Permanent minor physical effects

4Severe Health Hazard:

� Permanent significant physical effects

5Catastrophic:

� Life Threatening

RATING Device Functionality Severity Evaluation Criteria

1

Insignificant:

� Cosmetic defect

� The failure will not have any perceptible effect

on the performance of the product

2

Low Significance:

� User nuisance

� Dissatisfaction on the part of the end user

� Product may be operable at reduced

performance

3

Moderate Significance:

� Compromised function / loss of minor function

� The user may notice a negative impact on the

product or system performance

4

High Significance:

� Compromised function / loss of major function

� Loss of system function - device completely

unusable

5

Extreme Significance:

� Regulatory issue

� Involves non-compliance with government

regulations


Design FMEA Rating: Occurence

RATING Occurrence Evaluation Criteria for Design FMEA

Qualitative Approach : Probability

of Failure due to specific cause

1 Remote: Failure unlikely

2 Low: Relatively few

3 Moderate: Occasional

4 High: Repeated failures

5 Extreme: Almost inevitable

Probability: Qualitative vs Quantitative

Better to have a very accurate qualitative definition

versus an inaccurate quantitative description.


Design FMEA rating: Detection

RATING Detection Evaluation Criteria for Design FMEA

Failure Mode likely to be detected during:

1 Prototype / Sample Production – Design house work

2

Design Verification and Validation – Benchtop testing or cadaver studies

3

Process Validation – Manufacturing facility validation for consistent

product.

4 Initial Clinical Use – After implanted in humans

5

After Commercialization – after mass production and use (this is FDA

reportable event)


Is risk acceptable?


Acceptable risk continued…


Now, what do you do about the

risk?


Risk Control

� Control options

� Implementation of control measures

� Residual risk evaluation

� Risk / Benefit analysis

� Risk from control measures

� Completeness of risk control


Risk Control

� Can the risk be reduced?

� How best can the risk be controlled?

– Design of device (best)

– Protective measures

– Information on safety

� Once risk control measures have been identified, they must be implemented and evaluated for effectiveness.


Risk Control

� Redesign

� Add protective measure

� Provide safety information


Has the risk control worked?


Implementation of Risk Control

Verification

� Verification is required to:

– Make sure risk control has been implemented in the final

design.

– To ensure the measure, as implemented, actually reduces risk.


Identification of Hazards

� Vary vague in ISO 14971

– “manufacturer shall compile documentation…”

� Common ways of identifying hazards:

– Fault tree

– Hazards Analysis


Is the remaining risk acceptable?


Remaining Risk

� Residual risk evaluation

– Is the risk acceptable?

– Can the risk be reduced (decrease severity, occurrence or

detection)?

� Risk / Benefit Analysis

– Is the benefit worth the risk when further risk mitigation is

not possible or practical?

� New Risks arising from risk management

� Completeness of Risk Control

– For example, maybe there are not electrical risks to your

device but you may want to consider documenting that fact.


Residual Risk:

Moving from weeds to balcony

� Broad perspective

� What do you do about the remaining risk?

� “Low as reasonably practicable” approach

– Does benefit outweigh risk?


Communication, communication,

communication….

Did I mention communication?


Risk Management Report

� High level document that provides evidence that plan has been fulfilled and objective achieved

� Prior to release for commercial distribution

� Has RM been appropriately implemented?

� Is overall residual risk acceptable?

� Are appropriate methods in place to obtain relevant production and post-production information?

� Report needs to be reviewed by appropriate personelle.


Production and Post Production

Information

� Must collect and review information about the medical device during production and post production phases.

� Consider things like:

– Installation

– Use

– Maintenance

� Should collect and review publicly available information about similar medical devices

– Is there an new or not considered risk?


Basic concepts of risk

management

� Identification of hazards

� Risk estimation

� Risk acceptability

� Risk control

� Risk / Benefit Analysis

� Overall risk evaluation


Hazards arising from faults

� A fault does not always result in a hazardous situation

� A hazardous situation does not always result in harm.

� Types of Faults:

– Random Faults (electronic component fails)

– Systemic faults (fuse to prevent harm is at the wrong rating or

an assay does not detect a new strain of a disease)


Getting help…


Guidance for Conducting Risk

Assessments

� ANSI/AAMI/ISO 14971

– Defines 3 regions of risk

• Broadly acceptable region

• ALARP Region

• Intolerable region

� FDA Guidance for Industry: Incorporating human factors engineering in

risk management

� FDA Guidance for Industry: Premarketing Risk Assessment

(specifically for drug products including biological drug products

� FDA Guidance for Industry: Design Control for Medical Device

Manufacturers

– Speaks to design review and risk assessment is an important part of design

review


ISO 14971

� “The manufacturer shall establish, document, and maintain throughout the lifecycle an ongoing process for identifying hazards associated with a medical device, estimating and evaluating associated risks, controlling these risks and monitoring the effectiveness of controls and shall include:

– Risk analysis

– Risk evaluation

– Risk Control

– Production and post production information


Insights for the trenches…


Biggest Mistakes We See…

� Risk assessment as an afterthought

� Static document versus living document

– It shouldn’t be just a checked off item for FDA

� Team assigned to perform FMEA

– Not diverse

– Too close to design

� Not using a yellow light as a warning


Scoring Changes Depending on

Phase of Lifecycle…

RATING Detection Evaluation Criteria for Design

FMEA


1Prototype / Sample Production

2Design Verification and Validation

3Process Validation

4Initial Clinical Use

5After Commercialization

RATING Detection Evaluation Criteria for Process FMEA


1Remote probability of leaving manufacturing area

containing the defect. Obvious defect.

2Low probability of product leaving the manufacturing

area containing the defect. Defect easily detectable.

3

Moderate probability of product leaving the

manufacturing area containing the

defect. Defect is somewhat more difficult to

detect.

4

High probability of product leaving the

manufacturing area containing the

defect. Detection requires special inspection.

5

Very high probability of product leaving

manufacturing area with defect. Defect may

not be found with sophisticated detection

techniques.


Risk Assessment of Human

Factors

� Devices used in ways not anticipated

� Device used in anticipated way that was not properly controlled

� Device is inconsistent with user’s “intuition” on how it should work


Details of Analysis

� Component level

� Combination level

� Effect of sequence of events


When is risk management

finished?

� Never as long as your device is available

� There are mistakes you will not find until it has been used and abused.

� Part of a RM plan is to make sure management is evaluating risk on a reasonable schedule.

� Must have a plan to act swiftly if something unexpected happens


Risk Assessment… Good for device and company

Making sure Design, Regulatory and Manufacturing are on the same page…

braddon sbme risk assessment presentation

Documents

risk managementbring

acceptable risk

application of risk

medical device companies

medical devicesbring

lifecycle of device

risk balance goal

success plan