fmea matrix

UB TCIE | School of Engineering & Applied Sciences 1576 Sweet Home Road, Suite 212, Amherst, NY 14228

© University at Buffalo, All rights reserved.

www.tcie.buffalo.edu

v1.0

FMEA: Risk Perception and the

RPN Index

Harrison W. Kelly III, Ph.D.

and

James Davie, CQE

0

1960s - First formal FMEAs used during the Apollo missions

1974 - Navy developed MIL-STD-1629 regarding use of FMEA

1993 - AIAG- ASQ copyrighted FMEA standards

1995 - International Maritime Organization adopted FMEA

Today - common part of comprehensive quality systems (21 CFR 820, ISO 13485, TS 16949, ISO 9001, etc.)

FMEA Origins to Today

1

– What might go wrong with the product, process,

or system?

– What effect would this failure have?

– How significant is it if it occurs?

– What might cause the failure?

– How often will it occur?

– How likely is it that we can find it?

– What should we do about it if we find it?

Basis of FMEA = Simple questions

2

Captured on FMEA Form

3

FMEA Focus

• DFMEA - Eliminate

potential failures

before they are

engineered into

products and systems

– Reduce product costs

by eliminating changes

and rework

– Improve overall design

standards

• PFMEA - Prevent

failures from reaching

the customer

– Increase reliability of

products by detecting

failures before the

product is sold

4

• Process: – Used to consider failure modes associated with the

manufacturing and assembly processes.

• Project: – Used to consider failures that could happen during a major

program.

• Software: – Used to consider failure modes associated with software

functions.

• Design: – Used to consider failure modes of products and components

long before they are manufactured; should always be completed well in advance of a prototype build.

• System (Equipment): – Used to consider failure modes for system and subsystem level

functions

Applications of FMEA

5

Objective: Risk Abatement

• FMEA Team:

– Identifies failures

– Rates the failures

• RPN = Severity * Occurrence * Detection

– Determines corrective actions

– Takes corrective action

• Can this be achieved objectively?

6

Discussion of Measurement Scales

• Ordinal measurement scales

– Likert ranking scales (1-10) presented with

descriptive examples to assist in the consistent

evaluation of events

– Does not indicate how much of a given

characteristic an item may have, but simply that

the item has more or less of the characteristic

– The magnitude of difference between two items

cannot be interpreted

7

Severity

• Rating of the seriousness of the effect of the potential system failure mode to the system or user and should be based on the worst-case scenario Stamatis (2003)

• Teng & Ho (1996) proposed a general severity scale guideline for organizations to follow when customizing the scale for their specific needs.

• 10 Catastrophic Effect (Death or Near Death)

• 9 & 8 Critical Effect (Immediate Medical Attention)

• 7 & 6 Major Effect (Medical Attention Required)

• 5 & 4 Minor Effect (First Aid Required)

• 3 & 2 Trivial Effect (Minor Physical Issue)

• 1 No Effect (No impact)

8

Severity Ranking Guidelines

Borr

ow

ed h

eavily

fro

m P

ote

ntial F

ailu

re M

ode a

nd E

ffects

Analy

sis

(F

ME

A),

Refe

rence M

anual, 1

995. P

g. 35. C

hry

sle

r C

orp

ora

tion, F

ord

Moto

r C

om

pany,

Genera

l M

oto

rs C

orp

ora

tion.

9

Severity Ranking

10

Occurrence

• The estimated number of failures that could

occur associated with a specific failure mode

Stamatis (2003)

• The occurrence scale does not provide a

narrative description of situations, but does

provide a breakdown of failure rates in

percentages, or even PPM.

• In the case of low-volume manufacturing, the

scale should be redefined to meet the scope

and objectives of the organization. 11

Occurrence Ranking Guidelines

Borrowed heavily from Potential Failure Mode and Effects Analysis (FMEA), Reference Manual, 1995. Pg. 39. Chrysler Corporation, Ford Motor Company,

General Motors Corporation.

12

Occurrence Ranking

13

Detection

• The likelihood that the system will identify a

failure before it reaches the customer

Stamatis (2003)

• If direct data is not available for a system, i.e.

the system is under development, the data

used for the purpose of risk analysis can be

collected from existing similar process and/or

technologies

14

Detection Ranking Guidelines

Borrowed heavily from Potential Failure Mode and Effects Analysis (FMEA), Reference Manual, 1995. Pg. 39. Chrysler Corporation, Ford Motor Company,

General Motors Corporation.

15

Risk Priority Number (RPN)

• RPN = Severity*Occurrence*Detection

• Indicates that one failure mode has more or

less “risk” than other failure modes

• After risk reduction and mitigation activities

have been completed, the failure mode is

reassessed and a new RPN is determined

– No inference can be to the amount of risk that has

been eliminated, only that the risk is less than the

original analysis

16

• Action to eliminate or reduce high-risk failure modes: – Ideally the failure mode should be eliminated

– May not be possible, in which case the risk of failure should be reduced

– Easiest approach is to increase the probability that the failure will be detected (D) • e.g. warning bells, alerts

• often costly and do not actually improve the quality of the product

– Reducing the severity (S) is important for safety purposes

– Best opportunity for improvement is to reduce the occurrence of failure

Reducing and/or Eliminating Failures

17

Re-Evaluate Risk

18

Some Areas of Concern

• Process by which scoring occurs

– Group decision-making

– Anchor bias

– Confirmation bias

• Process by which a point of action is chosen

• Does the process of rating failure mode effect

reflect natural perception of risk or does it

introduce error through abstraction?

19

UB TCIE | School of Engineering & Applied Sciences 1576 Sweet Home Road, Suite 212, Amherst, NY 14228

© University at Buffalo, All rights reserved.

www.tcie.buffalo.edu

v1.0

An Analysis of Risk Perception and

the RPN Index within Failure Mode

and Effects Analysis

James L. Davie

Thesis Work

Problems with RPN Calculations

• Uniqueness

– There are many different combinations of SOD

values that can generate the same RPN

– SOD scales should be equally weighted when

determining the associated risk (Franceschini,

Galletto, 2001)

21

Problems with RPN Calculations

• There are only 120 individual RPN outcomes

– Within these outcomes, some RPN values can be

generated by as many as 24 different

combinations of SOD

• RPN does not satisfy the usual requirements

of measure and there is no algebraic

expression that reflects this scale (Ben-Daya

& Raouf, 1996)

22

Problems with FMEA

• The FMEA makes the assumption that only

single-point failures can occur within the

system (Puente et al, 2001)

• FMEA should not be used as a stand-alone

risk analysis tool

• Should be combined with FTA or process flow

diagram to account for compound failure

events

23

Problems with FMEA Risk

Assessment

• There is no specified method for determining

when action is required

• Strategies used

– Pre-defined value

• May be used as a “goal post” to not exceed

– Pareto

• May require risk reduction when not required

24

Risk-Based Decision Making

• Wong (2005) identified that predictors of risk-taking behaviors do not exert direct effects on risky decision-making. Rather, their effects are mitigated by risk perception and risk propensity.

– Risk perception is the decision maker’s assessment of inherent risk in a situation, i.e. natural perception.

– Risk propensity is the decision maker’s tendency to take or avoid risk.

• MacCrimmon & Wehrung (1990) show that more mature decision makers, in terms of age and seniority, have a lower risk propensity and are more risk averse than those subjects considered to be less mature.

25

Research Objectives

This research focuses on the ability of

individuals to evaluate risk associated with a

given failure mode, based on their natural

perception of risk and the rank established by

the pre-defined component scales.

26

Approach

An experimental design was created to study

these factors using established FMEA ranking

scales to analyze the participants’ natural

method of interpreting risk when presented in

a narrative format vs. the FMEA method.

27

Methodology

• A sampling of employees from manufacturing

companies were used in order to compare the natural

human perception of risk when presented in a

narrative situation vs. the perception when presented

in the FMEA format.

• The results of this study will assist in establishing a

relationship between the way people naturally

perceive risk and the way risk is analyzed in the FMEA

matrix.

• This comparison will help to determine if a correlation

exists between innate risk perception and FMEA risk

analysis.

28

Methodology

• Factors

– The comparison of the “Narrative Survey” vs. “FMEA Matrix” was the first factor of this experiment. The second factor is the order in which the questions were presented.

– The same questions were used for the narrative survey and FMEA matrix. The order of the questions were randomized through the creation of six different ordered versions, generating 25 unique order pairs.

• Covariates

– Covariates within designed experiments are uncontrolled variables that influence the response. The covariates for this research were gender, educational background, and FMEA experience. These items were analyzed through an ANOVA analysis.

29

Participants

• The participants for this study included employees of

manufacturing organizations representing different areas of

system responsibility.

• Participation was completely voluntary and participants

had the right to withdraw from the study at any time.

• It was clearly communicated that participating or not

participating in the study would not impact employment

status.

• Each participant was presented with a consent document

that detailed their rights as a participant, and obtained

approval for the response information to be used as part of

this study.

30

Participant Training

• Introductory training was provided to participants on the FMEA

method, describing the concepts of Severity, Occurrence, and

Detection.

• A generalized scale was provided for participant use to assist

in the selection of the rank values.

• The survey consisted of unbiased and non-leading closed

form questions designed to study the difference in the

perceived risk associated with a situational description.

• The participants were provided a 10-minute break between the

narrative and FMEA matrix sections of the survey.

• The ordering of the questions was randomized from the

narrative portion to the FMEA matrix, removing any bias that

could be introduced from the repeated ordering of questions.

31

Experimental Procedure

• Participants were asked to complete the following: – Read and sign the provided consent form indicating that they

understand their rights as a participant

– Complete the three demographic questions at the top of the survey

– Complete the 27-question narrative survey to assess perception of risk associated with the situational circumstances

– Take a 10-minute break

– Read the training information provided for the FMEA form

– Complete the 27-question FMEA matrix to assess perception of risk associated with the situational circumstances

• Each questionnaire was estimated to take 15 to 30 minutes to complete, but participants were able to use more time if needed

32

Results – Demographic Measure

Analysis

Of 100 candidates identified as potential participants, 34 provided responses in a timely manner. They completed the following demographic questions.

• Gender

– 25 were male (73.5%) and 9 were female (26.5%)

– The gender demographic was the only one found through the ANOVA analysis to impact the perception of Severity and Occurrence. Gender did not impact the perception of Detection.

• Education

– The Mean, Median and Mode indicate that the average participant has

a bachelor’s degree. Educational level was not found to have an

impact on the perception of Severity, Occurrence or Detection.

• Years of Experience

– The participants’ Mean experience level was 9.7 years. The Median

was 7 years and the Mode was 0.5 years, indicating that this

demographic is not normally distributed. Years of experience did not

impact the perception of Severity, Occurrence, or Detection.

33

Analysis of Risk Perception Responses:

No Experience

• Participants with no prior

FMEA experience were not

able to accurately assign

risk to the narrative survey,

or to the FMEA survey with

any correlation to the

planned RPN values.

• This indicates the

importance of clear and

effective training of

personnel responsible for

conducting risk analysis

activities.

RPN

R_

RP

N

10008006004002000

1000

800

600

400

200

0

S 0.478224

R-Sq 48.4%

R-Sq(adj) 48.3%

Experimental RPN Condition vs RPN Response when Exp = 0logten(R_RPN) = - 0.01938 + 0.9155 logten(RPN)

RPN

Sco

re

10008006004002000

120

100

80

60

40

20

0

S 0.375057

R-Sq 37.1%

R-Sq(adj) 37.0%

Experimental RPN Condition vs SCORE Response when Exp = 0logten(Score) = 0.3143 + 0.5690 logten(RPN)

34


Two Years of Experience

• Here the participants were

presented with the same

problem, but generated a

very different result.

• When people are trained,

they are able to assess risk

as planned by the FMEA.

• However, this assessment

of risk does not correlate

with their actual perception

of risk, as shown in the

RPN condition vs. Score

Response chart.

RPN

R_

RP

N

10008006004002000

900

800

700

600

500

400

300

200

100

0

S 0.129724

R-Sq 95.3%

R-Sq(adj) 95.1%


RPN

Sco

re

10008006004002000

140

120

100

80

60

40

20

0

S 0.484197

R-Sq 49.4%

R-Sq(adj) 47.4%

Experimental RPN Condition vs SCORE Response when Exp = 2logten(Score) = - 0.6312 + 0.9114 logten(RPN)

35


Four Years of Experience

• For these participants, the Log-

Log relationship accurately

describes the data set for both

comparisons.

• The score response data

demonstrates less variation

than the responses for 2 years.

• The variation observed here

shows that the narrative scores

were viewed as more risky than

the planned value. This is due

to the influence of severity on

the occurrence in risk

perception.

RPN

R_

RP

N

10008006004002000

800

700

600

500

400

300

200

100

0

S 0.0706140

R-Sq 97.0%

R-Sq(adj) 96.9%

Experimental RPN Condition vs RPN Response when Exp = 4logten(R_RPN) = 0.5611 + 0.7646 logten(RPN)

RPN

Sco

re

10008006004002000

120

100

80

60

40

20

0

S 0.506764

R-Sq 21.3%

R-Sq(adj) 18.2%


36


Eight Years of Experience

• With this group, the

concern is that the RPN

interpreted low risk and the

FMEA characterized the

situation as low risk.

• Therefore, the failure mode

will not be considered for

additional risk analysis and

reduction.

• However, when asked to

rate the risk without the

FMEA, these situations

were considered to be a

high risk.

RPN

R_

RP

N

10008006004002000

1000

800

600

400

200

0

S 0.189741

R-Sq 90.8%

R-Sq(adj) 90.6%


RPN

Sco

re

10008006004002000

90

80

70

60

50

40

30

20

10

0

S 0.398748

R-Sq 22.0%

R-Sq(adj) 20.5%


37


Seventeen Years of Experience

• For this data set, both the

RPN response and score

response closely followed the

planned values.

• Some narrative conditions

were assessed higher than

the planned condition due to

the influence of severity on

the perception of occurrence.

• This data indicates that

participants with this level of

experience are able to

consider the components of

risk without being prompted

by the matrix.

RPN

R_

RP

N

10008006004002000

1200

1000

800

600

400

200

0

S 0.0795966

R-Sq 98.9%

R-Sq(adj) 98.9%


RPN

Sco

re

10008006004002000

140

120

100

80

60

40

20

0

S 0.266956

R-Sq 66.3%

R-Sq(adj) 64.9%


38


Twenty-Five Years of Experience

• This data set shows the

strongest correlation of both the

RPN response and Score

response to the planned

condition.

• Here, the concern is that the

RPN was low and the FMEA

characterized the failure modes

as low risk. Therefore, no

additional action is required.

• When asked to rate the risk

without the FMEA, some

situations were perceived as

higher risk than the RPN

indicates.

RPN

R_

RP

N

10008006004002000

1200

1000

800

600

400

200

0

S 0.0135864

R-Sq 100.0%

R-Sq(adj) 100.0%


RPN

Sco

re

10008006004002000

200

150

100

50

0

S 0.363871

R-Sq 62.5%

R-Sq(adj) 61.0%

Experimental RPN Condition vs SCORE Response when Exp = 25logten(Score) = - 0.3668 + 0.8951 logten(RPN)

39

Discussion of Results

• Research results indicate that an individual's perception of

risk is influenced differently by each of the three components

utilized by the FMEA model.

• In the application of the model, risk analysis is completed in a

group setting where all aspects of the system can be

discussed and a consensus decision of risk can be made.

• Using the team-based approach may cause the resulting

decisions to be normalized with respect to risk perception,

personal/experience, and cultural bias.

• Based on the results of participants with no experience, it is

important to pair new team members with strong team

leaders and/or team facilitators to provide training and offer

mentoring through the process.

40

Discussion of Results

• Overall, these experimental findings confirm that the RPN does not accurately reflect a person’s innate perception of risk.

• Significant findings show that survey participants assessed risk in a dissimilar fashion when the same situation was presented within the context of the FMEA structure.

• The use of the FMEA form allows people to visualize the components of the situation and make informed decisions of the associated risk based on the severity, occurrence, and detection.

• The FMEA form included in the survey did not follow the exact format of the FMEA. For purposes of this investigation, the failure mode, effect, and detection methods were organized in a one-to-one ratio.

• The concern is that low planned RPN values not requiring further action were perceived as high risk when assessed without the FMEA.

41

Future Study – Action Point Method

• An action point method will need to be established as a

bivariate or univariate analysis of Severity, Occurrence,

and Detection, where the Severity and Occurrence

interaction can be incorporated into the calculation.

• An alternate action point method could utilize a

calculation where only Severity and Occurrence

determine if risk reduction activities are required.

42

Future Study

• An interesting factor to study with a group dynamic experiment would be the management of strong personalities and how they influence the decision- making process.

• Another interesting aspect of the group dynamic experiment would be results of the FMEA analysis when conducted with and without a team facilitator.

• It would be of significant interest to study the way that severity impacts occurrence through the group dynamic, when compared to the individuals’ results of this study.

43

Future Study

• In 2005, Kin Fai Ellick Wong published the results of a psychological study, which found that the differences between eastern and western cultures significantly impact the way a person perceives risk.

• Based on the findings of Wong (2005), executing this study with eastern participants would provide important information regarding the consistency of the FMEA tool across the globe.

• As more companies continue to expand their manufacturing capabilities across the global market, risk analysis activities that are conducted at international facilities by local personnel can become a significant challenge to manage.

• In this situation, variations in risk perception and the risk reduction activities could become inconsistent from one facility to another.

44

fmea matrix

Documents

fmea standards

fmea form

use of fmea

basis of fmea

fmea origins

minor effect

potential failures

failure modes of products