utilizing process capability analyses and gauge r&r studies to … · 2019-04-04 · utilizing...

Utilizing Process Capability Analyses and Gauge R&R Studies to

Improve Your Processes

ASQ 1106 Section MeetingApril 21, 2011

Kevin White, Eastman Chemical Company

Capability Metrics –Evaluation and Recommendations



Tonight’s agenda

3

Time Topic5:30 – 6:15pm Capability Metrics6:15 – 6:45pm Dinner6:45 – 7:30pm Gauge R&R (Measurement System) Metrics7:30 – 7:45pm Break7:45 – 8:30pm Effective Use of Capability and Gauge R&R Metrics

Presentation outline

Variation, short-term and long-termProcess Stability IndexProcess Capability IndicesProcess Performance IndicesNormality, data editing, sample size, one-sided specsLess common indices and six sigma metrics

4

Two types of variation

5

Common cause• Noise• Predictable• Routine• No assignable cause• Expected

Special cause• Signal• Not predictable• Exceptional• Assignable cause• Unusual

Control charts and control limits

6

The control chart is used to distinguish between common and special causesThe control limits represent the expected variation due to common cause

Stable versus unstable processes

7

A stable (in control) process is one in which the key process responses and product properties show no signs of special causesAn unstable (out of control) process exhibits both common and special causes

Visualizing variation in the “x-bar world”

8

••••••••••••

•••• ••••••••

•••• ••

•• ••••

•••• ••••••••

Time

Short-Term

Mea

sure

men

t

Long-Term

•••• •••• ••••••••

••••

••••••••

•••• ••••

•••••••• ••••

Time

Short-Term

Mea

sure

men

t

Long-Term

••••••••

•••• ••••••••

••••

•••• ••••

••••

•••••••• ••••

Time

Short-Term

Mea

sure

men

t

Long-Term

••••••••••••

•••••••••••• •••• •••• ••

•• •••••••• ••••

Time

Short-Term

Mea

sure

men

t Long-Term

Visualizing variation in the “individual world”

9

•• •• ••

•••• ••

••

•• ••

••••••Time

Short-Term

Mea

sure

men

t

Long-Term

•••• •• •

• •• •• •• •• ••••••••Time

Short-Term

Mea

sure

men

t

Long-Term

Time

Short-Term

Mea

sure

men

t Long-Term

•••• ••

•••• ••••

•• ••

••••••

Time

Short-Term

Mea

sure

men

t

Long-Term

•

•

•• ••

•

••• ••••

•

• ••

•

•••••

Short-term versus long-term variation

10

Short-term (σST)• Represents the process

capability• Captures variation due to

common causes• Measures the variation

within subgroups or between successive values

• Used for calculating control chart limits

Long-term (σLT)• Represents the total

process variation• Captures variation due to

common and special causes

• Measures the variation in all the data

• Should not be used for calculating control chart limits

Short-term versus long-term variation

11

Short-term (σST) Long-term (σLT)

1

)(1

2

−

−∑=

n

xxn

ii

2dR

4cs

2dMR

Stability Index

12

For a stable process, you would expect values near 1For an unstable process, you would expect values > 1Rules of thumb• < 1.33 – good process stability• 1.33 - 1.67 – marginal process stability• > 1.67 – poor process stability

ST

LT Index Stability σσ=

How much data do I need?

13

Short-term and long-term standard deviations can be calculated on any dataset• 10 observations or 1000 observations• Over one week or over one year

A long-term standard deviation calculated using 10 observations collected over a one week period is not long-termThe amount of time covered is as important as the amount of data

Switching gears to process capability

14

When talking process stability, we were comparing the process variation to the control limitsWhen talking process capability, we will be comparing the process variation (and location) to the specification limitsSpecification limits or tolerance• Voice of the customer• Usually stated as a LSL and USL• Some cases will have only a LSL or USL (one-sided spec)

In other words…

15

Process capability index, Cp

16

Average not in formulaOnly considers short-term variabilityMust have both LSL and USL to calculateRules of thumb?

Cp =USL - LSL

6σST

LSL USL

What if Process is Off-Target?

17

Process capability index, Cpk

18

Still only considers short-term variabilityCan be used for one-sided specificationsIf on-target, Cp ≈ Cpk

LSL USL

Cpk =Min (USL - avg, avg - LSL)

3σST

Process performance index, Pp

19

Average not in formulaBut considers long-term variabilityMust have both LSL and USL to calculateIf stable, Cp ≈ Pp

Pp =USL - LSL

6σLT

LSL USL

Process performance index, Ppk

20

Considers average and long-term variabilityCan be used for one-sided specificationsCp ≈ Cpk ≈ Pp ≈ Ppk if on-target and stable

LSL USL

Ppk =Min (USL - avg, avg - LSL)

3σLT

Summary of indices

21

Cp is the best indicator of potential process capability because it assumes a stable and on-target processCpk is also an indicator of potential process capability if the process is stable. It does take into consideration if the process is off-target.Pp is an indicator of actual process performance if the process is on-target. It does take into account the long-term variability.Ppk is the best indicator of actual process performance because it considers the process average and long-term variability

Some additional interpretation of the indices

22

Cp / Pp = Stability IndexCpk / Ppk = Stability Index

3(Cp-Cpk) = The number of short-term standard deviations the average is from target3(Pp-Ppk) = The number of long-term standard deviations the average is from target

ST

LTLT

ST

LT

ST

P

P

σσ

LSLUSL6σ

6σLSLUSL

6σLSLUSL

6σLSLUSL

PC =

−×−=−

−

=

Interpretation – example #1

23

Cp = 1.64Cpk = 1.63Pp = 1.61Ppk = 1.60


24

Cp = 1.34Cpk = 0.67Pp = 1.36Ppk = 0.68


25

Cp = 1.32Cpk = 1.07Pp = 0.75Ppk = 0.62

Normality

26

The indices discussed assume the individual data points are from a normal distribution• Check normality – histogram or normal probability plot• If not normal, why?

Unstable processUp against a physical boundary (yield, impurities)Time oriented data like time between failures (exponential)

• If underlying process is truly non-normal, a data transformation may be appropriate

• Calculate indices after successful transformationBe sure to transform the specifications too

Data editing

27

If you have an unstable process, consider editing data to estimate the short-term variability• Involves editing of ranges or moving ranges• Do not edit more than 5-10% of the values• One possible edit – make a one-pass edit excluding ranges

above the UCL

Do not edit data when estimating the long-term standard deviation• One possible exception is to have a “blunder” edit

Use good judgment

Sample Size

28

These indices are statistics just like the average and standard deviation and have variation associated with their estimationCommon software packages will provide confidence intervals for these indicesA simulation was performed with underlying Pp = 1.51000 Pp’s calculated based on sample sizes of 10, 30, 50, 100

Sample size – simulation summary

29

Sample size 5th percentile 95th percentile10 1.10 2.5030 1.25 1.9050 1.28 1.81100 1.35 1.70

Six Sigma metrics (Sigma Level, PPM Defective)

30

A Sigma Level of 6 corresponds to 3.4 ppm defective• Like having a Cp=2 (short-term)• Even with 1.5σ shift, would only have 3.4 ppm defective

Tables available to convert ppm levels to Sigma levelsCommon software will also produce these calculations (Sigma Quality, Z.Bench, etc)• Need to be careful though, some incorporate 1.5σ shift, some

don’t

Less common indices

31

Cpm =USL - LSL

6 σST2 + (avg-tgt)2

Cr =6σST

USL - LSL=

Cp

1

K =Average - Target0.5(USL - LSL)

Summary

32

Cp, Cpk, Pp, Ppk very useful indices for:• understanding process performance• Guiding future improvement efforts

Need to consider normality, data editing, sample sizeOther similar indices and metrics exist

Questions?

Tonight’s agenda

34


Gauge R&R Metrics –Evaluation and Recommendations



Tonight’s agenda

36



Introduction to…• Measurement systems analysis• Gauge repeatability and reproducibility (R&R) studies

Common gauge R&R metricsRelationships between the metrics

37

Issues with measurement systems

Bias (accuracy)LinearityStabilityPrecision (repeatability, reproducibility)

38

Precision

The focus of this presentation is on the precision aspect of measurement systems and the metrics often calculated from a gauge repeatability and reproducibility (GR&R) study• Repeatability – the variability from the gauge or test instrument

when used with the same operator or set-up to measure the same unit

• Reproducibility – the variability arising from different operators, set-ups, instruments, or time periods

39

Measurement systems analysis

Measurement Systems Analysis (MSA) occurs early in the Six Sigma DMAIC RoadmapThe Gauge R&R study is the primary tool taught to Belts• Quantify how much of the observed variability is real process

variation and how much is due to the measurement system• Isolate sources of variability in the measurement system

(repeatability and reproducibility)• Quantify amount of specification range consumed by the

measurement system

Overall, determine if measurement system is adequate for use in the broader project or application

40

Typical gauge R&R study

Ten (10) samplesThree (3) operatorsEach operator measures each sample three (3) times

41

Operator 1 Operator 2 Operator 3

Sample 1 2 3 1 2 3 1 2 3 1 11.0 10.8 11.0 … 2 11.4 11.3 11.5 … 3 10.2 10.3 10.3 … 4 11.4 11.5 11.4 … 5 10.0 10.2 10.1 … 6 M M M 7 8 9 10

The basic model

The total observed variation is equal to the real process variation plus the variation due to the measurement system

42

222msprocesstotal σσσ +=

22rpdrpt σσ +

Measurement system metrics

Some compare the measurement system variation to the total variation of the process• %GRR• Intraclass correlation coefficient (ρ)• ndc• DR

Some compare the measurement system variation to the specification limits• Precision-to-Tolerance Ratio (PTR)

43

%GRR

Addresses the "percent" of total variation taken up by the measurement system

Traditional guidelines state• <10% - Acceptable• 10 to 30% - May be acceptable• >30% - Not acceptable

44

100%GRR ×=total

ms

σσ

%GRR – a true percent of total variation?

Recall..

And…

A true percent would be…

45

222msprocesstotal σσσ +=

msprocesstotal σσσ +≠

100%MS 2

2

×=total

ms

σσ

%GRR versus %MS

46

0

10

20

30

40

50

60

70

80

90

100

%M

S

0 10 20 30 40 50 60 70 80 90 100%GRR

%GRR makes the measurement system appear

worse than reality

Intraclass correlation coefficient (ρ)

A traditional measure of association used to characterize the relative usefulness of a measurement systemProportion of variation due to process

Wheeler (EMP III)• First class monitors (>0.8)• Second class monitors (0.5 to 0.8)• Third class monitors (0.2 to 0.5)• Fourth class monitors (<0.2)

47

2

2

2

2

1total

ms

total

process

σσ

σσ

ρ −==

Number of distinct categories (ndc)

Number of non-overlapping 97% confidence intervals that will span the expected variationNumber of distinct categories a measurement system can readily indentify

Value should be truncated to the integer and ought to be greater than 5

48

ms

process

σσ

×= 41.1ndc

Discrimination Ratio (DR)

49

Very similar to ndcThe number of product categories which could be established within the natural process limits using these measurements (Wheeler, EMP 2nd Edition)

For simple measurements, should be > 4 (Wheeler)Improve measurement if < 2 (Wheeler)

1ndc1211D 2

2

2

R +=−=−+=

ms

total

σσ

ρρ

Relationships between metrics

50

100%GRR%MS

2

=

1%GRR

00141.1ndc2

−⎟⎟⎠

⎞⎜⎜⎝

⎛×=

2

100%GRR1 ⎟

⎠⎞

⎜⎝⎛−=ρ

1%

1002D2

R −⎟⎠⎞

⎜⎝⎛=

GRR

All of these metrics can be described as a function of %GRR

%GRR versus %MS and ICC

51

%M

S

ICC

%GRR versus ndc and DR

52

0123456789

10

ndc

/ DR

0 10 20 30 40 50 60 70 80 90 100%GRR

DR

ndc

Precision-to-Tolerance Ratio (PTR)

Addresses the percent of tolerance consumed by the measurement system

Normally, k=5.15 or k=6Traditional guidelines state• <10% - Acceptable• 10 to 30% - May be acceptable• >30% - Not acceptable

53

100Tolerance

kPTR ms ×= σ

Summary

54

There are many metrics that compare the measurement variation to the total variationBecause of their relationships, using more than one is redundantSome of the guidelines are a little contradictoryPTR is different because it compares the measurement variation to the tolerancePart of the next session is going to examine the relationship between Cp, %GRR, and PTR and propose new guidelines for the measurement system

Questions?

Tonight’s agenda

56


Effective Use of Capability and Gauge R&R Metrics



Tonight’s agenda

58



Recommended set of process capability metrics with guidelinesApproach for using the capability metric guidelines to identify process improvement opportunitiesRecommended set of measurement system metrics with guidelinesRelationship between process capability and measurement system metricsTwo-dimensional guidelines for measurement system metrics

59

Index overload

60

PpPpk

Cpk

Cp

Stability IndexσLT

σST

LSLUSL

ppmCpm

Cr

K

%GRR

%MS

ndc

DRPTR

ρ

The goal

61

The goal is to have processes that are:• Stable (in-control)• On-target• Within specifications• With adequate (capable)

measurement systems

Which indices can help us assess our performance against this goal?

LSL USL

Ppk – assessing actual performance (start here)

62

Ppk is the best indicator of actual process performance because it considers the process average and long-term variability

LTpk 3

LSL)x,xmin(USLPσ

−−=

Ppk guidelines

63

RED

YELLOW

GREEN

POOR

MARGINAL

ADEQUATE

Ppk

1.0

1.5

Stability index – assessing stability

64

Look at ratio of long-term variability to short-term variability

Note stability index is also same as Cp/Pp or Cpk/Ppk

ST

LT Index Stability σσ=

Stability index guidelines

65

RED

YELLOW

GREEN

POOR

MARGINAL

ADEQUATE

StabilityIndex

1.67

1.33

Target index – assessing ability to hit target

66

The target index is defined as the number of short-term standard deviations the process average is from the target

Note this is the same as 3*(Cp-Cpk)

ST

TxIndex Target

σ−

=

Target index guidelines

67

RED

YELLOW

GREEN

POOR

MARGINAL

ADEQUATE

TargetIndex

1.0

0.5

Cp – assessing potential performance

68

Cp is the best indicator of potential process capability because it assumes a stable and on-target process

STp 6

LSLUSLCσ−=

Cp guidelines

69

RED

YELLOW

GREEN

POOR

MARGINAL

ADEQUATE

Cp

1.0

1.5

Using the indices (flowchart)

70

PpkActual

Performance

Adequate

Marginalor Poor

Likely,do nothing

CpCommon cause

too large?

Target IndexOff target?

Stability Indexunstable?

Improve Stability(special causes)

Yes

Work on Level(Average)Yes

Improve Capability(common cause)

Yes

Routine Reporting

71

It is recommended that a routine report or scorecard (monthly, quarterly, bi-annual) be created that shows this information for all key processes and responsesWith some simple color coding, it can easily help identify improvement opportunities• What to work on• And the type of effort that it will involve

Example scorecard #1

72

1st Quarter 2010Process 1 Ppk Cp Stability Index Target Index

Response 1 1.67 1.67 1.00 0.00Response 2 2.78 8.33 3.00 0.00Response 3 0.95 1.33 1.40 0.00Response 4 0.33 1.67 2.55 2.50Response 5 1.59 2.38 1.00 2.38

Process 2 Ppk Cp Stability Index Target IndexResponse 1 1.11 1.11 1.00 0.00Response 2 2.08 2.78 1.33 0.00Response 3 0.62 1.33 1.40 1.40

Example scorecard #2

73

Process XYZ, Response ABCQtr/Yr Ppk Cp Stability Index Target Index

3Q 2008 1.52 3.09 1.91 0.564Q 2008 1.63 2.82 1.66 0.341Q 2009 1.45 3.09 2.04 0.372Q 2009 1.65 3.40 2.02 0.203Q 2009 1.75 3.21 1.83 0.004Q 2009 1.61 2.92 1.67 0.701Q 2010 1.49 3.03 1.91 0.55

Plots of indices over time can also be useful

Improving unstable processes

74

Involves finding and fixing special causesControl charts used in conjunction with control strategiesPotential special causes• Lack of standardization• Lack of consistent process conditions

Uncontrolled process variablesUnknown process variablesEquipment malfunctions

• Unstable measurement system• Changes in process inputs• Management intervention

Improving off-target processes

75

Actions based on:• Process understanding• Engineering theory• Design of Experiments• EVOP Experiments

Potential fixes• Raw material quality level• Change in operating conditions• Modify or change equipment• Improved measurement accuracy

USLLSL

Improving (reducing) common cause variation

76

Actions based on:• Process understanding• Engineering theory• Process variation studies• Design of Experiments• Measurement systems analysis• Stratification

Potential fixes• Improved measurement precision• Standardization• Engineering process control (EPC)• Change in operating conditions

USLLSL

What about the measurement system?

77

Does the measurement system need to be improved…• When the process is unstable?

So long as the measurement system is stable and you’re able to see instability in the process, work on the process instability

• When the process is off-target?Possibly, it may be hard to detect improvements if there is large amount of measurement variabilityReplicate testing to reduce measurement variation is also something to consider

• When the common cause variability is too large?Very possibleWill depend on the measurement system metrics

Measurement system metrics

78

%GRR• Compares the measurement system variation to the total

variation

PTR (Precision-to-Tolerance Ratio)• Compares the measurement system variation to the specification

range (tolerance)

Other metrics exist, but as you saw in the previous session, they are very related

100%GRR ×=total

ms

σσ

100Tolerance

kPTR ms ×= σ

Measurement system metric guidelines

79

RED

YELLOW

GREEN

POOR

MARGINAL

ADEQUATE

%GRR PTR

30% 30%

10% 10%

One problem with %GRR

80

%GRR looks at a ratio of standard deviations (measurement/total)Since standard deviations are not additive, %GRR is not a true percent of the total variationGuidelines may not be realistic for some measurement systems

0

10

20

30

40

50

60

70

80

90

100

%M

S

0 10 20 30 40 50 60 70 80 90 100%GRR

Percent Cp improvement with measurement improvement

81

Plot shows %improvement in Cp for

given %reductions in the measurement standard deviation

Relationship between capability and measurement system metrics

82

%GRR PTR Cp

σmsX X

σSTX X

Tolerance X X

If you know two of the metrics, then you can figure out the third without knowing the individual components.

Graphical depiction of relationship

83

0

10

20

30

40

50

60

70

80

90

100

P/T

Rat

io

0 10 20 30 40 50 60 70 80 90 100% Gage R&R

Cp=1.0

Cp=0.5

Cp=2.0

Cp=1.5

%GRR

PTR

Note: k=5.15 in PTR

calculation

Applying the measurement metric guidelines

84

0

10

20

30

40

50

60

70

80

90

100

P/T

Rat

io

0 10 20 30 40 50 60 70 80 90 100% Gage R&R

Cp=1.0

Cp=0.5

Cp=2.0

Cp=1.5

%GRR

PTR

Two-dimensional guidelines

85

0

10

20

30

40

50

60

70

80

90

100P

/T R

atio

0 10 20 30 40 50 60 70 80 90 100% Gage R&R

Cp=1.0

Cp=0.5

Cp=2.0

Cp=1.5

Sigma Level > 6

Improving process variation will help,but the measurement systemwill still need to be improved

(BOTH)

Meas. sys. is adequate,focus on process(PROCESS)

Focus on process variation first,measurement system may become

limiting after some level ofprocess variation improvement

(PROCESS)

The measurementsystem is primary

limitation on furtherprocess capability

Improvement(TEST)

Work on bothprocess andmeas. sys.

(BOTH)

%GRR

PTR

Example

86

0

10

20

30

40

50

60

70

80

90

100P

/T R

atio

0 10 20 30 40 50 60 70 80 90 100% Gage R&R

Cp=1.0

Cp=0.5

Cp=2.0

Cp=1.5

%GRR

PTR

Summary Comments

87

A limited number of metrics are all you need and will indicate process improvement opportunities• Process instability• Off-target• Excessive common cause variation• Inadequate measurement system

If you prefer another metric, you can probably substitute for one of the ones shown• Example: use ndc instead of %GRR

May want to tweak the guidelines (rules of thumb) for your specific industry

Questions?

utilizing process capability analyses and gauge r&r studies to … · 2019-04-04 · utilizing...

Documents