javier garcia - verdugo sanchez - six sigma training - w2 measurement system analysis

Measurement System Analysis Part 3: Continuous Measurements

Week 2

Knorr-Bremse Group

About this Module

Based on this technique you can assess and judge t t h b tt d ib d imeasurement systems much better as described in

the ISO 9000 standard.

• Part 1: Introduction of Measurement System Analysis– Concept definition and describing the basic termsConcept definition and describing the basic terms

• Part 2: Attributive Measurements– Kappa Analysis

• Part 3: Continuous Measurements– The Gage R&R Study

• Some exercises

Knorr-Bremse Group 05 BB W1 Cont. Measurement 08, D. Szemkus/H. Winkler Page 2/36

The DMAIC Cycle

ControlMaintain Improvements

SPC

DefineProject charterSPC

Control PlansDocumentation

Project charter (SMART)

Business Score CardQFD + VOC

D QFD VOC

Strategic GoalsProject strategy

C M

Measure

Baseline AnalysisImprove

AIBaseline Analysis

Process MapC + E Matrix

M tAnalyze

Definition of critical

pAdjustment to the

OptimumFMEA

Measurement System

Process Capability

Definition of critical InputsFMEA

Statistical Tests

Statistical TestsSimulationTolerancing

Process CapabilityStatistical TestsMulti-Vari Studies

Regression


Effects of Measurement Error

Measurement System

AverageBias -Determined through

“Calibration Study”

Accuracy

µ µ µtotal product measurement= +p

V i bilit

Measurement System Variability - Determined through “R&R Study”

Variability Precision

222 222tmeasuremenproducttotal σσσ +=


Index - P/T (Precision to Tolerance)

Usually expressed as percent100

6/ ×

×=TP MSσ

percent100/ ×=Tolerance

TP

• This value addresses, what percent of the Tolerance is taken by the measurement error (Precision).taken by the measurement error (Precision).

• It includes both repeatability and reproducibility.

• P/T < 10% Measurement system excellent

• P/T < 20% Measurement system acceptableP/T 20% Measurement system acceptable

• P/T < 30% Measurement system marginal acceptable

Note: 6 standard deviations accounts for 99,73 % of MS variation. 5.15 standard deviations accounts for 99% of MS variation (Industry standard)


standard deviations accounts for 99% of MS variation. (Industry standard).

Relation Precision to Tolerance

P/TProduct toleranceLSL USL

V i i f h

Product toleranceLSL USL

Variation of the measurement system

P/T = 20%

system

P/T = 100%

P/T = 200%


Use and Interpretation of P/T

The P/T-Ratio is the most often used evaluation for judging the precision of a measurement system.

– This is telling us how good the the system is working regarding the specification.

– But specifications might be to wide or to small

I h f j d i d i– In the area of judging a product to an important customer specification the P/T-Ratio is the best evaluation.

– The measurement error σMS includes 2 components

• Repeatability – Variation caused by the measurement devicep y y

• Reproducibility – Variation caused by the operator

– Regarding process capability and process improvement the single use of the P/T-Ratio can create a wrong feeling of safety


safety.

Index %R&R (Precision to Total Variation)

100&% ×= MSRRσ Usually expressed as

percent100&% ×=Total

RRσ

percent

• This value addresses, what percent of the total variation is taken by the measurement error (Precision).y ( )

• It includes both repeatability and reproducibility.

• P/T < 10% Measurement system excellent

• P/T < 20% Measurement system acceptabley p

• P/T < 30% Measurement system marginal acceptable

%R&R is the best possibility for a Black Belt, to asses the capability of measurement systems. Based on the %R&R we can decide where to


concentrate for improvements. Process or measurement system

Repeatability & Reproducibility

%R&R

Ob d i ti

%R&R = 20%

Observed process variation

Variation of the measurementmeasurement system

%R&R = 75%

%R&R = 100%


Use and Interpretation of %R&R

%R&R is the best tool for estimating the capability of a measurement system during process improvement activities.

– The %R&R – ratio estimates the capability of the measurement system in relation to the total process variationmeasurement system in relation to the total process variation. That means, it gives us the information how good the real process / product variation can be identified.

– The %R&R includes 2 components

• Repeatability – Variation caused by the measurement device

R d ibili V i i d b h• Reproducibility – Variation caused by the operator

– The use of statistical tools (and their conclusion) can beThe use of statistical tools (and their conclusion) can be influenced by high %R&R values. Therefore, the capability of the used measurement system should be checked in d


advance.

Comparison of P/T and %R&R


Ob d i tiObserved process variation

%R&R = 20%P/T = 20%

%R&R = 50%P/T = 50%

%R&R = 100%P/T = 100% %R&R = 100%P/T = 100%


Measurement system variation



Ob d i tiObserved process variation

P/T = 50% %R&R = 25%

P/T = 100% %R&R = 50%

P/T = 200% %R&R = 100%P/T = 200% %R&R = 100%





Observed process variationprocess variation

P/T = 10% %R&R = 20%

P/T = 20% %R&R = 40%

P/T = 50% %R&R = 100%P/T = 50% %R&R = 100%



Relation % R&R - Cp

A simplified illustration of the relation % R&R and Cp

Th l ti f h th f ll i lThe evaluation of a process shows the following values:

Mean = 204,33 with a StDev. = 2,31

The specification limits are LSL = 194 and USL = 210

This is equivalent to a process capability of 3 Sigma q p p y gwith a cp = 1,16 and a cpk = 0.82 The evaluation of the measurement

system results in a R&R = 54,67 %

If the portion of the measurement system variation can be reduced, the overall capability can be predicted as follows:

Gage R&R cp cpk54,67 1,16 0,82

30 1,30 0,9215 1,36 0,960 1 38 0 98


0 1,38 0,98

Still Other Statistical Indexes

The Signal-to-Noise Ratio (S/N Ratio) relates the product variation to the measurement system variation. The S/N Ratio should be as large as possible.

S / N Ratio =σ PS / N Ratio =σ MS

The Discrimination Index provides the number of divisions that the Measurement System can accurately measure across the part (sample)Measurement System can accurately measure across the part (sample) variation. If this index is less than 5, then it is marginal acceptable for a study. If the index is 2, then it is equivalent to a go/no-go gage. We would like to see a value ≥ 5 (QS 9000 recommendation)like to see a value ≥ 5 (QS 9000 recommendation).

σ p⎛⎜

⎞Discrim =

σ

σp

ms

⎛

⎝⎜⎜

⎞

⎠⎟* .141ndc =


„nnumber of ddistinct ccategories“

Number of Distinct Categories (ndc)• Assesses the sensitivity or effective resolution of a measurement system .

• How many categories of parts can the measurement system distinguish?

ndc Process control For analysis

Appropriate only, if process Not appropriate for the

variation is very small compared to the specification.

Not appropriate for the assessment of process parameter or indices.

<‐‐‐ 1 category ‐‐‐>

Control charts can be used

Not appropriate for the assessment of process

<‐ 2‐4 categories ‐>

Control charts can be used marginal.

parameter or indices.Only rough estimation can be made.catego es

Control charts can be used Recommended for production

<‐ >=5 categories ‐>

without restriction. and improvement activities.


Source: QS-9000 MSA Hand Book

Effects of P/T and S/N Ratios

• The effect P/T on Cpk

– Large P/T reduces the process Cpk from the true value to g p pksome smaller observed value.

• The effect of P/T on misclassification probabilitye e ec o / o sc ass ca o p obab y

– Large P/T increases the probability that we will misclassify product as defective although it’s good and vice versa.g g

• The effect of S/N ratio on control chart sensitivity

Small S/N increases the time before an out of control process– Small S/N increases the time before an out-of-control process is detected by a control chart

• The effect of the ndc (number of distinct categories)• The effect of the ndc (number of distinct categories)

– If the ndc = 2, only attribute data are available and sample sizes must be largersizes must be larger.

– If the ndc is 4 to 10, then discrimination is finer and sample sizes can be smaller


sizes can be smaller.

Gage R & R Study & Sample Size• Usually perform the study with 2 – 3 persons (inspectors, appraiser)

• At least 10 samples (parts/products) are required for a study p (p p ) q y

• Important! Take samples out of the process which covers the normal process variation (± 3 σ). Example: The manufacturing process produces p ( ) p g p pa material with a thickness from 200 +/- 15 µm. Samples should be taken over the range of 185 – 215 µm, independent if the parts are within the specification or not!specification or not!

• Caution! If you manufacture with a certain process different parts (different products) you have to perform a Gage R&R study for each(different products) you have to perform a Gage R&R study for each subgroup.

Example: You manufacture three different products with a thickness inExample: You manufacture three different products with a thickness in the range from 150 – 1000 µm, with different specification windows. All result are checked with the same gage. You should run three studies over the product range as described aboveover the product range as described above.

If you mix these samples the %R&R value will be artificial low, that means artificial good!


means artificial good!

Gage R & R Study & Sample Size• Every sample will be measured by each operator 3 times (3 trials per

operator) in a random order and “blind”.

• The total effort of the measurement system study is depending from the complexity and the costs of the analysis.

• Number of samples

– Select enough samples so that g p

number of samples (S) x number of operators (O) > 15

– If not practical or possible, choose number of trials so that:

• if S x O < 4, trials = 6


if S x O < 8 trials = 4• if S x O < 8, trials = 4



Procedure for Performing an R&R Study1. Use a calibrated gage.

2. Have the first operator measure all the samples in random order once.

3. Have the second operator measure all the samples in random order once.

4. Continue until all operators have measured the samples once (this is trial 1).

5. Repeat steps 1-3 for the required number of trials. Make sure that the operator will not see his own results or from the others operators.

6 U i Mi it b th G R&R ( d) ith th ANOVA M th d f th6. Use in Minitab the Gage R&R (crossed) with the ANOVA Method for the evaluation of measurement system capability.

– Repeatability (Displays the variation of the gage (equipment variation)– Repeatability (Displays the variation of the gage (equipment variation)

– Reproducibility (Displays the variation between the operators)

Standard deviations of each of the above– Standard deviations of each of the above

– %R&R and %P/T

7 A l lt d d t i f ll ti if


7. Analyse results and determine follow-up action, if any.

Example: Thickness Measurement Trial Operator Part Thickness Dicke µm

1 Mary 1 8 203,21 Mary 2 8 203,21 Mary 3 8 203,2 We manufacture a product with a nominal1 Mary 4 8 203,21 Mary 5 8,2 208,281 Mary 6 8 203,21 Mary 7 8,1 205,74

We manufacture a product with a nominal thickness of 200 µm. The specified tolerance is + - 15 µm.

1 Mary 8 8 203,21 Mary 9 8,2 208,281 Mary 10 8,1 205,742 Mary 1 8 203,2

For the measurement of the thickness we use a micrometer. Calibration of the

i t i i d 3 th2 Mary 2 8 203,22 Mary 3 8,1 205,742 Mary 4 7,9 200,662 Mary 5 8,2 208,28

micrometer is required every 3 month.

In order to evaluate the measurement t bilit 10 l h b2 Mary 6 8,1 205,74

2 Mary 7 8,2 208,282 Mary 8 8 203,22 Mary 9 8,1 205,74

system capability, 10 samples have been measured from 3 operator 3 times.

2 Mary 10 8,1 205,743 Mary 1 8 203,23 Mary 2 8 203,23 Mary 3 8,1 205,74 File: Thickness Measurement mtw

The format for the evaluation in Minitab. We need columns for the lt f th l d f th t

File: Thickness Measurement.mtw


results, for the samples and for the operators.

Gage R & R Study EvaluationStat

>Quality tools

>Gage Study

>Gage R&R Study crossed...

Here you can document the informationHere you can document the information regarding study and gage.

Important if you run more studies This window is to tell Minitab the


This window is to tell Minitab the tolerance window.

Gage R&R - ANOVA-Tables

The session window shows 2 ANOVA-tables.

Two-Way ANOVA Table With Interaction

Source DF SS MS F PPart 9 315 412 35 0457 22 3350 0 000

If there is no significant Part 9 315,412 35,0457 22,3350 0,000Operator 2 10,466 5,2330 3,3350 0,059Part * Operator 18 28,244 1,5691 1,0944 0,380Repeatability 60 86,021 1,4337

Operator x Part interaction, Minitab is running a second

Total 89 440,142

Alpha to remove interaction term = 0,25

gANOVA without the interaction term.

Alpha to remove interaction term 0,25

Two-Way ANOVA Table Without Interaction This gives more power to the operator

Source DF SS MS F PPart 9 315,412 35,0457 23,9230 0,000Operator 2 10,466 5,2330 3,5721 0,033

evaluation, because the DF can be distributed more accurate.p , , , ,

Repeatability 78 114,265 1,4649Total 89 440,142

more accurate.


Gage R&R Evaluation

• We are interested in the % Gage R&R for process improvement efforts

Gage R&R

improvement efforts.

• We are interested in the % P/T for acceptance of the Gage R&R

Process tolerance = 30

pmeasurement instrument in

production.

Study Var %Study Var %ToleranceSource StdDev (SD) (6 * SD) (%SV) (SV/Toler)Total Gage R&R 1,26116 7,5670 54,67 25,22Repeatability 1,21035 7,2621 52,47 24,21Repeatability 1,21035 7,2621 52,47 24,21Reproducibility 0,35440 2,1264 15,36 7,09Operator 0,35440 2,1264 15,36 7,09

Part-To-Part 1,93163 11,5898 83,73 38,63T t l V i ti 2 30689 13 8413 100 00 46 14Total Variation 2,30689 13,8413 100,00 46,14

Number of Distinct Categories = 2g

• Attention – watch for the small number of the Distinct Categories! For the use of a process improvement thi b h t b 5 t i i !


this number has to be 5 at minimum!

Graphical Evaluation

Gage name: Reported by : Tolerance:

Gage R&R (ANOVA) for Thickness µm

80 % Contribution

% S d210

Date of study : M isc:

Components of Variation Thickness µm by Part

40

0

Per

cent

% Study Var

% Tolerance

205

200

Part-to-PartReprodRepeatGage R&R0

e

Fred Joe Mary

10987654321Part

210

R Chart by OperatorThickness µm by Operator

4

2

0

Sam

ple

Ran

ge

_R=1,439

UCL=3,705

LCL=0MJF d

205

200

208

an

Fred Joe Mary

MaryJoeFredOperator

208Fred

Operator

Xbar Chart by Operator Operator * Part Interaction

204

200

Sam

ple

Mea __

X=204,329

UCL=205,801

LCL=202,856

10987654321

204

200

Ave

rageFred

Joe

Mary


200 10 98 76 54 32 1Part

Graphical Evaluation Control Chart

Reported by : Tolerance:

Gage R&R (ANOVA) for Thickness µm

• The Range Chart shows the effects in gradual resolution.

80 % Contribution

% S d210

Misc:


g g

• We like to see a minimum of 3 distinctive levels within the limits

40

0

Per

cent

% Study Var

% Tolerance

205

200• Based on the high samplePart-to-PartReprodRepeatGage R&R

0

e

Fred Joe MaryFred Joe Mary

10987654321Part

210


Based on the high sample size the X-bar chart should show points out of the limits.

4

2

0

Sam

ple

Ran

ge

_R=1,439

UCL=3,705

LCL=0

205

200

• This is indicating the part to part variation.

208

an

Fred Joe MaryFred Joe Mary

MaryJoeFredOperator

208Fred

Operator


p

• Only a few points out of the limits indicates that not the

204

200

Sam

ple

Mea __

X=204,329

UCL=205,801

LCL=202,856

10987654321

204

200

Ave

rage

FredJoe

Mary

limits indicates that not the whole variation is assessed.


200 10 98 76 54 32 1Part

Interpretation of the Average (x-bar) Chart

• If the average value for each operator is different, we may have a problem with the reproducibility.have a problem with the reproducibility.

• We want, that more average values are outside of the control limits but for all operators equally! This indicates more part to part variation which we like to see in a measurement system studymeasurement system study.

• The major portion of the dots shall be outside the controlThe major portion of the dots shall be outside the control limits!

– If this is the case and if the R-chart is in control, it indicates a capable measurement system. In addition we will determine how much variation (%) of the total variation is caused by thehow much variation (%) of the total variation is caused by the measurement system.


Interpretation of the Range Chart

• You assume insufficient resolution of the measurement system if:system if:

– the R-chart shows less than 3 distinctive levels within th t l li itthe control limits

– more than 4 levels for the Range available, but more g ,than 1/4 of the values are 0

• The repetition precision is questionable if the R chart• The repetition precision is questionable, if the R-chart displays events outside the control limits.

• If the range of one operator is out of control, but not for the others, than the method is questionable

• If the range for all operators out of control the system is sensitive against the methodology of the operators


sensitive against the methodology of the operators

Interpretation of the Variation Causes

Gage name: Reported by : Tolerance:

Gage R&R (ANOVA) for Thickness µm• This bar chart helps to understand the variation blocks of the study.

• Because of the bad resolution it is recommended to look to the numbers

80 % Contribution 210

Gage name: Date of study :

Tolerance: M isc:


in the session window.

80

40

Per

cent

% Study Var

% Tolerance

210

205

200

Part-to-PartReprodRepeatGage R&R0

Fred Joe Mary

10987654321200

Part

210


E h t i t d b4

2

0

Sam

ple

Ran

ge

_R=1,439

UCL=3,705

LCL=0

210

205

200

• Each operator is represented by one line.

• Crossing lines are indicating a Interpretation of the Interaction0 LCL=0

208

n

Fred Joe Mary

MaryJoeFred200

Operator

208 Operator


• Crossing lines are indicating a significant interaction.

• We like to see parallel lines for all

Interpretation of the Interaction

204

Sam

ple

Mea

n

__X=204,329

UCL=205,801

LCL=202,856

204

200

Ave

rage

Fred

Joe

Mary

We like to see parallel lines for all operators.

• Significant interactions need to be


200 10 98 76 54 32 1Part

ginvestigated and eliminated.

The Gage Run Chart

A nice tool to understand Operator-Sample-Trial-Interaction is the Gage Run Chart.

Stat

>Quality tools

>Gage Study>Gage Study

>Gage Run Chart...


The Gage R&R Run Chart

Gage name:Date of study :

Reported by :Tolerance:Misc:

Gage Run Chart of Thickness µm by Part, Operator

1 2 3 4 5 O t

Date of study : Misc:

210

207

1 2 3 4 5 O perator

Mary

F redJoe

ne

ss µ

m

Mean 204

201

6 7 8 9 10

Th

ick

n

210

207

6 7 8 9 10

204

201

Mean

Operator

Panel variable: Part

This diagram shows the mean for each sample by “Trial by Operator". Watch


This diagram shows the mean for each sample by Trial by Operator . Watch for anomalies! – we want to see flat lines at equal values for all operators!

Create a Gage R&R Study Worksheet

Stat

>Quality tools

>Gage Study

>Create Gage R&R Study Worksheet...


Gage Study: Example

After the defined actions had been completed, a new gage study has been conducted.

File: New Thickness.mtw

Please evaluate the study individually.

- How are the results differing?

- Interpret the graphical evaluations.


Investigation Report Gage R&R Study

Every measurement system study shall be documented together with the results The following information shalltogether with the results. The following information shall be included:

T t– Target

– Description of the gage / the measurement system

– Process- / product description and specification

P d t f th t d– Procedure to perform the study

– Results:

• Calibration, Accuracy, Precision, P/T and % R&R

C l i– Conclusions

– Recommendations for improvements


Measurement System Analysis Questions

• Have you picked the right measurement system? Is this measurement system associated with either critical inputs or outputs?

• What do the precision, accuracy, tolerance, P/T ratio, %R&R and trend chart look like?

• What are the sources of variation and what is the measurement error?

• What needs to be done to improve this system?What needs to be done to improve this system?

• Have we informed the right people of our results?

• Who owns this measurement system?

• Who owns trouble shooting?g

• Does this system have a control plan in place?

• What’s the calibration frequency? Is that frequent enough?

• Do identical systems match?


Summary

• It is very important to know about the measurement system BEFORE we start process improvement activities.

• Be careful in the study planning process and develop an adequate sample plan.

• Analyse the measurement system regarding the effects of operators, parts and trials.

• Take care that the measurement system has a good discrimination.

• Run always a study to document the status for variation caused byRun always a study to document the status for variation caused by repeatability and reproducibility.

• Measuring is a process itself It covers more than just the gage Try toMeasuring is a process itself. It covers more than just the gage. Try to understand the reasons for variation of the process and minimise it.


javier garcia - verdugo sanchez - six sigma training - w2 measurement system analysis

Engineering