Download - Measurement systems analysis v1.1
1© 2001 Six Sigma Academy
Measurement Systems Analysis (MSA)
2© 2001 Six Sigma Academy
Why Measure?
• To understand a decision:• Meet standards & specifications• Detection/reaction oriented• Short-term results
• Stimulate continuous improvement:• Where to improve?• How much to improve?• Is improvement cost effective?• Prevention oriented• Long-term strategy
“If you cannot measure, you cannot improve!”– Taguchi
3© 2001 Six Sigma Academy
Measurement System As A Process
Cleanliness
Temperature
Dimension
Weight
Corrosion
Hardness
Conductivity
Density
Sequence
Timing
Positioning
Location
Set-up
Preparation
Cleanliness
Temperature
Design
Precision
Calibration
Resolution
Stability
Wear
Cleanliness
Vibration
Atmospheric pressure
Lighting
Temperature
Humidity
Compliance-procedureFatigue
AttentionCalculation errorInterpretation
SpeedCoordination
Vision
Knowledge-instrumentDexterity
PeopleEnvironment
MeasurementError
MethodMaterial Machine
4© 2001 Six Sigma Academy
What Is An MSA?
Scientific and objective method of analyzing the validity of a measurement system
• A “tool” which quantifies:
1. Equipment Variation
2. Appraiser (Operator) Variation
3. The Total Variation of a Measurement System• MSA is NOT just Calibration• MSA is NOT just Gage Repeatability & Reproducibility (R&R)
Measurement System Analysis is often a “project within a project”
5© 2001 Six Sigma Academy
MSA Relationship To DMAIC
Measurement Systems Analysis• Quantitative evaluation of tools and processes used in making
discrete or variable observations
Measurement Systems Control • Established, documented, and continuously carried out • Ensures measurement system maintains an acceptable status • Often referred to as “Long Term Gage Plan”
Define Improve ControlMeasure Analyze
Define Improve ControlMeasure Analyze
6© 2001 Six Sigma Academy
MSA - A Starting Point
Before you…• Make adjustments• Implement solutions• Run an experiment• Perform a complex statistical analysis
You should…• Validate your measurement systems• Validate data and data collection systems
MSA quantifies a major source of process variation
7© 2001 Six Sigma Academy
Measurement Systems
• Examples• Precision gage• Data collection form• Survey• School entrance exam• Customer satisfaction• On-time delivery report
What is your system ?
8© 2001 Six Sigma Academy
Types of Measurement System Analysis
• Operational Definitions• Walking the Process• Gage R&R
• Variable Data• Attribute Data
9© 2001 Six Sigma Academy
MSA – Operational Definitions
The Measurement System can be validated using Operational Definitions
constructed by the Project Team to ensure that all measurement takers completely understand what is expected during the data
collection phase.
10© 2001 Six Sigma Academy
Developing Operational Definitions
•Operational definitions are descriptions written in a way that ensures consistent interpretation by different people
•The operational definition method of description will be used throughout the DMAIC process
11© 2001 Six Sigma Academy
•Operational Definition• The technique of defining an item, process or characteristic using
Operational Definitions is an effective way to communicate between Team Members and other people involved in the project. Because Operational Definitions are so effective, the technique is used in a number of locations within the DMAIC process. Remember, to be effective, an Operation Definition must be written in a way that ensures consistent interpretation by different people.CC
12© 2001 Six Sigma Academy
General Example – Operational Definitions
• Examples of Operational Definitions for data collection:
• Record the date that the lease company written notification arrives in the dealership using an MM/DD/YY format.
• List any cosmetic preparation in excess of the standard pre-delivery process required to render the vehicle acceptable for retail consumer sale.
• Record the weight of each package of coffee in ounces by pouring the coffee into the filter and placing the filter and coffee on the scale tray.
• Record the length of time that coffee remains in the urn by recording the actual time of day each time the Brew button is pressed to recharge the urn. Use 24-hour clock and round to the nearest minute.
13© 2001 Six Sigma Academy
MSA – Walking the Process
“Walking the Process” is a method of conducting MSA when it is not possible
to perform a Gage R&R.
14© 2001 Six Sigma Academy
How to “Walk the Process”
• Develop Operational Definitions for each of the measures to be collected
• Train data collectors prior to beginning the data collection activity• Follow the process from beginning to end and monitor the data
collection activities to determine if data is being collected properly• Continue walking the process until the data compiled accurately
reflects the existing process
15© 2001 Six Sigma Academy
Components Of Measurement Error
16© 2001 Six Sigma Academy
Components Of Measurement Error
• Resolution/Discrimination• Accuracy (bias effects)• Linearity• Stability (consistency)• Repeatability-test-retest (Precision) • Reproducibility (Precision)
Each component of measurement error can contribute to variation, causing wrong decisions to be made
17© 2001 Six Sigma Academy
Categories Of Measurement Error Which Affect Location
Accuracy/ Bias
LinearityStability
18© 2001 Six Sigma Academy
Categories Of Measurement Error Which Affect Spread
RepeatabilityReproducibility
Precision
19© 2001 Six Sigma Academy
Can change be detected?
Resolution/Discrimination
Resolution?
Accuracy/Bias?
Linearity?
Stability?
Precision (R&R)?
OK
OK
OK
OK
20© 2001 Six Sigma Academy
Resolution
• Simplest measurement system problem• Poor resolution is a common issue• Impact is rarely recognized and/or addressed• Easily detected• No special studies are necessary • No “known standards” are needed
21© 2001 Six Sigma Academy
Definitions:
• Resolution/Discrimination• Capability to detect the smallest tolerable changes
• Inadequate Measurement Units• Measurement units too large to detect variation present
• Guideline: “10 Bucket Rule”• Increments in the measurement system should be one-tenth the
product specification or process variation
22© 2001 Six Sigma Academy
Resolution/Discrimination
1 2 3 4 5
Better Discrimination
1 2 3 4 5
Poor Discrimination
Same process output being measured
1.3
1
23© 2001 Six Sigma Academy
Resolution Actions
• Measure to as many decimal places as possible• Use a device that can measure smaller units• Live with it, but document that the problem exists• Larger sample size may overcome problem• Priorities may need to involve other considerations:
• Engineering tolerance• Process Capability• Cost and difficulty in replacing device
24© 2001 Six Sigma Academy
Accuracy/Bias
Measurements are “shifted” from “true” value
Resolution?
Accuracy/Bias?
Linearity?
Stability?
Precision (R&R)?
OK
OK
OK
OK
25© 2001 Six Sigma Academy
Accuracy/Bias
Difference between the observed average value of measurements and the master value
Master Value(Reference Standard)
AverageValueMaster value is an accepted,
traceable reference standard
26© 2001 Six Sigma Academy
Accuracy/Bias
x x
x
xx
x
x
x
x x x
x
xx
x
x
x
x
Less accurateMore accurate
27© 2001 Six Sigma Academy
Accuracy/Bias Actions
• Calibrate when needed/scheduled• Use operations instructions• Review specifications• Review software logic• Create Operational Definitions
28© 2001 Six Sigma Academy
Linearity
Measurement is not “true” and/or consistent across the range of the “gage”
Resolution?
Accuracy/Bias?
Linearity?
Stability?
Precision (R&R)?
OK
OK
OK
OK
29© 2001 Six Sigma Academy
Linearit
Full Range of GageReference Value
No Bias
Observed Average Value
Bias
30© 2001 Six Sigma Academy
Linearity Actions
• Use only in restricted range • Rebuild• Use with correction factor/table/curve• Sophisticated study required and will not be discussed in this course
31© 2001 Six Sigma Academy
Stability
Measurement drifts
Resolution?
Accuracy/Bias?
Linearity?
Stability?
Precision (R&R)?
OK
OK
OK
OK
32© 2001 Six Sigma Academy
Stability
• Measurements remain constant and predictable over time • For both mean and standard deviation
• No drifts, sudden shifts, cycles, etc. • Evaluated using control charts
Time 2
Time 1
Master Value(Reference Standard)
33© 2001 Six Sigma Academy
Stability Actions
• Change/adjust components• Establish “life” timeframe• Use control charts• Use/update current SOP
34© 2001 Six Sigma Academy
Precision
Repeatability and Reproducibility
Resolution?
Accuracy/Bias?
Linearity?
Stability?
Precision (R&R)?
OK
OK
OK
OK
35© 2001 Six Sigma Academy
Good Precision Poor Precision
Precision
2total = 2
product/process + 2repeatability + 2
reproducibility
Master Value
A B
Also known as Gage R&R
36© 2001 Six Sigma Academy
Repeatability (A Component Of Precision)
• Variation that occurs when repeated measurements are made of the same item under absolutely identical conditions
• Same: • Operator• Set-up• Units• Environmental conditions
• Short-term
37© 2001 Six Sigma Academy
Reproducibility (A Component Of Precision)
The variation that results when different conditions are used to make the measurements
• Different:• Operators• Set-ups • Test units• Environmental conditions • Locations• Companies
• Long-term
38© 2001 Six Sigma Academy
R&R Actions
Repeatability• Repair, replace, adjust equipment• SOP
Reproducibility• Training• SOP
39© 2001 Six Sigma Academy
Attribute Measurement Studies
40© 2001 Six Sigma Academy
Purpose Of Attribute MSA
• Assess standards against customers’ requirements • Determine if all appraisers use the same criteria • Quantify repeatability and reproducibility of operators • Identify how well measurement system conforms to a “known master” • Discover areas where:
• Training is needed• Procedures are lacking• Standards are not defined
41© 2001 Six Sigma Academy
Attribute MSA - Excel Method
• Allows for R&R analysis within and between appraisers• Test for effectiveness against standard• Limited to nominal data at two levels
42© 2001 Six Sigma Academy
DATE: 1/4/2001Attribute Legend
5 (used in computations) NAME: Acme Employee
1 Pass PRODUCT: Widgets2 Fail BUSINESS: Earth Products
Known PopulationSample # Attribute Try #1 Try #2 Try #1 Try #2 Try #1 Try #2
1 Pass Pass Pass Pass Pass Pass Pass2 Pass Pass Pass Pass Pass Pass Pass3 Pass Pass Pass Pass Pass Pass Pass4 Pass Pass Pass Pass Pass Fail Pass5 Fail Fail Fail Fail Fail Pass Fail6 Fail Pass Pass Pass Pass Pass Pass7 Pass Pass Pass Pass Pass Pass Pass8 Pass Pass Pass Pass Pass Pass Pass9 Fail Fail Fail Fail Fail Fail Fail10 Pass Pass Pass Pass Pass Pass Pass11 Pass Pass Pass Pass Pass Pass Pass12 Pass Pass Pass Pass Pass Pass Pass13 Pass Pass Pass Pass Pass Pass Pass14 Pass Pass Pass Pass Pass Fail Pass15 Fail Fail Fail Fail Fail Pass Fail16 Pass Pass Pass Pass Pass Pass Pass17 Pass Pass Pass Pass Pass Pass Pass18 Pass Pass Pass Pass Pass Pass Pass19 Fail Fail Fail Fail Fail Fail Fail20 Pass Pass Pass Pass Pass Pass Pass21 Pass Pass Pass Pass Pass Pass Pass22 Pass Fail Fail Pass Pass Pass Pass23 Pass Pass Pass Pass Pass Pass Pass24 Pass Pass Pass Pass Pass Fail Pass25 Fail Fail Fail Fail Fail Fail Fail26 Pass Pass Pass Pass Pass Pass Pass27 Pass Pass Pass Pass Pass Pass Pass28 Pass Pass Pass Pass Pass Pass Pass29 Fail Fail Fail Fail Fail Fail Fail30 Pass Pass Pass Pass Pass Pass Pass
Operator #1 Operator #2 Operator #3
Attribute MSA Example
Open file MSA-Attribute.xls
Microsoft Excel Worksheet
43© 2001 Six Sigma Academy
Scoring Example
• 100% is target for all scores • <100% indicates training required
• % Appraiser score = repeatability• Screen % Effectiveness Score = reproducibility• % Score vs. Attribute
• individual error against a known population• Screen % Effective vs. Attribute
• Total error against a known population
100.00% 78.57% 100.00%
78.57% 64.29% 71.43%
SCREEN % EFFECTIVE SCORE - > 57.14%
SCREEN % EFFECTIVE SCORE vs. ATTRIBUTE - > 42.86%
% APPRAISER SCORE - >
% SCORE VS. ATTRIBUTE - >
44© 2001 Six Sigma Academy
Statistical Report
45© 2001 Six Sigma Academy
Statistical Report
46© 2001 Six Sigma Academy
Statistical ReportContinued
47© 2001 Six Sigma Academy
Attribute MSA – MINITAB™ Method
• Allows for R&R analysis within and between appraisers• Test for effectiveness against standard• Allow nominal data with two levels• Allows for ordinal data with more than two levels
48© 2001 Six Sigma Academy
MINITAB Method - Data Entry
• Same data as Excel example• Arranged in multiple columns• Data can also be stacked in single column
49© 2001 Six Sigma Academy
Attribute Study - MINITAB Analysis
Attribute MSA.mpj
Tool Bar Menu > Stat > Quality Tools > Attribute Gage R&R Study
Attribute MSA.MPJ
50© 2001 Six Sigma Academy
Attribute Study - MINITAB AnalysisContinued
1. Select “Multiple Columns” if data is un-stacked
2. Enter number of appraisers and trials
3. Enter name of column with “Known” 4. Select OK
1. Select “Single Column” if data is stacked
51© 2001 Six Sigma Academy
Attribute MSA - MINITAB Graphical Output
Bob Sue Tom
70
80
90
100
Appraiser
Per
cent
Within Appraiser
Bob Sue Tom
60
70
80
90
100
Appraiser
Per
cent
Appraiser vs Standard
Assessment AgreementDate of study: 1/03/2001Reported by: JoseName of product: XYZ ReportMisc:
[ , ] 95.0% CI
Percent
Lower variation within appraiser
Higher variation within appraiser
Lower variation appraiser vs. standard
Higher variation appraiser vs. standard
Not included if no “Known”
52© 2001 Six Sigma Academy
Attribute MSA – MINITAB Session Window Results
Each Appraiser vs. Standard
Assessment Agreement
Appraiser # Inspected # Matched Percent (%) 95.0% CI
Bob 30 28 93.3 ( 77.9, 99.2)
Sue 30 29 96.7 ( 82.8, 99.9)
Tom 30 24 80.0 ( 61.4, 92.3)
# Matched: Appraiser's assessment across trials agrees with standard.
Assessment Disagreement
Appraiser # Pass/Fail Percent (%) # Fail/Pass Percent (%) # Mixed Percent (%)
Bob 1 3.3 1 3.3 0 0.0
Sue 1 3.3 0 0.0 0 0.0
Tom 1 3.3 0 0.0 5 16.7
# Pass/Fail: Assessments across trials = Pass/standard = Fail.
# Fail/Pass: Assessments across trials = Fail/standard = Pass.
# Mixed: Assessments across trials are not identical.
Between Appraisers
Assessment Agreement
# Inspected # Matched Percent (%) 95.0% CI
30 24 80.0 ( 61.4, 92.3)
# Matched: All appraisers' assessments agree with each other.
All Appraisers vs. Standard
Assessment Agreement
# Inspected # Matched Percent (%) 95.0% CI
30 23 76.7 ( 57.7, 90.1)
# Matched: All appraisers' assessments agree with standard.
Individual vs. Standard
Disagreement assessment(repeatability)
Between appraisers(reproducibility)
Total agreement(against known)
53© 2001 Six Sigma Academy
MINITAB Method - Ordinal Data Entry
Ordinal MSA.mtw• Survey data rated on a 1 to 5 scale• Arranged in multiple columns
Minitab Worksheet
54© 2001 Six Sigma Academy
Attribute Study - Ordinal
Select “categories of the attribute data are ordered”
Analysis is same as 2 level data
55© 2001 Six Sigma Academy
Industrial Attribute MSA Exercise
• Evaluate samples supplied by instructor • Determine the screen and appraiser scores• Interpret the results• Recommend actions iGrafx Professional Document
attributecircles.MPJ
56© 2001 Six Sigma Academy
Variables Measurement Studies
57© 2001 Six Sigma Academy
Six Step Variables MSA
1. Conduct initial gage calibration (or verification)
2. Perform trials and data collection
3. Obtain statistics via MINITAB
4. Analyze, interpret results
5. Check for inadequate measurement units
6. On-going evaluation• What would be your long-term gage plan ?
58© 2001 Six Sigma Academy
Trials And Data Collection
• Generally two to three operators • Generally 5-10 process outputs to measure • Each process output is measured 2-3 times (replicated) by each
operator
Randomization is Critical
1 2 3
P1
1 2 3
P2
1 2 3
P3
1 2 3
P4
1 2 3
P5
Oper1
1 2 3
P1
1 2 3
...
1 2 3
P5
Oper2
1 2 3
P1
1 2 3
...
1 2 3
P5
Oper3
59© 2001 Six Sigma Academy
Randomization, Repeats, Replicates
Randomization• Runs are made in an arbitrary vs. patterned order • Average out effects of noise or unknown factors• Tradeoff - Invalid results versus slight inconvenience (if any)
Repeats • Running more than one sample of a single run• Results are averaged
Replication • Running entire experiment in a time sequence• MSA allows for repeatability study
60© 2001 Six Sigma Academy
Variables MSA - MINITAB Example
Variable MSA.mtwUSL=1.5LSL=0.5
Replicate 1 Replicate 2(Randomized order)
Variable MSA.MTW
61© 2001 Six Sigma Academy
MSA Using MINITAB
10 Process Outputs
3 Operators
2 Replicates• Have Operator 1 measure all
samples once (as shown in the outlined block)
• Then, have Operator 2 measure all samples once
• Continue until all operators have measured samples once (this is Replicate 1)
• Repeat these steps for the required number of Replicates
• Enter data into MINITAB in 3 columns as shown
USL=1.5LSL=0.5
Replicate 1 Replicate 2(Randomized order)
62© 2001 Six Sigma Academy
Manipulate The Data
Your data in MINITAB should initially look like this. You will need to STACK your data so that all like data is in one column only
Now you are ready to run the macro for data analysis
Use the commands> Manip> Stack > Stack Blocks of Columns
(Stack all Process Outputs, Operators, and Responses so that they are in one column only)
63© 2001 Six Sigma Academy
Note:c10, c11, c12 are the columns in which the respective data are found IN OUR EXAMPLE. You must have ALL data STACKED in these columns
Enter titles
Stacked And Ready For Analysis
64© 2001 Six Sigma Academy
Prepare The Analysis
Use the commands> Stat > Quality Tools > Gage R&R Study (Crossed)
Each process output measured by each operator
OR
> Gage R&R Study (Nested)For “destructive
tests” where each process output is measured uniquely by each operator
65© 2001 Six Sigma Academy
ANOVA method is preferred• Gives more information
Enter Gage Info and Options
Choose Method Of Analysis
66© 2001 Six Sigma Academy
USL - LSL=0.50
USL=1.0
LSL=0.6
USL=1.0
LSL=0.5
Adding Tolerance (Optional)
Upper Specification Limit (USL)
MinusLower Specification
Limit (LSL)
For this example:
67© 2001 Six Sigma Academy
Two-Way ANOVA Table With Interaction
Source DF SS MS F P
Part 9 2.05871 0.228745 39.7178 0.00000
Operator 2 0.04800 0.024000 4.1672 0.03256
Operator*Part 18 0.10367 0.005759 4.4588 0.00016
Repeatability 30 0.03875 0.001292
Total 59 2.24912
Gage R&R %Contribution
Source VarComp (of VarComp)
Total Gage R&R 0.004437 10.67
Repeatability 0.001292 3.10
Reproducibility 0.003146 7.56
Operator 0.000912 2.19
Operator*Part 0.002234 5.37
Part-To-Part 0.037164 89.33
Total Variation 0.041602 100.00
StdDev Study Var %Study Var %Tolerance
Source (SD) (5.15*SD) (%SV) (SV/Toler)
Total Gage R&R 0.066615 0.34306 32.66 68.61
Repeatability 0.035940 0.18509 17.62 37.02
Reproducibility 0.056088 0.28885 27.50 57.77
Operator 0.030200 0.15553 14.81 31.11
Operator*Part 0.047263 0.24340 23.17 48.68
Part-To-Part 0.192781 0.99282 94.52 198.56
Total Variation 0.203965 1.05042 100.00 210.08
Number of Distinct Categories = 4
MSA Output:
Gage name:Date of study:Reported by:
Tolerance:Misc:
00.30.40.50.60.70.80.91.01.1 1 2 3
Xbar Chart by Operator
Sam
ple
Mea
n
Mean=0.8075UCL=0.8796
LCL=0.7354
0
0.00
0.05
0.10
0.15 1 2 3
R Chart by Operator
Sam
ple
Ran
ge
R=0.03833
UCL=0.1252
LCL=0
1 2 3 4 5 6 7 8 9 10
0.4
0.5
0.6
0.7
0.8
0.9
1.0
1.1
Part
OperatorOperator*Part Interaction
Ave
rage
1 2
3
1 2 3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
1.1
Operator
By Operator
1 2 3 4 5 6 7 8 9 10
0.4
0.5
0.6
0.7
0.8
0.9
1.0
1.1
Part
By Part
%Contribution
%Study Var %Tolerance
Gage R&R Repeat Reprod Part-to-Part
0
100
200
Components of Variation
Per
cent
Gage R&R (ANOVA) for Response
What does all this mean?
Session Window Graphs
68© 2001 Six Sigma Academy
Gage name:Date of study:Reported by:Tolerance:Misc:
00.30.40.50.60.70.80.91.01.1 1 2 3
Xbar Chart by Operator
Sam
ple
Mea
n
Mean=0.8075UCL=0.8796
LCL=0.7354
0
0.00
0.05
0.10
0.15 1 2 3
R Chart by Operator
Sam
ple
Ran
ge
R=0.03833
UCL=0.1252
LCL=0
1 2 3 4 5 6 7 8 9 10
0.4
0.5
0.6
0.7
0.8
0.9
1.0
1.1
Part
OperatorOperator*Part Interaction
Ave
rage
1 2
3
1 2 3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
1.1
Operator
By Operator
1 2 3 4 5 6 7 8 9 10
0.4
0.5
0.6
0.7
0.8
0.9
1.0
1.1
Part
By Part
%Contribution
%Study Var %Tolerance
Gage R&R Repeat Reprod Part-to-Part
0
100
200
Components of Variation
Per
cent
Gage R&R (ANOVA) for ResponseMSA HealthSide
MSATroubleshoot
Side
Graphical Output - 6 Graphs In All
If only 1 operator, you won’t get these graphs
If nested study, you won’t get this
graph
69© 2001 Six Sigma Academy
Destructive Test
Gage name:Date of study:Reported by:Tolerance:Misc:
12
13
14
15
16
17
18 Bill ie Nathan Steve
Xbar Chart by Operator
Sam
ple
Mea
n
Mean=15.15
UCL=17.62
LCL=12.68
0
1
2
3
4
5 Bill ie Nathan Steve
R Chart by Operator
Sam
ple
Ran
ge
R=1.313
UCL=4.290
LCL=0
Billie Nathan Steve
13
14
15
16
17
18
Operator
By Operator
6 7 8 9 10 11 12 13 14 15 1 2 3 4 5Billie Nathan Steve
13
14
15
16
17
18
PartOperator
By Part (Operator)
%Contribution
%Study Var
Gage R&R Repeat Reprod Part-to-Part
0
50
100
Components of Variation
Per
cent
Gage R&R (Nested) for Response
Operator by process output interaction is not applicable
70© 2001 Six Sigma Academy
Graphical Output Metrics
Chart Output• Xbar Chart: Shows sampled process output variety
• Reproducibility/bias/sensitivity• R Chart: Helps identify unusual measurements
• Resolution/repeatability • Bar Chart: Distinguishes R&R from Process Output to Process
Output• Components of variation
These are your leading graphical indicators
71© 2001 Six Sigma Academy
Misc:Tolerance:Reported by:Date of study:Gage name:
0
4
3
2
1
0
-1
321
Xbar Chart by Operator
Sam
ple Mea
n
Mean=1.401
UCL=3.654
LCL=-0.8528
0
4
3
2
1
0
321
R Chart by Operator
Sam
ple Ran
ge
R=1.198
UCL=3.915
LCL=0
10 9 8 7 6 5 4 3 2 1
2.0
1.5
1.0
Part
OperatorOperator*Part Interaction
Ave
rage
1 2
3
321
3
2
1
Operator
By Operator
10 9 8 7 6 5 4 3 2 1
3
2
1
Part
By Part
%Contribution %Study Var
Part-to-PartReprodRepeatGage R&R
100
50
0
Components of VariationPerce
nt
Gage R&R (ANOVA) for ResponseBar Charts For Components
Much better
Needs help
Answers: “Where is the variation?”
72© 2001 Six Sigma Academy
Closer Look At The Xbar & R Charts
Xbar: at least 50% outside limits; R chart: in control
R Chart: Exposes gage Repeatability, resolution & stability
Xbar Chart: Test of sensitivity,
bias, & population variety
73© 2001 Six Sigma Academy
More R Chart Indicators
Both may indicate poor gage resolution
0
0.005
0.004
0.003
0.002
0.001
0.000
321
R Chart
Sam
ple
Ran
ge
R=4.33E-04
UCL=0.001416
LCL=0
0
0.15
0.10
0.05
0.00
321
R Chart by Operator
Sam
ple
Ran
ge
R=0.03833
UCL=0.1252
LCL=0
Randy
Rbar too small?
Plateaus
74© 2001 Six Sigma Academy
%Study
%Tolerance
%Contribution
Tabular Output Metrics
Number of Distinct Categories
75© 2001 Six Sigma Academy
% Contribution
• Measurement System Variation (R&R) as a percentage of Total Observed Process Variation
• Includes both repeatability and reproducibility
100* onContributi %TOTAL
2R&R
2
% Contribution
1%
9%
76© 2001 Six Sigma Academy
% Study Variation
• Looks at standard deviations instead of variance• Measurement System Standard Deviation (R&R) as a percentage of
Total Observed Process Standard Deviation• Includes both repeatability and reproducibility % Study
Variation
10%
30%
100* ationStudy Vari %TOTAL
R&R
77© 2001 Six Sigma Academy
AcceptanceCriteria
% Tolerance
• Measurement error as a percent of tolerance• Includes both repeatability and reproducibility• 5.15 Study Variation = 99% % Tolerance
10%
30%
100*Tolerance
*155 Tolerance %
P/T Tolerance to Precision
R&R
78© 2001 Six Sigma Academy
Distinct Categories
• Number of divisions that the Measurement System can accurately measure across the process variation
• How well a measurement process can detect process output variation- process shifts and improvement
• Less than 5 indicates Attribute conditions Number of DistinctCategories
10
5
R&R
Output Process*2 Categories Distinct of Number 2
2
79© 2001 Six Sigma Academy
Acceptability Summary
Tabular Method
% Contribution
1%
9%
ProcessControl% Study Variation
10%
30%
ProductControl
% Tolerance
10%
30%
Number ofDistinct
Categories
10
5
Desirable to Have All 4 Indicators Say “Go”
80© 2001 Six Sigma Academy
Keys To Successful MSA
• Define and validate measurement process• Identify known elements of the measurement process (operators,
gages, SOP, setup, etc.)• Clarify purpose and strategy for evaluation• Set acceptance criteria• Implement preventive/corrective action procedures• Establish on-going assessment criteria and schedules
81© 2001 Six Sigma Academy
Gage R&R - Which % Gage R&R Do I Use?
Depending on how variable your process is as compared to tolerance, your % Gage R&R values as a percent of Study variation, Tolerance and Process Variation will be quite different.
For example:
Consider a very stable process with low variability. Percent Tolerance will indicate that your gauge is very good (low % GRR) with high discrimination. On the other hand, when compared to process variation, the GRR will be poor (High % GRR).
As your process improves, you will need to move to more precise gauges if you wish to “see” decreases in variation due to the measuring system. On the other hand, if you truly only want to be able to tell when production is becoming less capable, then you are only interested in the precision of the gauge as it relates to your customer’s specification. See the Appendix at the end of this module for further examples
82© 2001 Six Sigma Academy
Buffalo, NY Plant
1.5 mmSix Sigma BB01/01/1998
Gage #020371
Misc:
Tolerance:Reported by:Date of study:
Gage name:
10 9 8 7 6 5 4 3 2 1
1.1
1.0
0.9
0.8
0.7
0.6
0.5
0.4
Part ID
OperatorOperator*Part Interaction
Ave
rage
1
2
3
Gage R&R (ANOVA) for Measure
Buffalo, NY Plant
1.5 mmSix Sigma BB01/01/1998
Gage #020371
Misc:
Tolerance:Reported by:Date of study:
Gage name:
321
1.1
1.0
0.9
0.8
0.7
0.6
0.5
0.4
Oper ID
By Operator
Gage R&R (ANOVA) for Measure
Buffalo, NY Plant
1.5 mmSix Sigma BB01/01/1998
Gage #020371
Misc:
Tolerance:Reported by:Date of study:
Gage name:
10 9 8 7 6 5 4 3 2 1
1.1
1.0
0.9
0.8
0.7
0.6
0.5
0.4
Part ID
By Part
Gage R&R (ANOVA) for Measure
Gage R&R, Graphical Output:
• Operator * Part Interaction:• Shows if any given part(s) was hard to manage for any given operator(s)• Appears as though at least two of the operators had trouble measuring part #10• What would the ideal graph look like?
• By Operator:• Shows if any operator(s) had higher or lower readings (on average) than the others• What would the ideal graph look like?
• By Part:• Shows the ability of all of our operators to obtain the same readings for each part• Also shows the ability of our measurement system to distinguish between parts (amount of
overlap)• What would be the ideal graph look like?
83© 2001 Six Sigma Academy
Buffalo, NY Plant
1.5 mmSix Sigma BB01/01/1998
Gage #020371
Misc:
Tolerance:Reported by:Date of study:
Gage name:
0
1.11.00.90.80.70.60.50.40.3
321Xbar Chart by Operator
Sa
mpl
e M
ea
n
X=0.8075
3.0SL=0.8796
-3.0SL=0.7354
0
0.15
0.10
0.05
0.00
321R Chart by Operator
Sa
mpl
e R
ang
e
R=0.03833
3.0SL=0.1252
-3.0SL=0.000
Gage R&R (ANOVA) for Measure
Gage R&R, Xbar & R:
• How do we evaluate the X-bar & R-chart?
• Why are the data points out of control on the X-bar and R chart?
84© 2001 Six Sigma Academy
1.080.980.880.780.680.580.480.38
5 4 3 2 1Part Num
Me
asu
re
12
3
1.080.980.880.780.680.580.480.38
10 9 8 7 6Part Num
Me
asu
re
Runchart of Measure by Part, Operator
Minitab, Gage Run Chart:
• Generates a run chart of measurements by operator and part id• Allows us to visualize repeatability and reproducibility within and between
operator and part• The center line is the overall average of the parts
• STAT > Quality Tools > Gage Run Chart
85© 2001 Six Sigma Academy
0.0
1.0
2.0
3.0
4.0
5.0
6.0
0.5 0.6 0.7 0.8 0.9 1.0 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 2.0
Observed Cp
Act
ual
Cp 0%
10%
20%
30%
40%50%
60%
70%
P/T Ratio
P/T Ratio Effect on Capability
86© 2001 Six Sigma Academy
Which Might Need The Most Attention?
Measurement System or Process Capability
Process %R&R Obs. Cp Decision ?
1 10% 0.5 ?
2 60% 1.4 ?
3 60% 0.5 ?
4 70% 6.5 ?
% R&R Vs. Capability
87© 2001 Six Sigma Academy
Which Might Need The Most Attention?
Measurement System or Process Capability
Process %R&R Obs. Cp Decision ?
1 10% 0.5 Capability
2 60% 1.4 Measurement
3 60% 0.5 Maybe Both
4 70% 6.5 Measurement
*Note: Process Step 4
Would improving %R&R really be worth the effort ?
% R&R Vs. Capability
88© 2001 Six Sigma Academy
Handling Poor Gage Capability:
• If a dominant source of variation is repeatability (equipment), you need to replace, repair, or otherwise adjust the equipment.
• If, in consultation with the equipment vendor or upon searches of industry literature, you find that the gage technology that you are using is “state-of-the-art” and it is performing to its specifications, you should still fix the gage. One temporary solution to this problem is to use signal averaging (see next page).
• If a dominant source of variation is operator (reproducibility), you must address this via training and definition of the standard operating procedure. You should look for differences between operators to give you some indication as to whether it is a training, skill, and/or procedure problem.
• Evaluate the specifications. Are they reasonable?• If the gage capability is marginal (as high as 30% of study variation)
and the process is operating at a high capability (Ppk greater than 2), then the gage is probably not hindering you and you can continue to use it.
89© 2001 Six Sigma Academy
• Note: If you want to decrease your gage error take advantage of the standard error square root of the sample.
• The signal averaging technique uses:
n
1
• n = the number of repeat measures taken on the same part
• the measurement = the average of “n” readings
• Example: a gage error of 50% can be cut in half if your point estimate is an average of 4 repeat measurements
2/14
1 • This technique should be used as a short term approach to
perform a study, but you must fix the gage.
xxxxxxxxxx
x
xxxx
x xx
Distributionof Means
Distributionof Individuals
Controlling Repeatability:
90© 2001 Six Sigma Academy
The Signal-to-Noise Ratio (S/N Ratio) relates the product variation to the measurement system variation. The S/N Ratio should be as large as possible.
The Discrimination Index provides the number of divisions that the Measurement System can accurately measure across the part (sample) variation. If this index is less than 4, then it is inadequate to provide data for a study. If the index is 4, then it is equivalent to a go/no-go gage. We would like to see the value of 5 or greater.
S / N Ratio
P
MS
Discrim=
p
ms
* .141
Other Statistical Indexes
91© 2001 Six Sigma Academy
Effects of P/T and S/N Ratios
• The effect of P/T on Cpk• Large P/T reduces the process Cpk from the true value to
some smaller observed value. • The effect of P/T on part assessment
• Large P/T increases the probability that we will misclassify product as defective when it’s really good and vice versa.
• The effect of S/N ratio on control chart sensitivity• small S/N increases the time before an out-of-control
process is detected by a control chart (refer to X-bar & range)
• The Effect of the Discrimination Index• If the Index = 2, only attribute data is available and sample
sizes must be larger.• If the Index is 5 to 10, then discrimination is finer and
sample sizes can be smaller.
92© 2001 Six Sigma Academy
Calibration Steps
• Determine if the measurement system needs to be recalibrated
• Determine the minimum number of measurements needed to make this decision
• Take data and make decision• If yes, recalibrate system• Why don’t we just recalibrate?
• Normal variation causes the measurement to be slightly different each time it is used
• Recalibration should be done only when the measurements are off by more than the normal variation
• Recalibrating a system when it is not needed can increase the variability in the measurements
93© 2001 Six Sigma Academy
Appendix
94© 2001 Six Sigma Academy
Interpreting Variables GR&R Results
Presented on the following slides are four Variable Gage R&R results - % Study (P/TV - Precision to Total Variation) and % Tolerance (P/T - Precision to Tolerance) along with a representative graphical illustration to help visualize the results and any required action to improve the Measurement System. Also discussed is the effect of the GR&R on Cp.
– There are an infinite number of GR&R results(combinations of % Study and % Tolerance) use these four relatively extreme scenarios to help you determine what actions that you need take given your own results. Remember we are looking for GR&R results of < 10%, although anything less than 30% is considered barely acceptable (proceed with caution).
– These graphs are not drawn to scale, therefore, when reviewing this information do not compare the relative size of the histograms between the scenarios, rather, compare the histograms within the scenario to the Spec Limits. Actual data was not used to create these histograms.
– These examples assume 10 parts were selected that represent the long-term capability of the process being investigated. Three operators, 2 trial.
– No assumptions have been made as to the problem with the Measurement System.
– Actual data was not used to calculate the Cp indices. They were visually estimated, but are assumed reasonable.
95© 2001 Six Sigma Academy
Scenario #1
9080706050
15% - % Study15% - % Tolerance
Gage Contribution(Precision)
Part Contribution(Part Variation)
Observed(Total Variation)
In this example we observe a GR&R result that is acceptable, where the % Study Variation is the same as the % Tolerance Variation. The results are the same because the relative size of the Total Variation -PV (5.15*sTotal) and the Tolerance- T (USL - LSL) are the same. Therefore, when we take the P/TV or P/T ratio, where P is the Precision of the Gage (5.15* sms) it is well below 30%.
This gage is deemed acceptable, no action is required. The only action is to improve the Process Capability.
Furthermore, the observed Cp of this process is probably close to 1, as it appears 6 standard deviations of the process can fit inside the tolerance once. Finally, as a result of the acceptable GR&R values the observed Cp (what we measure) is considered to be the actual Cp.
LSL USL
Tolerance
96© 2001 Six Sigma Academy
Scenario # 2
70% - % Study70% - % Tolerance
9080706050
Gage Contribution(Precision)
Part Contribution(Part Variation)
Observed(Total Variation)
Tolerance
In this example we observe a GR&R where the % Study Variation is the same as the % Tolerance Variation, however the results are extremely unacceptable. The results are the same because the relative size of the Total Variation -TV (5.15*sTotal) and the Tolerance- T (USL - LSL) are the same. Therefore, when we take the P/TV or P/T ratio, where P is the gage contribution (5.15* sms) it is very much above 30%. Thus, indicating the Measurement System can not effectively discern part to part differences. The impact of a poor GR&R results is to inflate the variability of the product standard deviation.
In this example we absolutely need to fix the Measurement System!!!
Finally the observed Cp of this process (using this poor gage) is probably close to 0.5, as it appears that only half of the 6 standard deviations of the process can fit inside the tolerance. The actual Cp is probably much higher maybe closer to 1 or 1.5. If the measurement system were improved and deemed acceptable the observed Cp would reflect actual Cp.
LSL USL
97© 2001 Six Sigma Academy
Scenario #3
9080706050
Gage Contribution(Precision)
Part Contribution(Part Variation)
Observed(Total Variation)
70% - % Study 5% - % Tolerance
LSL USL
Tolerance
Here we observe a GR&R where the % Study Variation is extremely unacceptable and the % Tolerance Variation is very acceptable. How can this be? In this example the Gage Precision - P (5.15* sms) compared to the Total Variation - TV (5.15*sTotal) P/TV is quite large - 70%. However, when we compare the Gage Precision with to the Tolerance (USL - LSL) P/T we observe a very acceptable GR&R - 5%.
Do we need to fix our Measurement System? Well that depends, if we are still looking for process improvement then we should fix the measurement system. If, however, we do not need to improve the process capability then our measurement system is acceptable.
In this example our observed Cp is probably close to 2 (99.73% of our process variability close can fit into our customer tolerance), where as the actual Cp may be significantly higher. If for some reason the PV began to increase to the size of the Tolerance then we would observe our gage as acceptable.
98© 2001 Six Sigma Academy
Scenario #4
9080706050
Gage Contribution(Precision)
Part Contribution(Part Variation)
Observed(Total Variation)
5% - % Study70% - % Tolerance
LSL USL
Tolerance
Here we observe a GR&R where the % Study Variation is acceptable and the % Tolerance Variation is very unacceptable. How can this be? In this example the Gage Precision - P (5.15* sms) compared to the Total Variation - PV (5.15*sTotal) P/TV is very small - 5%. However, when we compare the Gage Precision with to the Tolerance (USL - LSL) P/T we observe a very large GR&R - 70%%.
Do we need to fix our Measurement System? Yes, we need to fix the measurement system. In this example, the observed Cp will be the actual Cp and it is probably about 0.2 to 0.4. However, as we work our Six Sigma project and reduce the variability of our KPOV to improve our Process Capability our % Study Variation will become worse (% Tolerance, will remain constant). When our Process Variation is the same size as the Tolerance, both GR&R’s will be 70% and our observed Cp will not reflect the actual. Therefore improvement of the measurement system is required.