1nswc corona-ms interval dj june 2002 dr. dennis jackson 909-273-4492 dsn 933-4492...
TRANSCRIPT
2NSWC Corona-MS Interval DJ June 2002
CALIBRATION INTERVAL ANALYSIS: CURRENT AND
FUTURE
CALIBRATION INTERVAL ANALYSIS: CURRENT AND
FUTURE
Dr. Dennis JacksonMS30A1
June 2002
3NSWC Corona-MS Interval DJ June 2002
Overview
Current Calibration Interval Methods
Interval Analysis Results
New Approaches to Calibration Interval Estimation
4NSWC Corona-MS Interval DJ June 2002
Current Methods:What Is a Calibration?
Compare the measurement values from a UUT with the measurement values from a calibrator.
– Deviation = UUT Measurement – Calibrator Measurement
A UUT is considered in tolerance if:
– Lower Tolerance < Deviation < Upper Tolerance
Measurement Reliability is the probability of being in tolerance.
A Calibration Interval is the amount of time between calibrations that will meet a measurement reliability target (keeps the UUT in tolerance).
5NSWC Corona-MS Interval DJ June 2002
Current Methods:Calibration Interval Determination
0 6 12 18 24 30 36 42 48
Test Equipment ReliabilityTest Equipment Reliabilityvs. Calibration Intervalvs. Calibration Interval
Calibration Interval (Months)
100
90
80
70
60
50
40
30
20
10
0
Mea
sure
men
t R
elia
bili
ty (
%)
72% EOP Reliability for GPTE
85% EOP Reliability for Safety-of-Flight and Mission Critical
6NSWC Corona-MS Interval DJ June 2002
Current Methods: Stages of the Calibration Interval
Process
EngineeringInterval Est.
No Further No Further ReviewReview
Gather Relevant
Data
StatisticalInterval Est.
IntegratedInterval
Est?
DivisionDivisionReviewReview
PolicyPolicyReviewReview
TR-6
QA
Yes
METRL
1 2 3 4 5
No
7NSWC Corona-MS Interval DJ June 2002
Interval Analysis Results:NAVSEA Interval Changes
INTERVAL ACTIONINTERVAL ACTION COUNTCOUNT
IN PROCESS 148
INITIAL INTERVALS 332
EXTENSIONS 113
DECREASES 24
NO CHANGE 361
TOTAL 978
(FY 2002 through April 2002)
8NSWC Corona-MS Interval DJ June 2002
Interval Analysis Results:Annual Calibration Cost Avoidance
NAVSEA NAVY
EXTENSIONS $153K 1918 (M/H)
$372K4644 (M/H)
DECREASES -$40K-495 (M/H)
-$60K-749 (M/H)
COST AVOIDANCE $113K 1423 (M/H)
$312K 3895 (M/H)
(Based on changes made in FY 2002 Through April 2002)
9NSWC Corona-MS Interval DJ June 2002
New Approaches to Calibration Interval Estimation
Near Term - Binomial Calibration Interval Estimation Methods– More accurate interval estimates
– Alternative reliability models
– Visual analysis methods
Long Term - Variables Data Calibration Interval Estimation Methods– Fixes data problems
– More information on measurement characteristics
– Less data required
– MEASURE 2 capability with automated data
10
NSWC Corona-MS Interval DJ June 2002
Traditional Reliability Methods
Assumptions: You know when the failure occurs.
R = 1.0 at time 0.Data: Failure Times.
0
0.2
0.4
0.6
0.8
1
Time Since Calibration
Me
as
ure
me
nt
Re
lia
bil
ity Exponential Model:
R = exp(-t)
11
NSWC Corona-MS Interval DJ June 2002
Tolerance Testing Data
Characteristics: • The failure occurs during an interval.• R < 1.0 at time 0.
0
0.2
0.4
0.6
0.8
1
Time Since Calibration
Me
as
ure
me
nt
Re
lia
bil
ity
Note: The points on this graph are observed in tolerance proportions.
12
NSWC Corona-MS Interval DJ June 2002
Using Traditional Methods On Tolerance Testing Data
Problem: The estimates don’t match the data because the intercept must go through 1.0.
0
0.2
0.4
0.6
0.8
1
Time Since Calibration
Me
as
ure
me
nt
Re
lia
bil
ity
13
NSWC Corona-MS Interval DJ June 2002
0
0.2
0.4
0.6
0.8
1
Time Since Calibration
Me
as
ure
me
nt
Re
lia
bil
ity
Reliability Methods For Tolerance Testing Data
Assumptions: The failure occurs during an interval.
R < 1.0 at time 0.Data: Success/Failure (Binomial)
Intercept Exponential ModelR = Ro exp(-t) = exp(0+ 1t)
14
NSWC Corona-MS Interval DJ June 2002
Current Status of Near Term Efforts
2002 MSC Paper: “Calibration Intervals – New Models and Techniques”
– Binomial Analysis, New Models, Reliability Intercepts, Initial Variables Methods
Binomial Calibration Interval Analysis System
15
NSWC Corona-MS Interval DJ June 2002
Benefits of Binomial Calibration Interval Estimation
Methods
The use of Binomial estimation methods provides more accurate calibration interval estimates based on current statistical estimation theory.
Binomial estimation methods allow for alternative measurement reliability models, including intercept and multivariable models.
Better graphical tools provide more understanding of test equipment behavior.
16
NSWC Corona-MS Interval DJ June 2002
Long Term Approach: Variables Calibration Data
-15
-10
-5
0
5
10
15
20
0 3 6 9 12 15 18 21 24
Time Since Calibration
Dev
iati
on
Fro
m S
tan
dar
d
Deviation
Nominal
U Limit
L Limit
17
NSWC Corona-MS Interval DJ June 2002
Calibration Intervals Based on Variables Data
Compute a Drift Trend.
Compute a Variability Trend using residuals from the drift trend.
Obtain a Reliability Curve using the drift and variability trends.
Determine the Calibration Interval from the reliability curve.
Predict the Measurement Uncertainty using the drift and variability trends.
18
NSWC Corona-MS Interval DJ June 2002
Drift Trend Analysis
E(d) = B0 + B1 t (Weighted Linear Regression on d)
-15
-10
-5
0
5
10
15
20
0 3 6 9 12 15 18 21 24
Time Since Calibration
Dev
iati
on
Fro
m S
tan
dar
d
Deviation
Nominal
U Limit
L Limit
Pred Line
19
NSWC Corona-MS Interval DJ June 2002
Variability Trend Analysis
E(res2) = C0 + C1 t (Linear Regression on res2)
-15
-10
-5
0
5
10
15
20
0 3 6 9 12 15 18 21 24
Time Since Calibration
Dev
iati
on
Fro
m S
tan
dar
d Deviation
Nominal
U Limit
L Limit
Pred Line
U Pred
L Pred
20
NSWC Corona-MS Interval DJ June 2002
A Basis for Increasing Variability
Generally, a single serial number does not show increasing variability
-6
-4
-2
0
2
4
6
8
0 10 20 30 40
Time Since Calibration
Dev
iati
on
Fro
m S
tan
dar
d
21
NSWC Corona-MS Interval DJ June 2002
A Basis for Increasing Variability
However, several serial numbers could have slightly different slopes and intercepts:
-6
-4
-2
0
2
4
6
8
0 10 20 30 40
Time Since Calibration
Dev
iati
on
Fro
m S
tan
dar
d
22
NSWC Corona-MS Interval DJ June 2002
A Basis for Increasing Variability
The overall effect is one of increasing variability for the population
-6
-4
-2
0
2
4
6
8
0 10 20 30 40
Time Since Calibration
Dev
iati
on
Fro
m S
tan
dar
d
23
NSWC Corona-MS Interval DJ June 2002
Reliability Curve Analysis
-20
-15
-10
-5
0
5
10
15
20
0 3 6 9 12 15 18 21 24
Time Since Calibration
Dev
iati
on
Fro
m S
tan
dar
d
Deviation
Nominal
U Limit
L Limit
Pred Line
U Pred
L Pred
Norm 6
Norm 15
24
NSWC Corona-MS Interval DJ June 2002
Determining Calibration Intervals From Variables Data
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 3 6 9 12 15 18 21 24
Time Since Calibration
Mea
sure
men
t R
elia
bili
ty
Reliability Target
Calibration Interval
Reliability Curve
25
NSWC Corona-MS Interval DJ June 2002
Current Statusof Long Term Efforts
2002 MSC Paper: “Calibration Intervals – New Models and Techniques”– Binomial Analysis, New Models, Reliability Intercepts,
Initial Variables Methods
2003 MSC Paper: “Calibration Intervals and Measurement Uncertainty Based on Variables Data”– NPSL, SCE
Variables Analysis Excel Tool– Estimates Trends, Calibration Intervals, Measurement
Uncertainty
MEASURE 2– Automated/Electronic data
26
NSWC Corona-MS Interval DJ June 2002
Benefits of UsingVariables Data
MEASURE data is often suspect– In-Tolerance data is difficult to verify
(success/failure)– Engineering review required for nearly all
calibration interval determinations
Variables data is more trustworthy– This could significantly increase the number of
interval analyses
Variables data provides much more information– Requires fewer calibrations to accurately
determine a calibration interval than In-Tolerance data
Development of automated/electronic data recording could reduce calibration time.
27
NSWC Corona-MS Interval DJ June 2002
Summary
Calibration intervals minimize the amount of calibration effort required to keep test equipment adequately in tolerance.
Recent adjustments to calibration intervals will result in significant cost avoidance.
Near-term improvements using Binomial methods will provide better visual analysis and more accurate estimation techniques.
Long-term improvements using variables data methods will:– Fix data problems– Provide faster analyses with less data– Possibly reduce administrative part of calibration
time