comparison of alternative measurement methods (kuttatharmmakul)
Post on 22-Sep-2015
225 views
Embed Size (px)
DESCRIPTION
numero de replicas necesario para realizar la validaciónTRANSCRIPT
Comparison of alternative measurement methods
Siriporn Kuttatharmmakul, D. Luc Massart, Johanna Smeyers-Verbeke*
ChemoAC, Pharmaceutical Institute, Vrije Universiteit Brussel, Laarbeeklaan 103, B-1090, Brussel, Belgium
Received 19 February 1998; received in revised form 14 July 1998; accepted 14 July 1998
Abstract
A procedure to compare the performance (precision and bias) of an alternative measurement method and a reference method
has been extensively described. It is based on ISO 5725-6 which has been adapted to the intralaboratory situation. This means
that the proposed approach does not evaluate the reproducibility, but considers the (operatorinstrumenttime)-differentintermediate precision and/or the time-different intermediate precision. A 4-factor nested design is used for the study. The
calculation of different variance estimates from the experimental data is carried out by ANOVA. The Satterthwaite
approximation is included to determine the number of degrees of freedom associated with the compound variances. Taken into
account the acceptable bias, the acceptable ratio between the precision parameters of the two methods, the significance level and the probability to wrongly accept an alternative method with an unacceptable performance, the formulae to determinethe number of measurements required for the comparison are given. For the evaluation of the bias, in addition to the point
hypothesis testing, the interval hypothesis testing is also included as an alternative. Two examples are given as an illustration
of the proposed approach. # 1999 Elsevier Science B.V. All rights reserved.
Keywords: Comparison; Alternative measurement method; Bias; Precision; Repeatability; Time-different intermediate precision;
(Operatorinstrumenttime)-different intermediate precision; Nested design; ANOVA; Satterthwaite approximation; Interval hypothesistesting
1. Introduction
When a laboratory wants to replace an existing
analytical method by a new method (e.g. because
the latter is cheaper or easier to use) it has to show
that the new method performs at least as good as the
existing one. A comparison of the performance (pre-
cision and bias) of both methods has therefore to be
performed. One of the most advanced guidelines for
the comparison of two methods can be found in ISO
5725-6 [1]. However the ISO guideline is based on
interlaboratory studies and is therefore not applicable
in the intralaboratory situation. Indeed within a single
laboratory, the reproducibility, as evaluated by ISO,
cannot be determined but intermediate precision con-
ditions, such as changes in operator, equipment and
time should be considered since they contribute to the
variability of measurements performed in the labora-
tory.
In the ISO guideline the reference method is an
international standard method that was studied in an
interlaboratory test program and its precision (2) isassumed to be known. This assumption is reasonable
since the precision is obtained from a large number of
measurements. In the intralaboratory situation a
Analytica Chimica Acta 391 (1999) 203225
*Corresponding author. Tel.: +32-2477-4737; fax: +32-2477-
4735; e-mail: asmeyers@vub.vub.ac.be
0003-2670/99/$ see front matter # 1999 Elsevier Science B.V. All rights reserved.PII: S 0 0 0 3 - 2 6 7 0 ( 9 9 ) 0 0 1 1 5 - 4
laboratory has developed a first method and later on
wishes to compare a new method to the older already
internally validated method. For the latter, referred to
as the reference method, only an estimate of the
precision (s2) will be available since the precision is
determined from a rather limited number of measure-
ments. This of course determines the statistical tests to
be used in the comparison of the performance char-
acteristics of both methods.
Moreover, the ISO standard is meant to show that
both methods have similar precision and/or trueness
whereas a laboratory that performs a method compar-
ison study is interested to evaluate whether the new
method is at least as good as the reference method.
This implies that some two-sided statistical tests
included in the ISO guideline are not appropriate
for the comparison of two methods in a single labora-
tory, where example in the evaluation of the precision
one-sided tests have to be considered.
In the decision making concerning the new alter-
native method it is important (i) not to reject an
alternative method which in fact is appropriate, and
(ii) not to accept an alternative method which in fact is
not appropriate. The former is related to the a-error ofthe statistical tests used in the comparison and is
controlled through the selection of the significance
level. The latter is related to the b-error and when it isconsidered it is generally taken into account by includ-
ing sample size calculations. This approach is also
included in the ISO guideline.
In this article we propose an adaptation of the ISO
guideline to the intralaboratory comparison of two
methods. It is also applicable to the situation in which
two laboratories of, e.g., the same organisation are
involved, each laboratory being specialized in one of
the methods. For the evaluation of the bias, in addition
to the point hypothesis testing, interval hypothesis
testing [2] in which the probability of accepting a
method that is too much biased is controlled, is also
included.
Due to the specified acceptance criteria for the
alternative method, the proposed approach might lead
to a large number of measurements to be performed.
An alternative approach (which will be described in a
next article) is to perform the method comparison with
a user-defined number of measurements and to eval-
uate the probability that a method with an unaccep-
table performance will be accepted.
2. Methods
All symbols and abbreviations used in this paper are
defined in Table 1.
2.1. Experimental design
A 4-factor nested experimental design is used
[37]. This design is also one of the designs recom-
mended by ISO [3]. The schematic layout of the
design is given in Fig. 1. The four factors represent
four sources of variation that contribute to the varia-
bility of the measurements within one laboratory. The
factors considered are operator, instrument, time, and
random error. The experimental approach can be
described as follows. For each analytical method,
the sample is analysed by m operators. Each operator
performs, on each of q instruments, n replicated
measurements on each of p different days. To avoid
an underestimation of the day effect, the set of p
different days during which the measurements are
performed on each of the q instruments must be
different, i.e. two instruments cannot be operated on
the same day.
Fig. 1. Schematic layout for the 4-factor nested experimental
design applied. Only the nested structure under the ith operator, jth
instrument and the kth day is shown here. The nested structure
under other operators, instruments and days has the same pattern.
(instruinstrument, repreplicate).
204 S. Kuttatharmmakul et al. / Analytica Chimica Acta 391 (1999) 203225
Table 1
Definition of symbols and abbreviations applied in the document
d Absolute difference between the grand means obtained with two methods
D Component of day effect in a test result
E Random error component occurring in every test result
FI(OIT) Calculated F-value obtained from the comparison of (operatorinstrumenttime)-different intermediate precision (variance)FI(T) Calculated F-value obtained from the comparison of time-different intermediate precision (variance)
Fr Calculated F-value obtained from the comparison of repeatability variance
FB ;A Value of the F-distribution with B degrees of freedom associated with the numerator and A degrees of freedom associated withthe denominator; represents the portion of the F-distribution to the right of the given F-value
FA ;B Value of the F-distribution with A degrees of freedom associated with the numerator and B degrees of freedom associated withthe denominator; represents the portion of the F-distribution to the right of the given F-value
I Component of instrumental effect in a test result
m Number of operators
M General mean (expectation) of the test results
MS Mean squares
n Number of replicates performed on each day
n Average number of replicates performed on each dayN Total number of measurements
O Component of operator effect in a test result
p Number of days
q Number of instruments
s Estimate of
s2 Estimate of 2
tcal Calculated t-value obtained from the comparison of the means obtained with two methods
t/2 Two-sided tabulated t-value at significance level and degrees of freedom
t One-sided tabulated t-value at significance level and degrees of freedom
UCL Upper confidence limit
y Test resulty Grand mean of test resultsyi Arithmetic mean of the test results obtained from the ith operatoryij Arithmetic mean of the test results obtained from the ith operator and the jth instrumentyijk Arithmetic mean of the test results obtained from the ith operator, the jth instrument and the kth dayyijkL Particular test result related to the Lth replicate of the kth day, the jth instrument and the ith ope