comparison of alternative measurement methods (kuttatharmmakul)

Download Comparison of Alternative Measurement Methods (Kuttatharmmakul)

Post on 22-Sep-2015

225 views

Category:

Documents

0 download

Embed Size (px)

DESCRIPTION

numero de replicas necesario para realizar la validación

TRANSCRIPT

  • Comparison of alternative measurement methods

    Siriporn Kuttatharmmakul, D. Luc Massart, Johanna Smeyers-Verbeke*

    ChemoAC, Pharmaceutical Institute, Vrije Universiteit Brussel, Laarbeeklaan 103, B-1090, Brussel, Belgium

    Received 19 February 1998; received in revised form 14 July 1998; accepted 14 July 1998

    Abstract

    A procedure to compare the performance (precision and bias) of an alternative measurement method and a reference method

    has been extensively described. It is based on ISO 5725-6 which has been adapted to the intralaboratory situation. This means

    that the proposed approach does not evaluate the reproducibility, but considers the (operatorinstrumenttime)-differentintermediate precision and/or the time-different intermediate precision. A 4-factor nested design is used for the study. The

    calculation of different variance estimates from the experimental data is carried out by ANOVA. The Satterthwaite

    approximation is included to determine the number of degrees of freedom associated with the compound variances. Taken into

    account the acceptable bias, the acceptable ratio between the precision parameters of the two methods, the significance level and the probability to wrongly accept an alternative method with an unacceptable performance, the formulae to determinethe number of measurements required for the comparison are given. For the evaluation of the bias, in addition to the point

    hypothesis testing, the interval hypothesis testing is also included as an alternative. Two examples are given as an illustration

    of the proposed approach. # 1999 Elsevier Science B.V. All rights reserved.

    Keywords: Comparison; Alternative measurement method; Bias; Precision; Repeatability; Time-different intermediate precision;

    (Operatorinstrumenttime)-different intermediate precision; Nested design; ANOVA; Satterthwaite approximation; Interval hypothesistesting

    1. Introduction

    When a laboratory wants to replace an existing

    analytical method by a new method (e.g. because

    the latter is cheaper or easier to use) it has to show

    that the new method performs at least as good as the

    existing one. A comparison of the performance (pre-

    cision and bias) of both methods has therefore to be

    performed. One of the most advanced guidelines for

    the comparison of two methods can be found in ISO

    5725-6 [1]. However the ISO guideline is based on

    interlaboratory studies and is therefore not applicable

    in the intralaboratory situation. Indeed within a single

    laboratory, the reproducibility, as evaluated by ISO,

    cannot be determined but intermediate precision con-

    ditions, such as changes in operator, equipment and

    time should be considered since they contribute to the

    variability of measurements performed in the labora-

    tory.

    In the ISO guideline the reference method is an

    international standard method that was studied in an

    interlaboratory test program and its precision (2) isassumed to be known. This assumption is reasonable

    since the precision is obtained from a large number of

    measurements. In the intralaboratory situation a

    Analytica Chimica Acta 391 (1999) 203225

    *Corresponding author. Tel.: +32-2477-4737; fax: +32-2477-

    4735; e-mail: asmeyers@vub.vub.ac.be

    0003-2670/99/$ see front matter # 1999 Elsevier Science B.V. All rights reserved.PII: S 0 0 0 3 - 2 6 7 0 ( 9 9 ) 0 0 1 1 5 - 4

  • laboratory has developed a first method and later on

    wishes to compare a new method to the older already

    internally validated method. For the latter, referred to

    as the reference method, only an estimate of the

    precision (s2) will be available since the precision is

    determined from a rather limited number of measure-

    ments. This of course determines the statistical tests to

    be used in the comparison of the performance char-

    acteristics of both methods.

    Moreover, the ISO standard is meant to show that

    both methods have similar precision and/or trueness

    whereas a laboratory that performs a method compar-

    ison study is interested to evaluate whether the new

    method is at least as good as the reference method.

    This implies that some two-sided statistical tests

    included in the ISO guideline are not appropriate

    for the comparison of two methods in a single labora-

    tory, where example in the evaluation of the precision

    one-sided tests have to be considered.

    In the decision making concerning the new alter-

    native method it is important (i) not to reject an

    alternative method which in fact is appropriate, and

    (ii) not to accept an alternative method which in fact is

    not appropriate. The former is related to the a-error ofthe statistical tests used in the comparison and is

    controlled through the selection of the significance

    level. The latter is related to the b-error and when it isconsidered it is generally taken into account by includ-

    ing sample size calculations. This approach is also

    included in the ISO guideline.

    In this article we propose an adaptation of the ISO

    guideline to the intralaboratory comparison of two

    methods. It is also applicable to the situation in which

    two laboratories of, e.g., the same organisation are

    involved, each laboratory being specialized in one of

    the methods. For the evaluation of the bias, in addition

    to the point hypothesis testing, interval hypothesis

    testing [2] in which the probability of accepting a

    method that is too much biased is controlled, is also

    included.

    Due to the specified acceptance criteria for the

    alternative method, the proposed approach might lead

    to a large number of measurements to be performed.

    An alternative approach (which will be described in a

    next article) is to perform the method comparison with

    a user-defined number of measurements and to eval-

    uate the probability that a method with an unaccep-

    table performance will be accepted.

    2. Methods

    All symbols and abbreviations used in this paper are

    defined in Table 1.

    2.1. Experimental design

    A 4-factor nested experimental design is used

    [37]. This design is also one of the designs recom-

    mended by ISO [3]. The schematic layout of the

    design is given in Fig. 1. The four factors represent

    four sources of variation that contribute to the varia-

    bility of the measurements within one laboratory. The

    factors considered are operator, instrument, time, and

    random error. The experimental approach can be

    described as follows. For each analytical method,

    the sample is analysed by m operators. Each operator

    performs, on each of q instruments, n replicated

    measurements on each of p different days. To avoid

    an underestimation of the day effect, the set of p

    different days during which the measurements are

    performed on each of the q instruments must be

    different, i.e. two instruments cannot be operated on

    the same day.

    Fig. 1. Schematic layout for the 4-factor nested experimental

    design applied. Only the nested structure under the ith operator, jth

    instrument and the kth day is shown here. The nested structure

    under other operators, instruments and days has the same pattern.

    (instruinstrument, repreplicate).

    204 S. Kuttatharmmakul et al. / Analytica Chimica Acta 391 (1999) 203225

  • Table 1

    Definition of symbols and abbreviations applied in the document

    d Absolute difference between the grand means obtained with two methods

    D Component of day effect in a test result

    E Random error component occurring in every test result

    FI(OIT) Calculated F-value obtained from the comparison of (operatorinstrumenttime)-different intermediate precision (variance)FI(T) Calculated F-value obtained from the comparison of time-different intermediate precision (variance)

    Fr Calculated F-value obtained from the comparison of repeatability variance

    FB ;A Value of the F-distribution with B degrees of freedom associated with the numerator and A degrees of freedom associated withthe denominator; represents the portion of the F-distribution to the right of the given F-value

    FA ;B Value of the F-distribution with A degrees of freedom associated with the numerator and B degrees of freedom associated withthe denominator; represents the portion of the F-distribution to the right of the given F-value

    I Component of instrumental effect in a test result

    m Number of operators

    M General mean (expectation) of the test results

    MS Mean squares

    n Number of replicates performed on each day

    n Average number of replicates performed on each dayN Total number of measurements

    O Component of operator effect in a test result

    p Number of days

    q Number of instruments

    s Estimate of

    s2 Estimate of 2

    tcal Calculated t-value obtained from the comparison of the means obtained with two methods

    t/2 Two-sided tabulated t-value at significance level and degrees of freedom

    t One-sided tabulated t-value at significance level and degrees of freedom

    UCL Upper confidence limit

    y Test resulty Grand mean of test resultsyi Arithmetic mean of the test results obtained from the ith operatoryij Arithmetic mean of the test results obtained from the ith operator and the jth instrumentyijk Arithmetic mean of the test results obtained from the ith operator, the jth instrument and the kth dayyijkL Particular test result related to the Lth replicate of the kth day, the jth instrument and the ith ope

Recommended

View more >