Z. Anal. Chem. 273, 177--181 (1975) �9 by Springer-Verlag 1975

Use of Information Theory in Method Comparison Studies St. Grys

Biochemistry Division, Laboratory of Veterinary Hygiene, Warsaw, Poland

Received May 31, 1974; revised October 1, 1974

Anwendung der Informationstheorie fiir den Vergleich yon Analysenverfahren. Zur kritischen Bewertung von Ana- lysenmethoden werden sechs Kriterien vorgeschlagen und definiert: Nachweisgrenze, Best/indigkeit, Genauigkeit, Pr~zision, Leistungsffthigkeit und Kostenfaktor. Diese ftir das betreffende Verfahren errechneten Werte ersetzen die Ausdriicke fiir Informationsmenge und Entropie. Die Vergleichswerte sollten noch durch redundante Infor- mation erg/inzt werden, die Auskunft fiber Empfindlichkeit, Widerstandsfghigkeit gegen chemische und physi- kalische Faktoren, Prgzision, Genauigkeit, Geschwindigkeit sowie Kosten des Verfahrens geben. Die redundante Entropie sollte als Indicator ffir den Nutzen des Verfahrens in der Routineanalyse dienen.

Summary. The usefulness of information theory for critical estimation of different analytical methods is shown. Six functional concepts named detection limit, firmness, accuracy, precision, efficiency and cost are defined. Their values calculated for any multicomponent analytical procedure are substituted for the expressions for the amount of information (I) and entropy (H). The obtained comparison results should be completed by redundant infor- mation, which is a more useful criterion for estimating the method due to the difficulty of interpreting absolute I and H values. The information redundancy value provides information about the sensitivity, resistance against chemical and physical agents, precision, accuracy, speed and cost of analytical procedure. The redundancy of entropy is an indicator of the usefulness of analytical method for routine analysis.

Analysenverfahren, Informationstheorie; Vergleichswerte.

1. Introduction

The criteria for acceptability of a method are varying in different laboratories, and therefore the standard methods of analysis are not widely accepted, It must be also emphasized that due to rapid development of modern analysis the present codification of ana- lytical procedures contains the danger of suboptimi- zation in the future. Thus, the best treatment is to appoint only the admissible limiting values of the variables and to leave the analyst to decide himself according to the rules of games theory. In practice an approach is to compare the new procedure with the "reference" technique, by determining their sensi- tivity and precision. If the tested method agrees well it is judged to be acceptable.

Unfortunately different definitions of the main factors encompassed in applicability and performance, and moreover different analytical techniques do exist. Additionally, many authors do not announce all criteria for acceptability and therefore it is very diffi- cult to compare in all respects various methods (e.g. gas chromatographic, spectroscopic, colorimetric, catalytic, etc.). These divergences prove the need of common mathematical tests in a method of comparison studies.

12 Z. Anal. C~em., Bd. 273

2. The Evaluation Scheme

The practical value of any analytical method depends on its sensitivity, selectivity, speed of analysis and cost of analytical procedure, including equipment and reagent needed, and personnel requirements. These factors are encompassed in the term of appli- cability of a method. Decision of acceptability depends on both applicability and performance, which considers the errors. Usually we evaluate performance in terms of precision and accuracy. Precision concerns the random error, accuracy the systematic error [10].

For selection of the proper definitions characterizing each determinant of acceptability it was taken into consideration that according to the information theory [1] the respective event (element) should fulfil the following criteria:

0 < P ~ < l (1)

and n

Z'e~ = 1, (2) l

and afford the possibility for comparing different techniques. In both expressions P~ states the probability of the event (i). It is obvious that because of these

178 Z. Anal. Chem., Band 273, Heft 3 (1975)

conditions, most "classical" terms of the events encompassed in both applicability and performance, including sensitivity, selectivity and accuracy, are useless for the calculations in a method of comparison studies. Therefore, five new functional concepts named detection limit (D), firmness (F), accuracy (A), effi- ciency (E) and cost (C), in addition to precision (P) defined recently by Westgard and Hunt [I0], were outlined. They encompass the criteria (simple events) resulting in judgment on the acceptability of a method. Because of condition (1) all events are expressed as decimal fractions and calculated as follows:

D - limit o f detection shows a concentration of an element or compound in a native sample, that gives the reading equal to twice the confidence half-interval of a series of t e n determinations of blank test value, determined with a 99 ~ certainty. The measure of this value is mg per kg (ppm). Thus we obtain formula:

D = 2L (3)

Well known is the confidence interval

/ i (3 a) n S ~ I ~- to~ n ( n - - l )

where t0~ denotes t value written out from the Student's t-table at the confidence level c~, and n--1 degrees of freedom (~). The other symbols mean: n =sample size, S---= standard deviation, and S 2 = variance. When appropriate values are substituted for symbols (3.25 for t0.01, and 10 for n), we obtain the integration formula:

D = 2.17 S mg/kg. (3 b)

This definition is near to that used in atomic absorption spectrometry and does fulfil Kaiser's [5] requirement to calculate the sensitivity of the analytical procedure as a whole.

F -- f irmness (resistance) provides information about the factors which affect results in different ways. We judge the firmness evaluating the interferences and conditions of chemical reaction. F -- value is calcu- lated as a total of deviations from expected value, caused by a presence of equimolar amounts of inter- fering substances or connected with the 5 ~ changes in optimum reaction conditions. The expression for firmness is the following formula:

F=~ v,--Vo 1 V, (4)

where differences between expected and obtained values (Ve--Vo) for individual factors are analysed without the signs -1- and --.

A - accuracy is interpreted in terms of recovery (A1) and reproducibility (As). Recovery is calculated

from different losses during the whole procedure ex- pressed as a percentage.

A1 = 2J l (l -- individual losses). (5 a) 1

Reproducibility reflects linearity of the method expressed as a straight portion of a curve in a Ringbom plot. The measure of this event is the ratio of ideal range to the full range, multiplied by 100 and raised to the negative first power. [,. ]1 A2 = f--.-~. " 100 (i.r. idealrange, f .r , fullrange) (5b)

In order to connect A1 and A~ with A we consider the formula

A = AI + .42. (5)

P - precision refers to random errors. Westgard and Hunt [10] stated that the most useful parameter for determining this error is the standard error of estimate (standard deviation of residuals) calculated from the equation:

i f .s" ay = n - - 1 (6)

(n ----- sample size; S 2 ----- variance; n - - 1 = number of degrees of freedom), ay is in units of concentration and that is why we evaluate it in term of relative standard deviation, which is unitless.

G cr 70 = - - (6b) /z

(/z = mean value of series of at least ten determi- nations).

E -- efficiency (laboriousness) yields an information about the time consumption during the whole procedure and is expressed as:

E = 10-9t (7)

(t = time of effective labour needed for one sample, in minutes).

C - cost shows the expenditure of materials and apparatus used for analysis of one sample by means of a new method in relation to the least expense method.

Cnew. c - 1 0 - 3 . ( 8 )

Clnexp.

In the case of equality of Chew. and Cinexp. C = 0.001. Higher values are indicators of a greater expenditure.

In addition to the separate evaluation of individual acceptability factors we can judge the set of information simultaneously for all events, i. e. evaluate the method as a whole. For critical estimation of analytical pro- cedure it should be taken into account both the amount of information (I) and the entropy (H), which con-

St. Grys: Use of Information Theory in Method Comparison Studies 179

stitute different measures of the same event. Thus we obtain two new formulae:

I = ZIb , (9)

where Ib = --3.322 lg Pt and denotes the amount of information and

H = --3.322 S P~" lg PC. (10) 1

According to equation (2) the detailed P~ values for entropy are calculated as percentages of a sum of all events. The expression (9) for /provides an information about the quality of data obtained by means of a new method and a "reference" method. Because of condition (1) one should reject the method which gives P~ values greater than 1.00. In compliance with the probability theory the method is more efficient, if the obtained P~ values are smaller. This implies that the better methods give the higher I values.

The quantity of entropy indicates the order of information. The high H value points to the equivalent contribution of all determinants of the method. The methods yielding the larger H values are more suitable for routine work.

The direct judgment on absolute values of amount of information and entropy makes some difficulties and therefore they should be completed by redundant information. Absolute redundancy makes a difference between the maximum and calculated information value.

RH --/-/max--/-/, and Rz = I m a x - L bit/event. (11, 12)

The relative redundancy represents the ratio of absolute to maximum redundancy

R~ RI (13, 14) rn ---- H=a-----7 and rz = I~x

and is expressed as a percentage or decimal fraction. In the case of equal probabilities of all events Hmax = 3.322 bit/digit, but customarily we write that Hmax = 4.0 bit/digit. This value when substituted to the ex- pression for the set of information (9) gives/max = z" 42 because the probability level of 0.0001 makes the limit for individual P~ values, and so we obtain /max = 96.0 for z events = 6,

The small values of information redundancy are indicators of acceptability for sensitivity, resistance, accuracy, precision, speed and cost. Analysis by redundance of entropy is usefull when decision on acceptability depends equally well on all criteria.

3. Examples of Estimation To give an example for acceptability evaluation via comparison studies we have chosen two different

procedures for iodide determination: catalytic [6,7] and gas-chromatographic [2--4]. The data essential for calculations are shown in Table 1.

Detection Limit

2.17.0.027 De -- 0.020 -- 2.93 ng/0.1 ml =0.029 ppm

Do = 2.17 x0.0005 = 0.001 ppm Detection limit calculated for the catalytic method (0.029 ppm) differs significantly from the value of 0.004 ppm given by Ke et al. [6]. Both quantities are larger than that ob- tained for GC technique, and so it appears to be more sensitive.

Firmness

F, = 0.00008 + 0.000008 + 0.000074 + 0.000074 + 0.000081 q- 0.00007 + 0.63 + 0.34 q- 0.015 q- 0.040 q- 0.0403 + 0.05 = 1.116 where the succeeding constituents of the sum represent the differences in the readout values, expressed as decimal fractions, caused by a presence of equimolar amounts of" Cu, Cr, Ni, Co, Zn, Fe, Hg and Ag and 5 Yo changes in: acidity, reagent concentration, reaction time and end volume. For example the value regarding the effect of various reagent concentrations was calculated as follows:

0.124 -- 0.126 0.124 -- 0.121 0.124 -- 0.016, 0.124 -- 0.024,

0 .124- 0.134 0.124- 0.119 0.124 -- 0.080, and 0.124 -- 0.040.

0.016 + 0.024 q- 0.080 § 0.040 Thus we obtain = 0.040

4 Fg = 0.000015 -}- 0.05 ----- 0.050.

The high F, value is to a great extent connected with the interferences caused by mercury and silver, because the F~ value calculated for these two elements amounts to 0.97, and for the remaining factors to 0.146. This implies that the catalytic method is applicable entirely to samples containing an iodide concentration much greater than that of Hg and Ag. On the contrary, the GLC technique is resistant to physical and chemical factors and the result depends mainly on the end volume of the tested sample.

Accuracy

[ 1 0 . 0 - 0 . 4 1 - 1 A~ = 0.046 q- 40 100 = 0.046 + 0.042 = 0.088

Ag cannot be fully calculated because of the lack of A, (recovery) value. A2 for the GLC method equals to 0.01.

Precision

P~ = V 60"593.52 -- 3.52 ~ = 0.0352, and Pg = 0.0246.

Both percentage values are converted to decimal fractions to fulfil condition (1). There is not a great bias in accuracy between the analysed methods, although the GLC technique appears to be more precise.

12"

180 Z. Anal. Chem., Band 273, Heft 3 (1975)

Table 1. Compiled Set of Comparison Data useful in Making Decisions on the Acceptability of Two Methods for Iodide Determination in Milk Samples

Term Values for

Catalytic Method (c) Gas-chromatographic Method (g)

Sensitivity Provides possibility for detection of Detection limit defined as in AAS 0.4 ng (absorbance = 0.008) in equals to 0.001 ppm. 0.1 ml sample.

Blank Test Value Absorbance (A) = 0.784 -t- 0.027. 0.014 4- 0,0005 ppm of iodide.

Interferences caused by a presence of Diverse Elements

In a sample containing 8 ng (0.063 nM) of iodide, 5 ~tg a m o u n t s of: Cu(II), Ni(II), Co(II), Zn(II) and Fe(III), and 40 ~xg of Cr(IIl) yield 10 ~o error. 10 ng of Hg(II) or Ag(I) give 50 ~ deviation.

Other halides do not interfere. The effect of metals was not tested.

Effect of Acidity Acidity of 0.155 N is a requisite for the maximum differential absorbance. 5 ~o changes in acidity cause a 1.5 % decrease in sensitivity.

The variation from 0.1 to 1.0 M of H + does not affect the amount of the reaction product.

Effect of Reagent Concentration At the molar ratio As(III): Ce(IV) of 2.0 differential absorbance of standard equals to 0.124. A 5 increase or decrease in concentration of As and Ce give the values of 0.126, 0.121, 0.134 and 0.119, respectively.

The amount of reaction product is stable when ketone concentration varies between 0.1 and 1.0 M.

Effect of Reaction Time The reaction time of 30 min was selected. The differential absorbances are lower or higher by 0.01 when the incubation period is by 3 min shorter or longer.

Reaction is completed within 5 min and extraction is rapid. Recommended conditions are: reacting 30 rain extracting 5 min. In samples incubated for 24 hrs, a 1.4 ~ decrease in the amount of reaction product is observed.

Recovery The mean difference between iodide Recovery was not tested. added and found amounted to 4- 4.6 ~ .

Reproducibility The changes in absorbance occur in the ranges from 0 to 40 ng. The straight line is obtained between 0.4 and 10 ng.

The detector response is linear in the full scale.

Precision

Time Consumption regarding 40 Samples

The relative standard deviation for a series of 60 determinations amounted to 3.5 ~o.

Protein precipitation with consequent filtration = 80 min. Adding of reagents, mixing, measuring = 120rain a, or 1500 min b. Incubations = 55 rain ~ or 30 rain b.

Relative standard deviation of residuals is equal to 2.46 ~ .

Adding of reagents = 60 rain. Incu- bation ----- 30 rain, centrifuging = 5 min, and extracting = 5 rain. Double injections to GC = 240 min.

Expenditure for Reagents and $ 0.6 ~, or $ 20 b $ 6.0 Apparatus for 40 Analyses

using stopping reagents, b kinetic method.

St. Grys: Use of Information Theory in Method Comparison Studies 181

Efficiency

80 + 120 + 55 E c =

40

80 + 1500 + 30 = 0.4025 b

40" 100

8 0 + 6 0 + 3 0 + 5 + 5 - t - 2 4 0 Eg = 40" 100 = 0.105.

1 100 = 0.0638 a, or

So, significant differences in time consumption exist between the GC method and catalytic techniques. The most laborious is the kinetic version of catalytic method.

nique because of the superior parameters. Only in the case of t he catalytic method using a stopping reagent the parameters regarding the speed of analysis and cost are more convenient and, therefore, as well as due to the small- est redundancy of entropy this method seems to be more applicable for daily use especially when large numbers of analysis are carried out. Of course, the reaction milieu should contain the Ag and Hg ions only in negligible amounts, as it was stressed in discussion of firmness. The relative large values of information redundancy and of entropy are pointing out that the present methods for iodide determination are not excellent and further searches are needed.

Cost

Cost was calculated in relation to the titrimetric method which is the most inexpensive.

6.0 1 Cg -- 0.1 1000 = 0.06, and Cc = 0.006 a, or Ce = 0.20 b

The kinetic version of catalytic method appears to be the most expensive method and that using a stopping reagent is relatively inexpensive.

When the gas-chromatographic method is compared with two catalytic techniques as a whole, we obtain the following values for amount of information, entropy and relative redundancy.

Ig = 3.322 (lg 0.001 -t- lg 0.05 -F �9 �9 �9 -b Ig 0.06) = 33.59

lc = 27.57 a, or Ic = 19.85 b

Hc = 2.166 a, or Hc = 2.098 and Hg = 2.031

rzc = 70.8 70 a, or rzo = 79.1 ~o b and rz, = 64.5 7o

rHo = 34.7 70 a, or rHo = 36.8 70 b and rH, = 38.8 Yo

From the obtained values fo r / , H and r it may be as- sumed that the gas-chromatographic method for iodide determination is more acceptable than the catalytic tech-

References

1. Gottschalk, G. : Z. Anal. Chem. 258, 1--12 (1972) 2. Grys, S.: J. Chromatog. 100, 43---48 (1974) 3. Hasty, R. A.: Mikrochim. Acta 1970, 348--352 4. Hasty, R. A. : Mikrochim. Acta 1972, 621--624 5. Kaiser, H.: diese Z. 260, 252--260 (1972) 6. Ke, P. J., Thibert, R. J., Walton, R. J., Soules, D. K. :

Mikrochim. Acta 1973, 569--581 7. Matthes, W., Kiss, T., Stoeppler, M.: diese Z. 267,

8 9 - 9 5 (1973) 8. Miller, D. W., Starr, M. K. : Executive Decision and

Operations Research. New Jersey: Prentice-Hall, Inc. 1969

9. Palm, E. : diese Z. 256, 25--27 (1971) 10. Westgard, J. O., Hunt, M. R.: Clin. Chem. 19, 49--57

(1973)

Doc. Dr. habil. Stanislaw Grys Laboratory of Veterinary Hygiene 21, Lechicka Str. 02-156 Warsaw Poland