on the theory of meaningfulness of ordinal comparisons in measurement

Fundamental measurement concepts

On the theory of meaningfulness of ordinal comparisons in measurement F. S. Roberts

Department of Mathematics, Rutgers University, New Brunswick, New Jersey 08903, USA

This paper describes the concept of meaningfulness of statements using numerical scales. It then analyses the meaningfulness of ordinal comparisons, assertions that one item's assigned scale value is larger than that of another. It presents conditions under which ordinal comparisons are meaningful and meaningless.

1 The concept of meaningfulness

If my credit card number is 213414 and yours is 426828, no one would think of saying that yours is twice as large as mine. That is a meaningless assertion. Similarly, if the licence plate number on my car is 2112 and that on yours is 6336, no one would want to say that my plate number divides yours. This, too, is meaningless. One of the goals o f measurement theory has been to understand what statements we can meaningfully make using a numerical scale of measurement. This paper briefly discusses the literature on meaningfulness, and then concentrates on developing a theory for the meaningfulness of ordinal comparisons, namely, assertions that one item's assigned scale value is larger than that of another.

This approach to meaningfulness is based on the literature of measurement theory. See Krantz et al (vol 1, 1971; vol 2, in press), Pfanzagl (1968) or Roberts (1979a) for a summary. In particular, see Roberts (1979a) for all unde- fined terminology.

Suppose f is a homomorphism from one relational system A to another B. We call (A, B, f ) , or f alone when A and B are understood, a scale. It is called a numerical scale if the set underlying B is a set o f real numbers. The theory of meaningfulness is based heavily on S. S. Stevens' theory of scale type, and on the notion that the properties of a scale are captured by studying admissible transformations of scale, transformations which lead from one accept- able scale to another. (It is important to specify that admissible transformations are defined on the range of scale values. Thus, if (A, B, f ) is a scale and A is the underlying set of A, an admissible transformation is defined on f (A) . ) As Roberts and Franke (1976) point out, the theory of scale type based on the notion of admissible transformation can lead to ambiguities unless all o f the scales in question are regular in the following sense:

I f f and g are two such scales, there is a transformation ~b so that g = ~ of. In particular, if a numerical scale f is regular, it is called an ordinal scale if the class o f all admissible transformations corresponds to the class of monotone increasing functions on f (A) , the range o f f ; an interval scale if the class o f all admissible transformations is the class o f all linear transformations 4~(x) = t~x +/3, tx > 0, on f ( A ) ; and a ratio scale if the class of admissible transformations is the class o f all similarities ¢(x) = a x , a > 0, on f (A) .

A theory of meaningfulness, based largely on the theory o f scale type and admissible transformation, has been

developed by Adams et al (1965), Falmagne and Narens (1983), Luce (1978), Narens (1981), Pfanzagl (1968), Roberts (1979a, 1980), Roberts and Franke (1976), Robin- son (1963), Suppes (1959), and Suppes and Zinnes (1963). In general, we say that a statement involving numerical scales is called meaningful if its truth value remains unchanged if every scale or homomorphism (A, B, f ) is re- placed by another scale or homomorphism (A, B, g). Meaningfulness can usually be studied by considering admissible transformations of scale. In particular, Suppes (1959) and Suppes and Zinnes (1963) say that a statement involving numerical scales is meaningful if its truth or falsity is unchanged after any (or all) of the scales is trans- formed independently by an admissible transformation. Thus, for example, the statement that your credit card number is twice as large as mine is meaningless, because a change by one number in the last digit of your card number - certainly an admissible transformation - could change the statement from a true one to a false one. And the statement that your licence plate number divides my licence plate number is meaningless because, again, a small change in the numbering system, such as using five digits instead of four, could change this from a true statement to a false one.

Notice that meaningfulness is not the same as truth; a false statement can be meaningful, so long as all statements obtained from it by admissible changes of scale are false. For example, the statement that you weigh more than the Eiffel Tower is false, independent o f what scale is used to measure weight, and hence is a meaningful statement. The notion of meaningfulness is concerned with what assertions it makes sense to make, and which are just artifacts of the particular version of the scale of measurement you happen to be using.

Let us give some further examples. According to the Suppes-Zinnes definition, it is meaningful to say that the Eiffel Tower weighs one million times what you weigh. For if this statement is true (or false) in one scale of weight such as pounds, it is also true (or false) under all scales derived by admissible transformations, namely multiplica- tion by a positive constant. In general, the statement ' a ' is a million times as large as 'b ' , or twice as large as 'b ' , etc, is meaningful if we deal with a ratio scale. However, it is not meaningful if we deal with an interval scale. For example, the statement ' a ' is twice as warm as ' b ' is meaningless, since it might be true on the Centrigrade scale, while being false on the Fahrenheit scale. See Batchelder (in press), Luce (1978), Pfanzagl (1968), Roberts (1979a, b; in press)

Measurement Vol 2 No 1, Jan--Mar 1984 35

Roberts

and Suppes and Zinnes (1963) for a variety of applications of this theory of meaningfulness.

As Roberts and Franke ( t976) point out, the Suppes Zinnes definition of meaningfulness agrees with the more general definition if all scales in question are regular. How- ever, it leads to trouble in situations where there are irregu- lar scales: it can lead to seemingly meaningless statements being called meaningful. Hence, we shall adopt the more general definition here. While there seems to be good agree- ment on the spirit of our definition of meaningfulness, there is disagreement on the details. Luce (1978), Roberts (1980), Narens (1981), Falmagne and Narens (1983), and others consider alternative refinements and generalisations. Narens (1981) believes that there is no 'correct ' concept of meaningfulness; the choice of the meaningfulness concept will depend on the intended application. Falmagne and Narens study what they describe as numerical coding of empirical data, and describe meaningfulness as the property that the information content in such numerical coding be unaffected by an admissible change of the scales. (They provide a more technical definition, too.) Falmagne and Narens are strongly influenced in their approach, as are we, by a fundamental paper of Luce (1959) in which he argues that, in the formulation of scientific theories or laws, admissible transformations of one or more of the independent variables should lead only to admissible transformations of the dependent variables. This principle leads to sig- nificant limitations on the forms of scientific laws. Luce's approach has been criticised by Rozeboom (1962a, b). See the reply by Luce (1962). The approach to meaningfulness taken by Falmagne and Narens is in the spirit of the definition we have given. However, they point out that our definition must be applied with care, for if it is interpreted casually, it could lead to serious misunderstanding.

Before closing this section, let us address the question: must every scientific assertion be meaningful? It is not clear whether the answer is 'yes ' or 'no ' . However, as Falmagne and Narens (1983) point out, the scientific community has a strong preference for assertions that are meaningful, for a variety of reasons. First, from a purely practical point of view, it is much easier to state laws or conclusions without specifying the particular scales used. Second, laws or conclusions which are stated in general terms, but which really depend on the particular scales employed, can be mis- leading unless the dependence on the scale is made explicit. It would quickly lead to chaos in the scientific effort if the form of a law depended on the scales used. Finally, if the assertion is used in decision making, and if a decision based on a numerical scale can be reversed by simply modifying the numbers, and if both sets of numbers legitimately repre- sent the decision problem, then we have no good reason to make one decision rather than the other. For these various reasons, it seems important and natural to study the meaningfulness problem.

2 W h e n is f(a) > f(b) m e a n i n g f u l ?

A systematic investigation of meaningful statements might proceed as follows. Suppose f is a homomorphism from A into B and S is a statement involving f. Under what conditions on B is S meaningful? Typical statements S might be such ordinal statements as: f (a) > f (b) , f (a) is the largest element of f ( A ) , f(a) is between f (b ) and f (c) . Other statements would be: f (a )= 2f(b), f (a )= k f (b ) , k constant, f (a) - f ( b ) > f (c ) f (d) , and so on.

In this paper, we illustrate the types of results which

36

Fundamental measurement concepts

would be useful by beginning the systematic investigation of the meaningfulness of the comparison f ( a )> f(b). In particular, we study relational systems B = (Re, S), where S is an M-dry relation. It will always be understood that M > 1. In this section, we consider the special case when S is also M-point homogeneous, a concept introduced into the literature of meaningfulness by Falmagne and Narens (1983) and by Luce and Narens (in press). In the next section, we make a remark about a case where M-point homogeneity fails.

We start by introducing some fundamental definitions. We say that (Re, S) is M-point homogeneous if, whenever Xl > x2 > . . . > xM and ),~ > Y 2 > - - • > YM are real numbers, there is an automorphism 0 of (Re, S), a one-to-one homomorphism from (Re, S) into (Re, S), so that O(xi) = Yi, i = 1, 2, . . . , M. (Note: In this paper, automorphisms do not have to be 'on to'.) If" M is a positive integer, M > 1, let M = 11,2 . . . . . M}. Aranking e l m is a strict weak ordering. We shall usually denote rankings by giving them from highest to lowest, with a dash indicating a tie. For instance, the ranking 3 , 4 5, 2, 1 , 6 h a s 3 highest, 4 and 5 tied for second, 2 third, and so on.

Each M-tuple (xl , x : , . . . ,xM) from Re corresponds to a ranking rr of M. We rank first all of those i such that for no j is x / > x i. Having ranked il, i2 . . . . . ip, we rank next all those i such that for no j different from il, i2 . . . . . ip, is x / > x i. For instance, the 5-tuple (2, 6, 6, 8, 3) has 4 ranked first, because x4 = 8 is the highest entry. Then 2 and 3 come second, because x 2 = 6 and x 3 = 6 are the next highest entries. Since xs = 3 is next, 5 comes third. Finally, 1 is last because x~ = 2 is lowest. The corresponding rr is 4 , 2 - 3 , 5 , 1 .

Conversely, suppose lr is a ranking of M. Let the M-dry relation T ~ On Re be defined to consist of all M-tuples (xl , x2 , . . . , xa4 ) whose corresponding ranking is rr. For instance, i fMis 2 and rr is l, 2, then T ~ = {(x~,x2) ~ Re x Re; Xl>X2}, ie, T ~ is the > relation on Re. Similarly, rr = 2, 1 corresponds to the < relation, and lr = 1-2 to the = relation.

Theorem 1

Suppose S is an M-dry relation on Re and S is M-point homogeneous. Then for all rankings ~ of M, either T ~ ~ S o r T M ~ S = O .

Proof It suffices to show that i f x = (xl , x2 . . . . . xM) and Y = (Yx,Y2 . . . . ,YM) are in T~, then

( x l , x 2 . . . . . X M ) ~ S iff ( Y l , Y 2 . . . . . y M ) ~ S.

If we order the elements x i in non-decreasing order, we get xi, > xi~ > . . . > xi. • Since x and y are both in T~, it follows that y i>=Yi i~ . . . >.Vim, with strict > or = in exactly the same places. By M-point homogeneity, there is an automorphism ~b of (Re, S) such that for a l l / , O(xii) = Yij, ie, for all i, O(Xi) =Yi. It follows that

( x l , x 2 . . . . . X M ) ~ S iff [ 0 (x , ) , 0 (x2 ) . . . . . ~XM) ] ~ S

iff ( Y l , J 2 . . . . . YM) ~ S. QED

Corollary 1.1 If 7rl, zr2, . . . , are all the rankings of M and if S is an M-point homogeneous, M-dry relation on Re, then

. . . .

where (TM) ' is T M or O.

Measurement Vol 2 No 1, Jan--Mar 1984

Fundamenta l measurement concepts

TABLE 1: The possible binary relations (*)

Corre- Corresponds Relation sponds to Relation to

T= , < T~'-2 U T~',, =<_ m~'-2 = T~,,t U T~-2 U T~, 1 Re X Re T 2 u T~-2 > ~,2 = ~

It should be noted that the converse of Corollary 1.1 holds as well. That is, if an M-ary relation S has the form (*), then S is M-point homogeneous. Indeed, every monotone increasing 4~ is an automorphism for any such (Re, S).

In case M = 2, there are eight possible relations (*). These are given in Table 1. In case M = 3, we have such important relations as betweenness, which corresponds to T3,2,3 u T3,2,1.

As an aside, we note that Corollary 1.1 helps us to classify the possible representations which give rise to ordinal scales.

Theorem 2

Suppose f: ( A , R ) ~ (Re, S) is a homomorphism and S is an M-ary r.dation. Suppose f is a (regular) ordinal scale with I f ( A ) [ >-- M. Then S is of the form (*).

Proof It suffices to show that S isM- point homogeneous. To do this, it suffices to show that every monotone increasing ¢: Re-->Re is an automorphism of (Re, S). Given such a function ¢, let Xl, x2 . . . . . XM in Re be given. Then (x l , x2 . . . . . XM) corresponds to a ranking 7r of M. Since I f (A) [ ->_ M, we can find M numbers f (a l ) , f(a2) . . . . ,f(aM) So that [ f(al) , f(a2) . . . . . f(aM)] also corresponds to the ranking rr. Then there is a monotone increasing if: f (A ) Re so that ~[f(ai)] =xi. Because f is ordinal, f i e f is a homomorphism. Hence, so is O o (~ of). Thus, we have

[q~(X1), ¢(X2) . . . . , ~b(XM) ] ~ S

iff [¢o(ff o f ) (a l ) , qSo(ff of)(a2),

. . . . ¢)o(~ of)(aM)] ~ S

iff (ahas . . . . aM) ~ R

iff [(t~ of)(al) , (~ of)(a2) . . . . . (~7 of)(aM)] ~ S

iff (x~,x2 . . . . . XM)~S.

We conclude that q~ is an automorphism. QED

Corollary 2.1 Suppose f : ( A , R ) - + (Re, S) is a homomorphism and S is a binary relation. Suppose f is a (regular) ordinal scale with I f (A) [ > 2. Then S is >, >, <, or _-<.

Proof The only S of the form (*) are given in Table 1. It is easy to see that for the remaining four S's, no homomorphism f w i t h I f (A) I >- 2 gives an ordinal scale. QED

Corollary 2.2 Suppose f : (A, R) -+ (Re, S) is a homomorphism and S is a binary relation. Suppose f is a (regular) interval scale. Then I f (A) I = 1.

Proof A proof analogous to that of Theorem 2, using M = 2, shows that if I f (A) I > 2, then S is one of the relations in Table 1. But for none of these relations is there a homomorphism f w i t h I f (A) I _2 2, so that f is an interval scale.

QED Note: If I f (A) [ = 1, then f can be an interval scale.

We now consider homomorphisrns from (A, R) into (Re, S), where the M-ary relation S has the form (*). We ask: for


Roberts

which such f is the comparison f(a) > f(b) meaningful? We give here the results for the case when M = 2 and when S consists of just one relation T M.

If 7r is a ranking, its height k is the number of distinct levels in rr. For instance, 1,2, 3 - 4 has height 3.

Theorem 3

Suppose 7r is a ranking of M of height k - > - 2. Suppose f is a homomorphism from (A, R) into (Re, T~'), and suppose I f ( A ) I >_- k. Then f(a) > f(b) is meaningful.

Proof Suppose g is any other homomorphism. Since I f (A) [ >_- k, we can find a set X of k different numbers in f (A) . Since k > 2, the numbers f(a) and f(b) can be included in X. Arrange the numbers in X in the order rr, using duplicates as necessary. Thus, there are al ,a2 , . . . ,aM in A so that If(a1), f(a2) . . . . . f(aM)] is in T M. Hence, (al, a2 . . . . . aM) is in R. This implies [g(al), g ( a2 ) , . . . , g(aM)] is in T M. Since f(a) and f (b) are in X, we have a = a i and b = aj, some i, 1". By definition of T M,

f(ai) > f(a]) iff g(ai) > g(aj).

Thus,

f(a) > f(b) iff g(a) > g(b).

We conclude that f(a) > f(b) is meaningful. QED

Corollary 3.1 Suppose 7r is a ranking of M of height k > 2. If there is one homomorphism f f r o m (A, R) into (Re, T M) with I f (A) I >-- k, then I s (A) I _-> k for all homomorphisms g.

Proof This is a corollary of the proof. QED

We next show that Theorem 3 is false for k = 1 or I f ( Z ) I < k.

Theorem 4

Suppose f is a momorphism from (A, R) into (Re, TM2-...-M), and I f (A) [ > 1. Then f(a) > f(b) is meaningless.

Proof TM2- . . . -M iS (X,X . . . . , x ) ; x ~ R e . Then if ¢ is any one-to-one transformation on f (A) , we have

(al, a2 . . . . . aM) ~ R

~=~ [f(al), f(a2) . . . . . f(aM)] e TM2-. . . -M

*=* [(¢ o f ) (a l ) , (~ o f)(a~) . . . . . (ep ° f)(aM)] ~ TM2-. . .-M.

Therefore ¢ is admissible. But since I f (A) I > 1, there are a, b such that f (a )>f (b) . Define ¢ by e l f ( a ) ] = f (b ) and ¢[ f (b ) ] = f(a) and ¢ ( x ) = x otherwise. Since (Oof)(a)< (¢ of)(b) , f(a) > f(b) is meaningless. QED

Theorem 5

Suppose that n is a ranking of M of height k > 3. Sup- pose f is a homomorphism from (A, R) into (Re,TM), and suppose I f (A) 1 < k. Then f(a) > f(b) is meaningless.

Proof Since I f ( A ) l < k, it follows that for all al,a2 . . . . . aM in A [f(al) , f (a2) , . . . , f (aM)] could not be in T M. Hence, (aa, a2 . . . . . aM) could not be in R, and so R = 0. Note that by Corollary 3.1, I g (A) l < k for any other

37

Roberts

homomorphism g. Note also that any function g: A-+Re with I g (A) l < k is a homomorphism, since [g(al), g(a2), . . . . g(aM)] could never be in T~ M and since R = 0.

If f(a)> f(b), let a =f(a) , /3 =f(b) . Let ~(a) = j3, q~(13) = a, O(x) =x otherwise. Then g = ~ o f is a homomorphism since [ g(A) [ < k. But g(a) < g(b). Hence, f(a) > f(b) is meaningless. Suppose not f(a) > f(b). If f(a) < f(b), then in the same g, g(a) > g(b), so again f(a) > f(b) (though false) is meaningless. If f(a) =f(b) , let a =f (a ) and let 13 be any real number less than a. Define g(b) =/3 and g(u) = a for u 4=b. Then g is a homomorphism because [g(A) l < k. Moreover, g(a)>g(b). Thus we have shown that f(a)> f(b) (though false) is meaningless. QED

To illustrate this theorem and proof, we take A = Re, R = 0, M = 3, and zr = 1, 2, 3. Then taking f ( x ) = 0 if x < 0 and f (x ) = 1 if x > 0 defines a homomorphism. The function

defined by ~(0) = 1, ~(1) = 0 is an admissible transformation. We have f (2 ) > f ( - 1), while (4~of ) ( - 1)> (~ o f ) (2 ) .

Theorems 3 - 5 do not handle the cases I f (A) l = 1, k < 2. In these cases, by Corollary 3.1, every homomorphism has ]g(A)I < 2, ie, [g(A) t = 1. Thus, for all homomorphisms f and g, f(a) =f(b) andg(a) =g(b). Thus, the conclusion f(a) > f(b) is meaningful.

3 The case of semiorders

Relations S of the form (*) do not define all relational systems (Re, S) into which there is a homomorphism f with f(a) > f(b) meaningful. We illustrate this point by considering the case of semiorders.

Let 8 be a fixed positive number. Define S to be the binary relation >~ on Re, where x >8 Y i f fx > y + 8. I fA is finite, the binary relations (A, R) homomorphic to (Re, >~) are exactly the relations called semiorders. (See Roberts 1979a for definitions.) Note that (Re, >~) is not M-point homogeneous for M = 2. For instance, if 8 = 2, then 5 >~ 1, but not 3 >~ 2. Thus, although 5 > 1 and 3 > 2, there is no automorphism of (Re, >~) which takes 5 into 3 and 1 into 2. We conclude, by the remark after Corollary 1.1, that >~ is not of the form (*).

If f is a homomorphism from (A, R) into (Re, >6), then f(a) > f(b) can be meaningless. For instance, suppose 8 = 1, A =/2, 0.1, 0.2}, and R = >1. Then two homomorphisms into (Re, >1) are given by f(2) = 2, f (0.1) = 0.1, f (0.2) = 0.2, g(2) = 2, g(0.1) = 0.1, g(0.2) = 0. We have f(0.2) > f(0.1), but g(0.2) <g(0.1) . The conclusion f(a) > f ( b ) is not meaningful. There are, however, situations where this comparison is meaningful. If a and b are in A, let us say aEb holds if for all c, arc iff bRc and eRa iff cRb. Then it is easy to show that E is an equivalence relation on A. We have the following theorem.

Theorem 6

Suppose A is finite and f: (A, R) ~ (Re, >6) is a homomorphism. Then the comparison f(a) > f(b) is meaningful for all (a, b) in A × A iff for all x ~ y , ~ xEy.

Proof This follows easily from results of Roberts (1971) about the uniqueness of compatible simple orders. For details of a proof, see Roberts (in press). QED

Acknowledgements

The author thanks Arundhati Ray-Chaudhuri, Larry Harvey, and Jeff Steif for their helpful comments, and the National Science Foundation for its support under grant number IST- 83-01496 to Rutgers University.

38

Fundamental measurement concep ts

References

Adams, E. W., Fagot, R. F. and Robinson, R. E. 1965. 'A theory of appropriate statistics', Psychometrika, 30, 99 127.

Batchelder, W. H. In press. 'Inferring meaningful global network properties from individual actors' measurement scales', in Methods in social network methodology, L. Freeman, D. White, and A. K. Romney (eds).

Falmagne, J. C. and Narens, L. 1983. 'Scales and meaningfulness of quantitative laws', Synthese, 5 5 , 2 8 7 - 3 2 5 .

Krantz, D. H., Luce, R. D., Suppes, P. and Tversky, A. 1971. Foundations of" measurement, Vol. 1, Academic Press, New York.

Ktrantz, D. H., Luce, R. D., Suppes, P. and Tversky, A. In press. Foundations of measurement, Vol 2, Academic Press, New York.

Luce, R. D. 1959. 'On the possible psychophysical laws', PsycholRev, 66, 81-95 .

Luce, R. D. 1962. 'Comments on Rozeboom's criticisms of "On the possible psychophysical laws"', Psychol Rev, 6 9 , 5 4 8 - 5 5 5 .

Luce, R. D. 1978. 'Dimensionally invariant numerical laws correspond to meaningful qualitative relations', Philos Sci, 45, 1 - 16.

Luce, R. D. and Narens, L. In press. 'Classification of real measurement representations by scale type', Proceedings of the Workshop on Fundamental Logical Concepts of Measurement, Turin, 1983.

Narens, L. 1981. 'A general theory of ratio scalability, with remarks about the measurement-theoretic concept of meaningfulness', Theory and Deeision, 13, 1-70.

H'anzagl, J. 1968. Theory of measurement, Wiley, New York.

Roberts, F. S. 1971. 'On the compatibility between a graph and a semiorder', J Comb Theory, 11, 28 -38 .

Roberts, F. S. 1979a. Measurement theory, with applications to decision making, utility, and the social sciences, Addision-Wesley, Reading, MA.

Roberts, F. S. 1979b. Structural modelling and measurement theory', Technol Forecasting and Soc Change, 14, 353 365.

Roberts, F. S. 1980. On Luce's theory of meaningfulness', Philos Sci, 4 7 , 4 2 4 - 4 3 3 .

Roberts, F. S. In press. 'Applications of the theory of meaningfulness in measurement', Proceedings of the International Conference on Mathematical Psychology, Brussels, 1983.

Roberts, F. S. and Franke, C. H. 1976. 'On the theory of uniqueness in measurement', J Math Psychol, 14, 2 1 1 - 218.

Robinson, R. E. 1963. 'A set-theoretical approach to empirical meaningfulness of empirical statements, Tech Rept No 55, Institute for Mathematical Studies in the Social Sciences, Stanford University, Stanford, CA.

Rozeboom, W. W. 1962a. 'The untenability of Luce's principle', Psychol Rev, 6 9 , 5 4 2 - 5 4 7 .

Rozeboom, W. W. 1962b. 'Comment ' , Psychol Rev, 69, 522.

Suppes, P. 1959. 'Measurement, empirical meaningfulness and three-valued logic', in Measurement: definitions and theories, C. W. Churchman and P. Ratoosh (eds), Wiley, New York, 129 143.

Suppes, P. and Zinnes, J. 1963. 'Basic measurement theory', in Handbook of" mathematical psychology, Vet 1, R. D. Lute, R. R. Bush and E. Galanter (eds), Wiley, New York, 1-76.


on the theory of meaningfulness of ordinal comparisons in measurement

Documents