a re-examination and attempted replication of. the

206
1 A RE-EXAMINATION AND ATTEMPTED REPLICATION OF . THE STRUCTURE OF CATTELL'S SIXTEEN PERSONALITY FACTOR QUESTIONNAIRE STEPHEN FREDERICK BLINKHORN Ph. D. Institute of Education University of London

Upload: others

Post on 24-Feb-2022

3 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

1

A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE STRUCTURE OF CATTELL'S

SIXTEEN PERSONALITY FACTOR QUESTIONNAIRE

STEPHEN FREDERICK BLINKHORN

Ph. D. Institute of Education University of London

Page 2: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

2

ABSTRACT

A RE-EXAMINATION AND ATTEMPTED REPLICATION OF THE STRUCTURE OF CATTELL'S SIXTEEN PERSONALITY FACTOR

QUESTIONAIRE

Stephen Frederick Blinkhorn

Cattell's Sixteen Personality Factor Questionnaire is amongst the most widely used of psychometric measures of personality, yet little support for its structure is to be found in the critical literature. Consideration of what might constitute a decisive analysis leads to the view that previous studies have been deficient in concentrating unduly on technical criteria in the conduct of factor analysis to the exclusion of an analysis of psychometric and statistical features which could give rise to observed results.

Using a large sample of undergraduate subjects, both within-scale and between scale analyses were performed. The claim that the scales of the questionnaire are heterogeneous, accepted by Cattell, is shown to be invalid in statistical terms, with a few exceptions. Evidence for a limited but substantial degree of support for the structure proposed by Cattell is adduced from both traditional forms of factor analysis and the newer technique of structural analysis of covariance matrices. In the process doubt is cast on the relevance of certain technical disputes regarding the conduct of factor analysis, and the value of Cronbach's coefficient alpha as an index of psychometric adequacy is supported. The importance of reliability of variables in factor analysis is emphasised, and attention is drawn to the problems associated with the use of heterogeneous samples and ad libitum selection of variables.

Consideration of the results of the analyses suggests a reinterpretation of Cattell's scales in terms of homogeneity of content and possibly a facet analysis of the questionnaire synthesising a variety of viewpoints often presented as conflicting.

Page 3: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

3

CONTENTS

Page No.

Abstract 2

List of Tables 4

List of Figures 6

Acknowledgements

Introduction 8

Chapter 1 11

Chapter 2 19

Chapter 3 30

Chapter 4 36 The Treatment of Error 39 Sampling of Subjects 48

Sampling of Variables 50 Choice of Matrix 54 Variable Type 57 Concluding Comments 63

Chapter 5 The Data Base 64

1. airripling and Administration 2. Size and Match of Sample 67 3. Internal Evidence of Quality 69 4. Initial Analyses 75

Chapter 6 80

Chapter 7 Introduction to Factor Analyses 89 Unrestricted Factor Analyses 91

Chapter 8 111

Chapter 9 134

Chapter 10 161

Appendix of Tables 168

References 199

List of the Factor Scales of the 16PF foldout inside

back cover

Page 4: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

4

LIST OF TABLES

Table Page No.

1 Numbers of protocols returned 68

2 Percentages of university population of second- 68 year undergraduates and of sample by region and size/type classification

3 Chi-squared tests for fit of sample to population 70

4 Standard deviations of raw scale scores by sex 72 in three standardisation samples, for forms A and B of the 16 PF combined

5 Means and standard deviations of forms A-+B 77

scale scores

6 Indices of internal consistency and reliability for 85

the primary factor scales

7 Factor-pattern elements for the seven factor 92

solution

8 Test statistics for maximum likelihood factor 104

analyses of data for four sex x discipline groups

9 Test statistics for maximum likelihood factoring 105

of the pooled within-groups correlation matrix

10 Factor-pattern elements for the seventeen-factor 106 solution

11 The ten groups and combinations of groups, with 114

sample sizes, for the restricted analyses

12 Estimated factor-pattern coefficients for the ten 118 SIFASP analyses

13 Factor covariance (lower triangle) and correlation 120 (upper triangle) matrix for male arts group

14 Test statistics for the ten SIFASP analyses 123

15 Tucker and Lewis "reliability" coefficients•for 125 the restricted analyses

16 Residual covariances from pooled within-group 127 restricted analysis

17 Unique factor loadings from the ten restricted 130 factor analyses

18 Reordered factor correlation matrix, with 0, Q2 135 and Q4 reversed in sign, for the total group

19 Correlation matrix for the total group after 138 multivariate correction for attenuation

Page 5: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

5

LIST OF TABLES

(font' d. )

Table Pale No.

Al Covariance matrix for male arts group 169

AZ Covariance matrix for male science group 171

A3 Covariance matrix for female arts group 173

A4 Covariance matrix for female science group 175

AS Covariance matrix for male total group 177

A6 Covariance matrix for female total group 179

A7 Covariance matrix for arts total group 181

A8 Covariance matrix for science total group 183

A9 Pooled within-group covariance matrix 185

A10 Covariance matrix for total group 187

All Factor covariance (lower triangle) and 189 correlation (upper triangle) matrix for male arts group

Al2 Factor covariance (lower triangle) and 190 correlation (upper triangle) matrix for male science group

Al3 Factor covariance (lower triangle) and 191 correlation (upper triangle) matrix for female arts group

A14 Factor covariance (lower triangle) and 192 correlation (upper triangle) matrix for female science group

Al5 Factor covariance (lower triangle) and 193 correlation (upper triangle) matrix for male total group

A16 Factor covariance (lower triangle) and 194 correlation (upper triangle) matrix for female total group

All Factor covariance (lower triangle) and 195 correlation (upper triangle) matrix for arts total group

A18 Factor covariance (lower triangle) and 196 correlation (upper triangle) matrix for science total group

Al9 Factor covariance (lower triangle) and 197 correlation (upper triangle) matrix for pooled within-group analysis

A20 Factor covariance (lower triangle) and 198 correlation (upper triangle) matrix for total group

Page 6: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

6

LIST OF FIGURES

Figure Page No.

1 Eigenvalue plot for the total group 93

2 Eigenvalue plot for the male arts group 96

3 Eigenvalue plot for the male science group 97

4 Eigenvalue plot for the female arts group 98

5 Eigenvalue plot for the female science group 99

Page 7: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

7

ACKNOWLEDGEMENTS

This thesis could not have been written without the

cooperation of rather more than 1,000 anonymous individuals

who began undergraduate studies in 1971, and of the careers

services of most of the universities in the United Kingdom.

The staff of the Hatfield Polytechnic Computer Centre showed

remarkable patience and tolerance in allowing me on frequent

occasions to monopolise their facilities and bore them with

idiosyncratic problems. Professor Harvey Goldstein and

Dr. Mike Berger never failed to encourage and, respectively,

served to remind me that psychologists have no monopoly on

statistics, and that there is more to psychology than statistics.

Dr. Peter Saville has been a valued adversary and colleague,

and Dr. Tony Gibson has contributed perhaps more than he will

recognise to the development of the ideas which structured this

thesis. Particular thanks are due to Linda Brassington for

undertaking the typing with remarkable accuracy and despatch.

Finally, my wife Jenny has been truly biblical in her support:

"She openeth her mouth with wisdom; And in her tongue is the law of kindness. "

(Ecclesiastes 31, xxvi)

Page 8: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

8

INTRODUCTION

Cattell's Sixteen Personality Factor Questionnaire is

amongst the most widely used of all psychometric techniques for

the assessment of personality, yet its structure has consistently

failed to find support in the results of independent research.

This thesis is an attempt to test that structure directly, and

results from the author's view that the crucial analyses have not

elsewhere been carried out, nor has sufficient consideration been

given to the problems of sample homogeneity and the reliability of

variables in the use of factor analysis on personality test data.

The primary emphasis of the thesis is psychometric and

confined to the questionnaire and its interpretation, and the order

of discussion has been arrived at in an effort to minimise

repetition and the discussion of methodological issues and

empirical results simultaneously. As this order is not entirely

conventional a brief guide to it may help the reader follow the

line of argument.

Chapters 1 and 2 deal respectively with the historical

context of the development of the questionnaire and some of the

issues involved in the use of trait terms in the description of

personality and the explanation of behaviour. Chapter 3 outlines

the commonest criticisms that have been made of the questionnaire

Page 9: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

9

and its construction. Chapter 4 is largely concerned with

methodological issues in factor analysis, pointing out the problems

which may arise from sample heterogeneity and unreliability of

variables, and developing the approach to analysis adopted for this

study. Chapter 5 contains an account of the fieldwork. In

Chapter 6 the question of scale homogeneity is discussed in the

light of the general claim that Cattell's scales are notably

heterogeneous: with some exceptions this claim is shown to be

ill-founded. Chapters 7 and 8 contain accounts of a number of

factor analyses, both unrestricted and restricted, which tended

to produce mutually consistent results confirming those of

Chapter 6. Chapter 9 begins with a discussion of certain

features of the factor correlation matrices from the analyses of

Chapter 8, shows that similar results arise from a multivariate

correction for attenuation of the scale inter correlaions, and then

proceeds to an inspection of the item content of certain scales,

followed by a sketch of a possible account of the structure of the

16PF in terms of content analysis of items. Chapter 10 then

summarises and reviews the empirical results.

What is presented is not therefore an account which claims

to be definitive confirmation or disconfirmation of the notional

structure of the questionnaire. Rather it is shown that with a

fairly homogeneous sample a moderately good approximation to

Page 10: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

10

that structure can be found, but that the departures from the

notional structure may in themselves be informative and lead to

a positive reappraisal of personality questionnaires.

Page 11: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

II

CHAPTER 1

In many respects attempts to construct questionnaire

measures of personality have been strongly influenced by develop-

ments in the measurement of mental ability. By the mid-1930's,

intelligence tests had taken a form substantially similar to that of

those currently in use. Development in this area had been aided

by the emergence of correlational techniques, and perhaps most

notably of factor analysis, which had been applied to the study of

differential aptitudes with some success.

Attempts to pursue mental testing into the area of

personality were perhaps inevitable, and indeed sufficient

development had taken place for Cattell (1933) to publish a review

article, and Guilford and Guilford (1934) to undertake one of the

first factor-analytic studies of extraversion. However, one

difference between ability and personality tests has particular

implications for the application of psychometric methods, in that

in general the former, being tests of maximum performance, may

be expected to correlate with measures of external criterion

performances where there is general agreement as to what

constitutes a better or more correct level of performance. Thus

the construction of the original Binet test could proceed without

any guiding theory as to the fine structure of abilities, and later

Page 12: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

12

work could build from the notion of undifferentiated ability to

finer-grained structures.

In the case of personality tests, however, the question of

the dimensionality of personality is more pressing. It is by no

means clear that a single scale, say ranging from inadequate to

adequate personality, has the same kind of utility, even as a

starting point, for the development of measures of personality.

Indeed, articulated accounts of the structure of personality existed

before the development of psychometric measures, most notably

perhaps Galen's typology, adopted also by Kant, and the theories

of psychoanalytically and psychiatrically oriented authors. Given

the lack of clear criterion performances, the development of

personality questionnaires was therefore more dependent on the

theoretical orientation of test authors, and indeed between the two

world wars they proliferated.

Cattell's (1946) initial attempts to formulate a factor-based

theory of personality may be seen as an attempt to bring order

into a confused field in which the fact that two scales shared a

name was no guarantee that they measured related traits.

Furthermore, in taking as his starting-point the list of trait words

extracted from the dictionary by Allport and Odbert (1936), he

implicitly rejected all theoretical approaches other than those

embedded in the language. His stand was to assume that the

Page 13: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

13

English language contained terms for all the distinctions that had

been found valuable in the description of personality, and that

appropriate methods of analysis applied to scores on rating scales,

questionnaires and objective performance measures would yield a

widely, or even universally, applicable framework for the

development of a definitive theory of human personality. It is

more than clear that he continues to hold this view, and also the

opinion that his methods of analysis have at any given time been

the most appropriate then available for this purpose (Cattell 1973).

Essentially, then, the only important theoretical

assumption made by Cattell at the outset was that personality is

properly conceived of in terms of traits, although, to anticipate

the discussion in the next chapter, the precise meaning he

attaches to the term is not altogether clear. Such an assumption

was at the very least quite in keeping with the historical context

in which it was made, especially given the apparent success of

Thur stone and the London school, in their different ways, in

giving a dimensional, trait-like account of the structure of mental

abilities. Amongst those who accept this assumption as

reasonable, then as now criticism of Cattell's work has centred on

detailed matters of techniques of analysis and indices of

psychometric adequacy, and it is with these matters that this

thesis is principally concerned.

Page 14: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

14

Before passing on to an outline account of the development

and present state of Cattell's theory, there are two points which

give rise to occasional misunderstanding, and are best settled in

advance. First, although Cattell is perhaps best known to

psychologists in general for his work on the development of

personality questionnaires, originally he did not regard them as

having a long-term future as first-rank measures. Recognising

the imperfections of the questionnaire method, such as its

susceptibility to faking, he expected the technique to fall out of use

as more objective measuring devices were developed. Secondly,

there is a matter of terminology: Cattell uses the terms 'primary

factor' and 'source trait' interchangeably. Source traits are

seen as the fundamental variables of personality, discovered by

factor analysis of 'surface traits', which are clusters of observed

behaviour, somewhat analogous to pathological syndromes in

medicine. According to Cattell, source traits cause surface

traits, and bear the burden of explanation insofar as individual

differences in personality are responsible for differences in

behaviour. Thus the term 'primary factor' has ontological

rather than merely operational significance. What is a first-

order factor from the point of view of the conduct of a factor-

analytic investigation is not necessarily a primary factor in this

sense. Cattell has attempted to popularise a distinction between

factor orders (operational significance) and factor strata

Page 15: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

15

(ontological significance), but with little success to date.

The systematic account of the 'primary factors of

personality' appearing in Cattell (1946) is based on behaviour

rating scales. Working with a reduced list of trait words drawn

from Allport and Odbert (1936), 100 subjects were rated on 171

bipolar scales, and clusters were identified. On the basis of

this analysis, a smaller set of scales was factor analysed to yield

12 'primary factors' or 'source traits'. Cattell then proceeded

to match these factors with those found in the analysis of

questionnaire responses, and made tentative identifications in

the domain of physiological and performance measures.

Howarth (1976) has presented a critical reanalysis of the

original behaviour rating study, and raises the possibility that

inaccuracies and inadequacies in this study had a detrimental

effect on the eventual published questionnaires Cattell produced.

However the link between this first exposition and the current

versions of the questionnaires is not as rigid as might be

supposed. In particular certain factors, e. g. D, J and K, are

not represented in the standard adult versions, and as

successive revisions of the scales have appeared there has been

a certain amount of reinterpretation of the meanings of factors

as well as claimed psychometric improvements of the scales

which represent them.

Page 16: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

16

The original 12 factors proposed by Cattell were labelled

by letters, A to L, in descending order of the proportion of

variance accounted for by each. From the first, Cattell has

preferred oblique factor solutions on the grounds that the real

causal influences he sought to identify by factor analysis were

unlikely to be uncorrelated, even though 'functionally independent'.

Subsequently, (Cattell 1950), he extended his investigation of the

factors to be found in questionnaire response data, to yield the

structure now represented in his Sixteen Personality Factor

Questionnaire (16PF). As has been mentioned, three of the

original factors are missing, but three others have been added

which are claimed to have matches in rating data, and four

additional factors are included which have only been found reliably

in questionnaire data, and hence are labelled Q1 to Q4 .

Since the factors are correlated, factor analysis of their

intercorrelations is possible, and hence second-order factors

may be derived. Four of these Cattell claims to be well-

established, and four or five others tentatively identified

(Cattell, Eber and Tatsuoka 1970). Of these, the first two have

received particular attention, since they appear to align well with

Eysenck's (1947) dimensions of Extraversion and Neuroticism

(Cattell's labels being Exvia and Anxiety). Again, correlated

second-order factors may themselves be factored to yield

Page 17: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

17

third-order factors: an example of such an analysis being

presented by Cattell, Eber and Tatsuoka (1970).

For convenience of reference, and to avoid repetitive

descriptions of the interpretation of Cattell's factors, a list of

factor labels, 'technical names' and descriptive adjectives is

bound as a fold-out at the back of this thesis.

Revised verions of the 16 PF appeared successively up to

the definitive 1967/68 editions, since when a seven-factor

supplement has appeared, and downward extensions to high-

school age and below have also been produced. A long series of

publications by Cattell and his collaborators from 1940 to the

present day has detailed the elaboration of his theory: apart

from the improvement of the questionnaires, this research has

extended the scope of Cattell's theory to include an account of

motivation and moods, has pursued factor matching between

different measurement media, has contributed technical advances

in factor-analytic methods, and has attempted to establish a

central role for personality, and for factor analysis, in many

areas of psychology.

Despite the ambitious scope of Cattell's work and the

wide-ranging claims he has made for his theory, critical treat-

ment of his approach has tended to concentrate on the validity of

Page 18: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

18

his original claim to have discovered the fundamental variables

of personality and produced a questionnaire which measures

them. Two areas of controversy are particularly relevant to

this attempt to replicate Cattell's structure: the status of

Cattell's concept of trait, and the psychometric criticisms which

have been levelled at the 16 PF, to which questions we now turn.

Page 19: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

19

CHAPTER 2

Only rarely has Cattell offered definitions of the word

'trait'. Two are, however, forthcoming in Cattell (1965):

"Trait: a unitary configuration in behaviour such that when one part is present in a certain degree, we can infer that a person will show the other parts in a certain degree." (p. 375)

"By a trait, therefore, we obviously mean some relatively permanent and broad reaction tendency."

(1)- 28)

These scarcely capture the distinctive characteristics of

his use of the term, but more can be gleaned from a careful

reading of his more recent writings, in particular Cattell (1973).

One distinction recurs constantly, that between 'surface

traits', observable consistencies in behaviour, which are or may

be determined by more than one 'source trait', and source traits

themselves, which are not directly observable, but are revealed

by factor analysis of surface traits. Source traits are seen as

the fundamental variables of personality, substantially invariant

across social and cultural groups (Cattell 1973, p. 330ff),

although not necessarily invariant either in terms of their

expression in behaviour or in terms of which items or tests are

most appropriate for their measurement.

Page 20: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

20

The overriding importance which Cattell attaches to source

traits is exemplified by his account of the origins of the second-

order Exvia factor, which, as has been mentioned, seems to be

closely comparable with Eysenck's dimension of Extraversion.

Whereas Eysenck (e. g. Eysenck and Eysenck 1969) treats this

factor as the source of lower-order factors, which in his view are

far from reliably identified, and as related to the functioning of the

ascending reticular activating system in the brain stem, for

Cattell it is the result of a complex positive-feedback interaction

amongst source traits mediated by social mechanisms (Cattell

1957), and not a fundamental variable in its own right.

Next, there is the unusually strong emphasis placed by

Cattell on the use of factor analysis in the discovery of source

traits. Source traits are 'factor-dimensions', variations along

which are determined by single unitary influences (Cattell 1965,

p. 374). That is to say, he appears to view factor analysis not

merely as a convenient technical device but as the sine qua non of

research in this area, and he has devoted some effort to the

demonstration of the efficiency of factor analysis in revealing

known structures, e. g. the dimensions and contents of coffee cups

(Cattell and Sullivan 1962). Along with this emphasis has gone an

interest in the development and refinement of factor-analytic

techniques.

Page 21: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

21

Likewise, despite hints (Cattell 1952, 1973 p. 22) of

possible nonlinearities in the regression of source traits on overt

behaviour, and of interactions between traits, states and situations,

multiple linear regression is the method of choice for prediction

purposes. Thus a comprehensive attempt to predict overt

behaviour would involve a specification of weights to be applied to

source trait scores, to state variables and to situational variables,

all of which may be classified by factor-analytic means.

Finally, the traits proposed are identified and named

a posteriori. Cattell's initial work was an attempt to define the

universe of discourse by reference to the dictionary, rather than

to build on previous work. Perhaps in consequence, the list of

traits measured by the 16PF reads as a rather eclectic collection

with no guiding theoretical orientation. Some traits, e. g. factor

C, Ego-strength, and factor G, Super-ego strength, have at least

a nominal link with psychodynamic theory; a group of five

(A, E, F, H, Q2) reflect different aspects of extraversion, a notion

with a long and complex history; one, factor I, seems not unlike

William James' notion of tender-mindedness, and others, e.g.

L and N, seem to be more or less peculiar to Cattell. From the

point of view of ontogenetic development the source traits are

equally heterogeneous. Factors A and H, he claims, have a

strong hereditary, physiological basis, whilst at the other extreme

Page 22: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

22

factors I and M are more or less exclusively the result of

learning in childhood.

These considerations make an overall assessment of the

conceptual status of the source traits impossible, since all they

have in common is their purported causal status and their discovery

by a particular statistical/mathematical technique. The emphasis,

to borrow the title of Cattell's early book, is on the description and

measurement of personality rather than on a unified account of the

sources and causes of personality differences.

This point has a bearing on one common criticism of trait-

oriented personality theory, succinctly stated by Mischel (1968,

p.42):

"Nothing is explained, however, if the state that we have attributed to the person from his behavior (the has a trait or state of anxiety') is now invoked as the cause of the behavior from which it was inferred. We quickly emerge with the tautology: 'He behaves anxiously because he has a trait of anxiety'. "

Thus Mischel claims that the use of trait names as

explanatory devices is based on a confusion between the

construction (or interpretation) of behaviour and the causes of

behaviour. Insofar as trait names are merely summary terms

for observed and interpreted behaviour, and so long as the only

basis for the adoption of traits is descriptive in this sense, the

Page 23: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

23

criticism holds. However, in the longer term the enterprise on

which Cattell embarked is not merely one of efficient description,

although it is at least that. Rather there is an eventual aim of

linking some traits to neural and endocrine events and structures

in such a way as to bring elements of social behaviour into the

same general nomological net as physical events. Given success

in this enterprise the methods yielding traits which do map onto

such physical variables have presumptive utility for the discovery

of traits whose sources are in, for example, child-rearing

practices.

In this respect, Cattell's work is closely modelled on the

natural sciences in which he received his early academic training.

For at various stages in the development of physics, Mischel's

criticism might well have been levelled at concepts such as mass,

valence or magnetic flux, concepts whose value derived at least

for a time from their place in a systematic account of empirical

phenomena, before their basis in the atomic and subatomic

structure of matter had been elaborated.

The extent to which links between Cattell's source traits

and neural and endocrine phenomena have actually been forged is

debatable. Royce (1973) and Fahrenberg (1977) review some of

the common ground occupied by various trait-oriented theorists,

and Cattell and Warburton (1967) present a compendium of

Page 24: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

24

physiological and performance tests which, it is claimed, relate

to certain of Cattell's second-order factors. However, for the

moment such developments are at best promising rather than

conclusive.

On the other hand, the converse argument is powerful: if

it can be shown that traits, or some set of traits, cannot be

consistently discovered or derived, then there is little chance of

finding physical phenomena to which they relate, whereas if they

can be replicated then some explanation is called for. Such an

explanation might simply be in terms of response sets, or it might

involve either the original theory or some reworking of it, such as

Gray's (1973) reinterpretation of Eysenck's three personality

dimensions.

In recent years there have been developments in thinking

concerning the sources of a given behaviour in a given situation

converging from several authors which have a bearing on the

interpretation of Cattell's traits. For instance Mischel (1973) in

a modification of his earlier view has argued that what is needed

for adequate prediction of behaviour is information on a wide range

of variables, some of which relate to the behaviour in question only

through the conscious anticipation of the subject as being possible

consequences of his actions. Bern and Allen (1974) in a similar

vein argue for idiographic assumptions about the sources of

Page 25: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

25

individual behaviour, even when that behaviour shows consistency

across situations and subjects. Argyle (1976) has moved away

from a notion of personality dominated by the situation to a

structuralist position in which behaviour is generated from a 'deep

structure' of rules, which may at once be idiosyncratic and yet

have elements of commonality across individuals.

Perhaps this move is best summed up by Harr and

Secord (1972) and in Harry's (1976) collection of articles

suggesting a cognitive approach to the understanding of social

behaviour. Essentially it involves a rejection of the 'classical'

model of physical science as applied to the explanation of social

behaviour, in particular insofar as this permits only of inert

entities and efficient causality. Harri and Secord state quite

explicitly (op. cit. p.256) that "A man's nature is a psycho-

physiological mix", the psychological part of the mix involving

such functions as anticipating the consequences of actions, giving

accounts of the sources of past actions and the like. In other

words they propose a formal reintroduction of the Aristotelian

notion of final causality, at least insofar as a person may

represent to himself future states of affairs and attempt to bring

them about, into the explanation of behaviour.

Page 26: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

26

As it happens, they cite Cattell's factors C and A as

instances of the kind of powers and complementary liabilities

which satisfy, at least in outline, their criteria for explanatory

notions, and remark especially with regard to factor A that it is

a notable instance of the kind of psychophysiological mix they

regard as desirable in a full account of the sources of social

behaviour.

However, Cattell's own position on this point is either

unclear or simply so eclectic as to permit a variety of inter-

pretations. The definitions of 'trait' quoted at the beginning of

this chapter fall under Alston's (1976) notion of 'T-concept' : we

think of different persons having different degrees of the

disposition, where the degree is a function of the frequency of

response in a representative set of situations along with the

average magnitude of the responses" (p. 67, abbreviations

expanded). Alston contrasts the T -concept' quite sharply with

purposive-cognitive concepts. T-concepts, like co-operative-

ness, sociability, persistence, refer to overt, observable

reaction tendencies, such that a given level of each is associated

with a characteristic frequency and strength of response in a

given situation. Purposive-cognitive concepts, on the other

hand, such as need for social contact, or desire for control over

others, may not give rise to overt behaviour in the form of actual

Page 27: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

27

social contact or actual exercise of control over others, yet there

is no contradiction in ascribing these characteristics to

individuals who do not involve themselves in social contact and

who do not exercise control over others.

It is at least possible that Cattell sees some of his source

traits as purposive-cognitive concepts, related causally to

T-concepts, for which read surface traits. Alternatively, one

might class some of his source traits as purposive-cognitive and

others as T-concepts. The distinction is not made explicitly by

Cattell himself, and inspection of his questionnaire items is of

little help. Some items, e. g. "As a teenager, I joined in school

sports: a. occasionally, b. fairly often, c. a great deal", seem

to demand a factual report of the respondent's past or present

behaviour, whilst others, e.g. "I would rather mix with polite

people than with rough, rebellious individuals", seem designed

to elicit a statement of preference between two hypothetical states

of affairs.

The status of questionnaire responses is far from clear:

they may be read as factual statements, including as facts needs,

preferences, values, interests and the like, as attempts to

express a self-concept in terms of item content, or as responses,

in the full-blown behaviourist sense, to stimuli, viz. the items,

whose content will not stand common-sense analysis. None of

Page 28: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

28

these approaches is wholly satisfactory, the first because one is

potentially overrating the individual's self-knowledge and access

to accurate information about his own behaviour, the second

because there is as yet little research to show that this in fact is

what subjects think they are doing, and the third because

respondents do seem to believe that they are indeed conveying

information about themselves, and often take pains to be as

accurate as possible in their responses - a phenomenon which is

difficult to quantify but which is very apparent in the course of

supervising a test session.

The question of what actually takes place between the

reading of an item andthe recording of a response is amongst the

least researched aspects of psychometrics; however, for present

purposes we may note the problem and leave it to one side. For

there is a distinction to be made between two stages in the

development of a scientific theory: in the first phase, one collects

as accurately as possible measurements and observations

appropriate to the phenomena under study, in an attempt to

discover and formalise regularities; in the second, theories are

constructed to account for the observed regularities and tested

against them. (Of course, theories may suggest as yet

unnoticed regularities whose presence or absence may result in

modifications to or rejection of a theory, in the Popperian

Page 29: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

29

hypothetico-deductive pattern, but here we are concerned with the

elements of theory-building, rather than the logic of scientific

progress).

The classic example of this procedure is the development

of the kinetic theory of gases as an account of the regularities

formalised in Boyle's and Charles' gas laws, these gas laws being

characterised by Harr(and Secord (1972) as a matter of 'critical

description' or 'proto-science' rather than of developed science.

Since a great deal of controversy has been fuelled by the

question of what regularities are to be found in Cattell's personality

scales, however, it seems that there is a need for some careful

critical description to discover just what regularities are present

to require a theoretical account. That is to say, the question of

whether there is a good approximation to Cattell's structure to be

found is prior to the question of why it should be present.

Page 30: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

30

CHAPTER 3

The 16PF has received a good deal of attention in terms of

its adequacy as a psychometric device, and in terms of the fit

between questionnaire factors and factors found in rating studies.

We are here concerned exclusively with the first of these.

Three lines of attack have been adopted by Cattell's critics.

Firstly, the value of traits in predicting behaviour has been

questioned. Mischel (1968) reviews a large number of studies

relating trait measures, not specifically Cattell's, to external

criteria, and concludes that their utility has been much exaggerated.

This is a line of argument which will not be pursued here, since the

problem of how best to incorporate trait measures into the

prediction of behaviour would need a thesis to itself, and since

past failures need not imply that the exercise is futile.

Secondly, the 16PF has been specifically criticised on

account of the apparently low homogeneity of the scales it

contains. Levonian (1961), Timm (1968), Eysenck and Eysenck

(1969), Greif (1970) and Howarth, Browne and Marceau (1971)

have taken this line. With attainment and ability tests, one is

accustomed to find that the items in reliable and valid scales

correlate moderately highly, and indices of reliability based on

these correlations have a long history. Cattell has never

Page 31: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

31

challenged the empirical results of these authors; instead he has

argued consistently that homogeneity is a red herring in the

construction of personality tests with high transfer reliability

(Cattell and Tsujioka 1964, Cattell 1973). Howarth and Browne

(1971a) attempt a rebuttal of this argument in terms of the require-

ments of simple structure as a criterion for rotation in factor

analysis. However, as will become clear when the results of

homogeneity analyses in the present study are presented, this

controversy is itself something of a red herring. It will be

argued that, with a small number of exceptions, Cattell's scales

meet accepted standards of internal consistency, length for length

being about as homogeneous as Eysenck's own scales in terms of

Cronbach's coefficient alpha (Cronbach 1951).

Thirdly and most controversially of all, the claim has been

made repeatedly that when items and/or scales from the 16PF are

factor analysed, Cattell's structure either cannot be identified, or

is not the most parsimonious solution. Becker (1961), Borgatta

(1962), Timm (1968), Eysenck and Eysenck (1969), Sells, Demaree

and Will (1970, 1971), Howarth and Browne (1971a) and Vagg and

Hammond (1976) all associate themselves with this point of view.

In other words, there is almost total agreement in published

studies arising outside Cattell's circle of collaborators that he is

simply mistaken (Guilford 1975).

Page 32: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

32

Two specific procedural points recur: his critics claim

that Cattell consistently rotates more factors than are indicated

by various criteria for the number of common factors present,

and that his use of 'parcels' of items as the basic variables in

his analyses, rather than single items, distorts the solution by

imposing his structure a priori. A third consideration is that

Cattell typically identifies items as being significantly loaded by

a given factor on the basis of unconventionally low factor pattern

coefficients.

The focus of attention throughout this long-standing

controversy is on the replicability of Cattell's primary factors.

There is little dispute about at least two and possibly four or more

second-order factors. Indeed Eysenck (1972) makes the straight-

forward claim that the primary-factor scales of the 16PF yield no

more reliable information than the second order, thus calling into

question one of Cattell's fundamental arguments for the use of

primary factor scales.

In response, Cattell (1972) has depended largely on

technical arguments as to the adequacy of the procedures and

criteria actually used by his critics in conducting their factor

analyses. Cattell (1972), Cattell, Eber and Delhees (1973), and

Burdsal and Vaughn (1974) have also presented factor pattern

matrices purporting to show that the nominal structure of the 16PF

Page 33: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

33

is indeed to be found as the outcome of an item-factor analysis.

However, in these studies many items have near-zero loadings on

the factors to which they supposedly contribute, and large factor-

pattern coefficients are few. This Cattell explains as being due

to 'suppressor action' - inclusion of items with low or even

negative loadings on a given factor but relatively high loadings

elsewhere purifies the questionnaire scale by cancelling out the

effects of unwanted variance associated with higher-loading items.

Put otherwise this may simply mean that factor-score coefficients

do not necessarily match factor-pattern coefficients, i. e. the best

set of items for achieving a good estimate of a factor score is not

always that set having the highest loadings. Certainly, as

Guilford (1975) points out, Cattell has never explicitly stated

which items are suppressing what.

On the question of the use of item parcels, Cattell is not

alone. Nunnally (1967) and Comrey (1970) both advocate parcelling

on the grounds that single items are too unreliable a basis for

factor analysis. More detailed consideration of this point follows

in a later chapter, as it is particularly relevant to the methods

adopted in this study.

Overall, the present status of Cattell's scales is rather

confused. There is controversy concerning both the appropriate-

ness of various technical procedure in assessing the adequacy of

Page 34: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

34

the scales and factor-analytic studies of them, and the inter-

pretation of published analyses. Little progress has been made

over a period of nearly twenty years towards a rapprochement

between Cattell and his critics, and no agreement has been

reached as to what would count as an experimentum crucis.

Meanwhile in commercial terms the 16PF has thrived alongside

the products of other laboratories, and currently vies with

Eysenck's questionnaires as the most widely used personality

questionnaire in the United Kingdom, ranking about fifth in the

USA. Whatever the psychometric arguments that have appeared

so far, there is clearly a body of users who actually find the

scales useful or are woefully blind to their inadequacies.

This thesis is therefore aimed at reconciling three points

of view, Cattell's, his critics' and his customers'. It takes a

fresh look at the controversy surrounding the 16PF from the point

of view of the prospective user of the test, asking to what extent

Cattell's structure can be replicated, and what steps are

necessary to yield such a replication, in the hope that in the process

light will be shed on the sources of dispute. The procedures

adopted take as their fundamental variables the scales of the

published test, since if the purported structure cannot be found at

this level, there is little chance of it being found at the level of

items (and even if it were, there would be little point in using

the 16PF in its present form).

Page 35: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

35

The next chapter therefore considers at length the problems

involved in attempts to replicate an hypothesised factor model with

a view to establishing the utility of the methods to be adopted.

Page 36: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

36

CHAPTER 4

The use of factor analysis in personality research has not,

in general, been directed simply at a reduction in the number of

variables to be considered in a given data set. Rather it has been

used as a strategic tool, with the aim of identifying replicable

factors which will serve as fundamental elements in a rounded

theory of personality. A great part of the controversy concerning

rival accounts of the structure of personality has been attributed to

differences in the technical procedures adopted by various

theorists. In particular, three issues can be identified as

important in the eyes of Cattell:

1. Method of decomposing the matrix to be analysed

(principal components, principal factors, image

factors, alpha factors, maximum likelihood etc.)

along with corresponding methods of estimating

communalities.

2. Basis of choice for number of factors to retain

and rotate.

3. Method of rotation and criteria for acceptance of

a solution as representing 'simple structure'.

In point of fact, none of these issues is capable of resolution by

strict proof in the context in which it arises, since the funda-

mental question being asked is not "which solution gives the best

reproduction of the original data?" but rather "which solution is

Page 37: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

37

most useful in the elaboration of a theory of personality?"

Nonetheless, they are debated at length in a highly proprietary

brand of disputation which is generally inaccessible to the non-

specialist.

Rather less thought appears to have been given to certain

other issues, notably:

4. Whether items, subscales ('parcels') or scales

should be the unit of analysis?

5. What kind of prior scaling of variables is most

appropriate for a given analysis?

6. How variables should be chosen, in terms both of

their number and of what is already known of

their interrelations.

And finally almost no discussion at all is to be found as to:

7. What the desirable characteristics of the sample

providing the data base might be.

8. Whether influences other than the personality

variables nominally under investigation may be

reflected in the data base.

It is the contention of the present author that these issues

should be tackled in reverse order, that is to say in the order,

more or less, in which they arise in the practical business of

choosing a sample, testing participants and analysing their

scores.

Page 38: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

38

From the point of view of the present work, factor analysis

is seen as a means of confirming or disconfirming a proposed

solution or model, itself derived by factor analysis. Thus we are

not concerned with the question of whether the common-factor

model is an appropriate aid in developing a personality theory,

rather we shall be investigating the fit of a particular structure

within the scope of that model.

The primary problem with factor analysis as a research

technique is its indeterminacy. When it is used as an exploratory

tool in the development of an hypothesis, this indeterminacy need

not be a great handicap. In attempts to test the adequacy or

replicability of a factor theory, however, it is a major problem

(Pawlik 1973). To make matters worse, in factoring personality

tests we are dealing with highly imperfect variables. Even with

perfectly reliable variables, there would still be indeterminacy

arising from the lack of definition of relevant common-factor

variance, the lack of a guaranteed foolproof method of rotation,

and sampling error in the matrix to be analysed. With unreliable

variables a further source of variation enters.

Thus we now turn to a discussion of the problem of error,

then to questions of sampling of both subjects and variables, and

finally to the matter of the choice of matrix to analyse and the

choice of items or scales as the units of analysis.

Page 39: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

39

THE TREATMENT OF ERROR

The common-factor model involves assumptions concerning

the structure of error variance in a correlation or covariance

matrix which are of particular importance in the present context.

Briefly stated, common-factor analysis decomposes an m x m

matrix into k common factors, where k is less than m, and m

'unique', 'specific' or 'error' factors ('error' in the sense of

making no contribution to the common-factor structure, rather

than implying unreliability). On the assumption that the unique

factors are orthogonal to each other and to the common factors,

the analysis then yields common factors which are formally

uncontaminated by measurement error.

In vector notation, the common-factor model may be

expressed thus:

X = p. + +

where X is the vector of observed scores for a given individual

is the constant vector

A is the m x k matrix of factor pattern coefficients

▪ is the vector, of length k, of common-factor scores for that individual

▪ is the vector, of length m, of specific factor scores

This formulation is general, in that no modification is required to

take account of correlations amongst the common factors, i. e. the

Page 40: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

40

factor-pattern coefficients are partial regression coefficients in

the above equations. On the assumption that all specific factors

are mutually uncorrelated, and uncorrelated with all the common

factors, the correlations amongst the observed variables are

accounted for entirely in terms of .,,- .

In matrix notation, the variance-covariance matrix of the

observed variables, denoted here by E , may be expressed as:

Z = A ON “.12

where A is again the matrix of factor-pattern coefficients or loadings

co is the variance-covariance matrix of .,

q)2 is the diagonal matrix of specific factor variances, or error variances (Lord and Novick 1968 p. 533)

Thus in terms of the formalisation of the common-factor

model, the only term which may be called error relates to the

uncorrelated parts of observed variables. There are two

components of the specific factor variances. The first is errors

of measurement, i. e. the unreliability of observed scores. The

second is the extent to which the reliable parts of observed

variables are only partly to be accounted for by the common

factors in a given factor model. For instance, when a real

influence on a given variable is represented only in scores on

that variable it will not of course appear in the common-factor

Page 41: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

41

structure, even though the contribution it makes to the variance of

that variable is consistent, and may be identified in a different

analysis involving more or different variables.

Thus, when it comes to the interpretation of an obtained

factor solution, even supposing (which is unlikely) there to be no

non-zero off-diagonal residuals, one cannot without further

information unpack the specific factor variances into measurement

error and reliable variances associated with single variables. A

further problem arises from certain inadequacies in the kinds of

measures psychologists typically use. Interest usually centres

on factors representing influences in some domain of theoretical

interest, in the present case personality. However, what are

nominally personality tests may well reflect variations in

intelligence, or social class, or test response sets. Thus a factor

analysis of personality tests may result in a structure which

accurately describes the relationships among the tests, but not

among the variables with which the investigator is primarily

concerned, i. e. the personality traits, only imperfectly

represented in test scores.

This kind of error, error with respect to some substantive

theoretical model, may arise from any of three sources. In the

first place, a trait relevant to the domain under investigation may

be reflected in scores on tests intended to measure other traits.

Page 42: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

42

Thus anxiety might influence scores on measures of sociability -

in other words the sociability tests are to this extent inadequate -

with the result that the factor pattern matrix shows a loading of

the anxiety factor on the sociability tests. In the second place,

unwanted influences may affect scores on two or more measures,

in which case they may be identified, and rotated into separate

factors, eliminated by careful test construction, or deliberately

measured and thus accounted for. Attempts to deal with

acquiescence and social desirability response sets and the like

(Edwards 1957) illustrate this problem. Finally, variables which

are not formally represented in the tests to be factored, but which

can be measured by other means, may affect the obtained factor

structure, in particular through their influence on the mean

vectors of test variables.

To a great extent personality tests are composed of items

which may be expected to vary in significance across social and

cultural groups. That is to say, the same response to the same

item may signify different levels of the same trait, or even a

different trait altogether, in different groups. If the aim of

personality testing were merely to outline dimensions of (reported)

social behaviour, rather than such sources of differing behaviour

as may be ascribed to consistencies in the individual, this point

would be irrelevant, but we have it from Cattell (1973) that his

Page 43: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

43

scales are intended to measure traits with characteristically

different expressions in different social, cultural and national

groups.

Thus one must ask to what extent, for instance,

differences in mean scale scores between identifiable social

groups represent real differences in personality rather than a

tendency for one group to respond more in the higher-scoring

direction on account of the cultural loading of specific items.

The consequences of spurious differences generated in this way

will be to reduce correlations between variables when single

scales are affected, and inflate them when both scales are

involved. Thus within-group correlations may vary considerably

from correlations computed over a total sample, and the more

pervasive the influence of the moderator variable, the greater the

distortion resulting.

Note that this problem is not identifiable with elements of

either of the formulations of the common-factor model presented

earlier. The formal model is concerned with accounting for the

observed variates, not the underlying theoretically important

variables. What is being suggested here is that parallel to the

effects of unreliability in increasing the size of the specific-

factor loadings is an effect of low validity on the common-factor

loadings. Specificially, when there are pervasive moderator

Page 44: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

44

effects, one would expect increased values for correlations

computed over the total sample when the algebraic product of the

signs of the moderator-induced shift in mean scores is the same

as the sign of the within-group correlation. Depending on the

extent of the moderator effect, this may well result in either a

reduction in the number of factors retained for rotation, when the

criterion for retention is based on the size of the eigenvalues, or

a spurious increase in correlations between common factors in an

oblique solution, or the appearance of factors which are not readily

interpretable in terms of the variables nominally entered into the

analysis.

Since traditional methods of factor analysis do not recognise

these effects as error (and nor are they from the point of view of

the formal model), from the point of view of the personality

theorist distorted solutions are likely to result.

Given the heuristic nature of exploratory factor analysis as

typically used, there is little that can be done about error arising

from contamination of one test by factors hypothetically relevant

only to others, apart from attempts to purify measures by what

Cattell has called 'progressive rectification', i. e. improving the

tests in the light of successive studies - which can be something of

a circular activity. Where the result has been that structure

remains in the residual matrix, one should doubt one's criterion

Page 45: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

45

for the retention of factors for rotation, but this is only likely to

be acceptable when nominally equivalent measures have

appreciable residual covariance. (Inspection of residuals has,

of course, been automated in the maximum likelihood and minres

methods of analysis).

The correction of moderator-induced error is rather more

feasible: likely moderators may be partialled out in advance of

factor analysis, or analysis may be performed on homogeneous

groups selected on appropriate measures of the hypothesised

moderator. Simply entering likely moderators into the analysis

will not do in general, however, since any criterion for the

number of factors to extract which is based on eigenvalues may

be too severe in consequence, the more so the greater the effect

of the moderator.

To sum up, the problem of error when using factor

analysis as an aid in theory building and testing is not exhausted

by considerations of reliability. Factor analysis deals not in

variables which have a place in some theoretical scheme of

things, but in correlations or covariances between observed

scores in a sample of variates which purport to reflect

theoretically important variables, hence one must consider

whether there may be influences affecting the validity of testsused

for the sample involved.

Page 46: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

46

Thus far we have considered errors in factor analysis

arising from the imperfect reliability and validity of measures, a

topic which has received relatively little consideration in

published studies, although occasionally (e. g. Pawlik 1973,

Cattell 1966 p. 112) theoretical writers have devoted some space

to them. Such error has usually been lumped together with

sampling error, and no distinction has been made for purposes

of reporting the 'proportion of variance accounted for' by the

factors retained for rotation. In general, however, factor

analysts envisage the possibility of inference from the analysis

of covariance matrices derived from sample data to some larger

and usually ill-defined population. Thus limitations on the method

may be imposed by the size and representativeness of the sample,

both of which will affect the adequacy of the sample matrix as an

estimate of the population matrix, the first by way of the standard

error of the elements, the second in cases of unintentional

selection on the variables included.

Note that the second of these is not identical with the

question of moderator effects discussed earlier: moderator

effects reduce to the intrusion of unwanted variables which should

be eliminated; unintentional selection may result in a poor

estimate of the population matrix, but by way of deficient sampling

of subjects, possibly leading to restriction of range, rather than

Page 47: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

47

from different scaling properties of the measures within the

sample.

Questions of sample adequacy can only be settled outside

the computational devices of factor analysis; the matter of the

effects of sample size on the procedure is internal to it, and

attention has been focussed on this aspect with the emergence of

maximum-likelihood methods. The smaller the sample size, the

fewer common factors are needed for a statistically acceptable

solution. That is to say, the goodness-of-fit test used may not,

with small samples, be powerful enough to reject the hypothesis

of a number of factors which, from the personality theorist's

point of view, is smaller than optimum. Conversely, with a large

sample many more factors may be indicated by the significance

test associated with maximum likelihood procedures than are

interpretable, or than would have been retained using other methods

and criteria (see esp. Kaiser 1976), particularly when large

numbers of variables are involved. Prior to the introduction of

maximum likelihood methods, the adequacy of the size of a sample

was judged by reference to rule of thumb, 'five times as many

subjects as variables' being one common criterion. However, the

apparent rigour of the chi-squared test for number of factors does

not really bear on the matter of the number of relevant factors from

the point of view of the investigator, since when error due to poor

Page 48: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

48

test validity is present in the matrix a sufficiently large sample

size will ensure that the test will suggest a poor fit even when all

factors of theoretical interest have been extracted.

A factor analysis of personality scales may reasonably be

undertaken for either of two purposes: a) to assess the adequacy

of or aid in the development of a factor theory of personality; or

b) to investigate the adequacy of the scales representing such a

theory, given that it is accepted. One must distinguish between a

good theory poorly represented psychometrically and a bad theory

pure and simple. Bad tests may support a theoretical model, even

though they are quite unsuited for application to real-world problems.

SAMPLING OF SUBJECTS

There seems to be no general agreement as to what

constitutes an appropriate sample for the purposes of conducting

a factor-analytic investigation - indeed there appears to be very

little discussion of the topic at all. Most textbooks of factor

analysis simply ignore the matter, although Fruchter (1954)

devotes a brief paragraph to the intrusion of unwanted variance

due to deficient sampling.

Cattell (1966a) states that " ... preliminary taxonomic

work is necessary to make clear when one is sampling within

species and when across them," but continues by suggesting that

Page 49: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

49

this presents no problem, since we "know what we are doing and

do not put men and monkeys in the same sample" (p. 113).

Eysenck (1966) takes a firmer line: "Ultimately what is

being asserted here is that correlations between variables a and b

may differ significantly in size and direction when total groups are

subdivided according to typological principles, and correlations run

over these subgroups." In the absence of such typological analyses

"correlational analysis ... is liable to gross errors ... " Yet

when it comes to one of the most often cited of his own factor

analytic studies (Eysenck and Eysenck 1969), not only is the sample

not divided, except by sex, on typological grounds; not only is no

typological analysis presented to justify the procedures adopted;

but (p. 182) apologetic remarks are made as to the inadequacy of

the subject pool as representative of the population of the country

as a whole.

Nonetheless, in principle at least, both seem to agree that

homogeneity of subjects with regard to taxonomic considerations

outside the personality domain is desirable. But at this point the

problematic nature of the concept of personality intrudes. For

the interaction between taxonomic groupings and test scores is in

a sense what personality measurement is about. Trivially,

supposing there to be extraverts and introverts as exlusively

defined categories, one seeks an interaction between test scores

Page 50: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

50

and group membership as the basis of measurement.

Less trivially, females typically show higher average

scores than males on measures of anxiety and neuroticism. This

may of course be attributable to a real difference between the

sexes in levels of anxiety, but on the other hand may result from

a greater willingness on the part of females to admit to minor

symptoms which in fact they suffer no more than males. In the

first case the factor analyst would be justified in not analysing

results separately for the two sexes, in the second he should split

by sex. In general, given adequate sample size, there is nothing

to be lost and possibly something to be gained by splitting a sample

on taxonomic grounds or by seeking homogeneity on external

criteria, in terms of achieving a better fit to the assumptions of

the common-factor model.

SAMPLING OF VARIABLES

On the matter of sampling of variables, one couldhardly do

better than to quote Guilford (1975):

"There must be a favorable pattern or combination. of experimental variables. Each factor must be equally well represented in terms of the number of variables per factor, and each set must be equally strong with respect to the amount of variance. These require-ments for selection are important because in connection with rotation of axes, there is a general principle that strong (better represented) factors tend to rob weak ones of their variances."

Page 51: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

51

Such requirements assume a good deal of a priori

information about the variables relevant to a given domain, and

since factor analysis has been used mainly to generate

hypotheses rather than test them, Guilford's argument appears

to suggest that in the absence of a fortunate combination of

circumstances most factor analyses will be inconclusive.

Guilford's point may be taken even further, for not only

may weak factors be robbed of their variances in rotation, they

may not be included amongst the factors retained for rotation

if better-represented factors account for the greater part of the

variance in the matrix, a problem which will be particularly

relevant when the criterion for retention of factors is based on

the absolute sizes of eigenvalues.

This question is of importance for a critical examination

of studies which include sets of scales from different authors in

the same analysis, e. g. Eysenck and Eysenck (1969) and Sells,

Demaree and Wills (1970). Both studies are concerned, not to

establish regression weights for one set of scales onto the others,

but to determine which author's scales give the 'best' picture, if

any. Both studies used principal components analysis, and had

they rotated all components there would have been little of

interest in these analyses. (This follows from the fact that

all positive definite matrices of a given order form a group

Page 52: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

52

under non-singular transformation, which is itself simply a

reflection of the fact that such matrices have an inverse. Thus

the full set of components may always be rotated to a specified

pattern, with consequences only for the matrix of correlations

between rotated components.)

However, since in each case only the largest components

were retained, one must ask what characteristics would lead to

the representation of variables in the components rotated.

Eysenck and Eysenck in their joint factorial study of the Guilford,

Cattell and Eysenck scales extracted the first twenty principal

components of correlation matrices for each sex, made up as

follows:

From Eysenck's laboratory:

10 subscales representing Extraversion and 96 items Neuroticism

2 lie scales 18 items

From Cattell's laboratory:

15 short scales, each representing one source trait 99 items

From Guilford's laboratory:

13 short scales, each representing one primary 109 items factor

In addition three acquiescence scales were scored, one

from the items contributed by each of the three authors. On the

not unreasonable assumption that the various scales best represent,

Page 53: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

53

length for length, their authors' thinking, we thus have for the

hypothesised personality factors:

48 each for the two Eysenck factors, in two groups of five scales.

6-7 each for the Cattell factors in fifteen scales

8-9 each for the Guilford factors in thirteen scales

Now consider three points: 1. the reliability of a test tends

to increase with length; 2. correlations are attenuated as a

function of decreasing reliability of the variables; 3. there is

general agreement that Cattell's and Guilford's scales relate to

Eysenck's as lower-order to higher order factors, hence Eysenck's

factors represent common ground. It is hardly surprising that of

the ten components extracted, after rotation the first two carried

the largest portion of variance and were "unambiguously identified

as neuroticism and extraversion". Simply on the grounds that

Eysenck's factors are very overrepresented it is clear that the

conclusion drawn, that Cattell's factors are not well replicated,

is more a comment on the procedures adopted than on the adequacy

of Cattell's model. One would need a great deal of faith in the

capacity of factor analysis to reveal the true structure of

personality traits regardless of the configuration of input

variables to accept this analysis as a reasonable test of the

adequacy of Cattell's primary factor scales.

Page 54: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

54

This study is perhaps the clearest example of how sampling

of variables can predetermine the outcome of an analysis, and

illustrate how factor analysis favours areas of overlap between

different systems at the expense of factors which may be

represented in only one set of scales. It is likely, though more

difficult to show clearly, that the same effect is present in other

studies relating different sets of scales and items, such as Sells,

Demaree and Wills (1970) and Howarth and Browne (1971b).

CHOICE OF MATRIX

Traditionally, factor analysis has been conducted on

correlation matrix. Rarely are grounds for the choice

specifically of correlations forthcoming from authors who use the

technique, but in principle there is at least one other appropriate

index, namely covariance. Perhaps the strongest reason for

using correlations rather than covariances is that in general the

units of measurement of psychometric tests are arbitrary, so that

for a single sample study neither raw score means nor raw score

variances have any absolute significance. Hence the removal of

mean and variance effects which is implicit in the calculation of

product-moment correlations seems to be a logical choice in the

absence of strong indications to the contrary. That is to say,

whilst there are no particular grounds for setting all variables to

the same scale, psychometricians rarely have enough confidence

Page 55: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

55

to choose scaling factors for each variable separately, hence the

use of correlations is a straightforward admission of ignorance.

Statements as to the proportion of variance extracted in an analysis

have therefore to be read as referring to weighted raw score

variance and covariance, rather than variance in personality or

whatever.

However, from time to time studies, e. g. Eysenck and

Eysenck (1969), appear in which attempts are made to relate

results from the factor analysis of two separate groups, often sex

groups, and factor-pattern comparisons are made across samples

and studies. Leaving aside the question of different rotational

principles and methods which may lead to apparent lack of

factorial invariance, the exclusion of mean and variance effects

results in different metrics for the variables in different analyses.

In confirmatory factor analysis one is concerned to discover

whether a given factor solution, in terms of high and low loadings

in the factor pattern matrix, is replicable on a different sample

of variables and/or subjects. In neither case may one safely

assume that factor variances will be identical across studies, or

between groups within a study. Consequently in the process of

selecting the strongest factors on the basis of the size of their

associated eigenvalues and then rotating them blindly there is a

risk that underextraction of factors may lead to quite different

Page 56: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

56

solutions for different samples, or that particular criteria for

rotation may result in different positions for the axes where a

common pattern, at least in terms of which factors have high and

low loadings on which variables, could be established.

As a first step away from the totally arbitrary allocation

of unit variance to each variable, the use of a common scale for

the same variables in each sample followed by the factoring of

variance-coveriance matrices seems prudent. This does not

solve the overall problem of the relative size of variances

between variables in the sample yielding the common scale, but

is at least some improvement over the blind use of correlations.

The common scale for each variable might be obtained by

standardisation of variances in the pooled sample, and then

proceeding to analyse within-group coveriance matrices. There

are doubtless other possible approaches to this problem, and as

with so many other aspects of psychometric methods a strict

resolution of the problem will only come, if at all, with the

establishment of non-arbitrary scaling factors for particular tests.

Cattell (1944) tackles the problem of comparing factor

solutions across groups and samples, suggesting "parallel

proportional profiles" as the criterion for an acceptable solution

when more than one sample is involved. In effect, the results

Page 57: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

57

of adopting his approach are similar to the method just outlined,

so long as an appropriate number of factors is retained for

rotation. Eysenck and Eysenck (1969) adopted Cattell's

suggestion as to the criterion for acceptability, and claimed poor

matches between factors in their samples of males and females.

However, they did not transform their factors to be as similar as

possible before attempting to match. Otherwise the critics of

Cattell's model have been notable for their omission of any

consideration of the problem.

VARIABLE TYPE

Among Cattell's practices in his use of factor analysis one

of the most criticised is his tactic of analysing correlations

between parcels, that is to say subscales of small groups of

homogeneous items, rather than between single items. Howarth

(1976) among others has claimed that in effect this rigs a factor

analysis to achieve the investigator's desired results. Eysenck

(personal communication) has suggested that a definitive

resolution of the debate between himself and Cattell must stem

from the factor analysis of items. Comrey (1970) and Nunnally

(1967) express themselves strongly against the use of factor

analysis of items, largely on the grounds that items are highly

unreliable, whilst Cattell (1972) and Burdsal and Vaughn (1974)

Page 58: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

58

imply that parcel factoring is both equivalent in terms of results

and simply more convenient from a practical point of view. A

further consideration, rarely if ever mentioned in the psychometric

literature, is that items such as Cattell's, which have only three

score categories, do not satisfy the linear model and hence

without rescaling are not appropriate variables for factor analysis.

Serious arguments against item factoring tend to concentrate

on the intrinsic unreliability of single items. The effect of test

length on reliability is too well known to need extensive discussion;

quite simply a single-item test will be very considerably less

reliable than a test composed of a dozen or more similar items.

Unreliability of variables in a factor analysis will tend (a) to

reduce the absolute size of off-diagonal elements in a covariance

matrix (the attenuation effect); (b) to increase the amount of

correlated error through the intrusion of unwanted, specific

covariance common to only two or three items; and consequently

(c) to make rotation of the resultant factor-pattern both more

difficult and less clear in outcome, since common-factor loadings

on items will tend to be small, making the identification of

salients problematic.

A further complication arises when factoring large numbers

of items, in particular when their unreliability is taken into account.

As a rule of thumb, a sample size of from five to ten times the

Page 59: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

59

number of variables is recommended so as to counteract the

accumulation of sampling error in the off-diagonal elements of

the matrix. Given the 368 scored items in forms A and B of the

16PF, a sample size for a single analysis of well over 1500 is

indicated. Attempts to circumvent this particular problem, and

the associated problem of arranging computer facilities to handle

a matrix of the required size, have included the analysis of

'marker' items, that is to say items which are thought on the basis

of previous analyses to be particularly efficient at delineating the

structure of the factor space. Such a strategy, however, involves

a trade-off: smaller samples are needed, but the consequent

reduction in the number of hypothesised salients for each factor

may make rotation difficult. A priori, there are only slight

grounds for assuming that the best marker items for an American

general population sample will also be best for a British under-

graduate sample.

For the purposes of confirmatory analysis, it is hard to

see why Cattell's critics have not attempted to analyse scale scores

on two or more forms of the 16PF. There is after all a clear

hypothesis implicit in the publication of the test, viz. that named

scales in each form will be loaded by corresponding factors, and

not by others. Since the rotational indeterminacy of factor

analysis, exacerbated by the low loadings characteristic of item

Page 60: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

60

factoring, seems to preclude any agreement between Cattell and

his critics at the item level, analysing scales offers certain

distinct advantages:

1. The rank of a reduced (by insertion of communalities in

the diagonal) covariance matrix based on scales will be a lower

bound to the rank of a matrix based on the items combined in the

scales. That is, since scale scores are the simple sums of

mutually exclusive sets of item scores, at least as many factors

should be found in an item factor analysis as in a scale factor

analysis.

2. Given the higher reliability of scale scores, minor

covariances between small numbers of items resulting from

specific cultural or social influence local to the sample will be

reduced in their influence (e. g. "I can always enjoy myself at

a gay party" and "I am always straight in my dealings with my

friends" are hypothetical items with local double meanings).

3. If the hypothesised model holds, factor salients will be

larger, and hence rotation facilitated.

4. The argument that items which do not have large loadings

on factors corresponding to the scales on which they are scored

are nonetheless contributing to the purity of measurement by

suppression of unwanted covariance between other items in the

scale and other factors becomes irrelevant.

Page 61: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

61

In other words, by analysing at the level of scale scores, the

question as to whether Cattell's model is reasonably represented

in his scales can be settled. If it does hold, then it may be worth

investigating why, against evidence from other researchers, it

does so. If it does not hold, then the way is open to develop

notions of what the structure may be at the item level.

Thus the analysis of scale scores is convenient in several

respects, requiring smaller samples, improving the chances of

clear rotation, reducing demands on computing facilities (and

hence allowing the use of more sophisticated techniques), and

yielding the general advantage of dealing with more reliable

variables.

Finally, against the use of item factoring, there is the

possibility that scales consisting of items with different difficulty

levels will suggest more or different factors in an item factor

analysis as a consequence of their difficulties. Guttman's (1954)

simplex model is but one example of how a matrix of correlations

between measures of a single attribute, each measure differing

from the others in terms of its difficulty or complexity, may have

factors which fail to reflect the underlying unidimensionality.

Lord and Novick (1968), Pawlik (1973) and Levy (1973) all point

out this difficulty in equating the dimensionality of the factor space

Page 62: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

62

with the dimensionality of the latent space. Whilst in practice

they may be uncommon, it is possible to envisage scales whose

items differ in terms only of their difficulty. The item

inter correlation matrix for a perfect scale of this type would have

values decreasing progressively with distance from the diagonal.

Such a matrix will normally be of full rank, and will yield

principal components following an oscillatory pattern, and whilst

the scale may be unidimensional in the sense that all items are

validly measuring the same attribute, the least and most difficult

items may have near-zero correlation, be loaded by different

factors in a factor analysis and give the impression of a hetero-

geneous scale. The compensatory model of scale construction

is thus not the only possible model implicit in personality tests:

conjunctive, disjunctive and mixed models may equally claim

consideration.

Cattell is not explicit on this point, nor, given the

unreliability of items, is it a straightforward matter to decide

which model is implicit in his scales, or the extent to which they

are mixed if that be the case. However the possibility of a

progressive ordering of difficulty among the items of a scale is

yet another counterindication for the use of factor analysis of

items as a basis for evaluating the goodness of fit of Cattell's

model.

Page 63: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

CONCLUDING COMMENTS

The considerations outlined in this chapter were not

developed in isolation from the practical problems involved in the

design and conduct of the research project to which we shall turn

shortly. Given the almost unanimously critical research

literature, the author's expectations at the outset were that

Cattell was likely to come out of a large-scale study rather badly.

To avoid the possibility of predetermined negative results,

especially given the acknowledged indeterminacy of factor analysis,

it seemed prudent to allow Cattell's hypothesised model the benefit

of the doubt and seek sources of contradictory findings other than

in the technicalities of the arbitrary decisions which have to be

made. In the event these were not hard to find. The author has

no favourite criteria or techniques, rather his aim was to discover

which techniques if any supported Cattell's model. A further

gesture in the same direction is the use of an undergraduate subject

pool. Cattell, Eber and Delhees (1973) state that most of the

development work on the 16PF was done on samples of around 200,

with a strong undergraduate or young adult representation. This

study involves a sample five times as large, and the factor analyses

reported were conducted with the specific aim of attempting to

identify Cattell's model, rather than attempting to reject it.

63

Page 64: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

64

CHAPTER 5

THE DATA BASE:

1. Sampling and Administration

The collection of the data on which this study is based has

been described in Saville and Blinkhorn (1976). Since perhaps the

most intensive use of Cattell's scales in this country is in higher

education, in research, careers and personal counselling and by

employers of graduates in recruitment procedures, there was, at

the time the project was begun, a need for representative under-

graduate norms in Britain. A literature search revealed that no

systematic sampling of undergraduates for this purpose had been

attempted even in the USA. Saville (1973) had used random

location sampling techniques with repeated follow-up to collect

general population standardisation data for the 16PF, but despite

the undoubted success of this method it had proved expensive,

involving commercial sponsorship and a large grant from a charit-

able trust to cover the services of the British Market Research

Bureau.

In addition to the cost involved, ues of the same method for

sampling of undergraduates was impractical, largely due to the

difficulty of determining the place of residence of individuals in

the target population. Some compromise between random and

Page 65: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

65

volunteer sampling had to be found, and eventually the cooperation

of the careers advisory services of almost all the universities in

the UK was enlisted, polytechnics and other colleges being

excluded for lack of detailed statistics of undergraduate enrolment.

Based on the latest statistics then available (1969-70 entry)

each university was asked early in 1973 to provide a sample

proportionate to its annual undergraduate intake. At the time,

students' sensitivity concerning the disclosure of personal

information led to the universities being unwilling to release lists

of names for random selection, hence selection of participants was

delegated to the careers services and could not be supervised.

Where possible the careers services arranged for random lists of

names to be generated by computer; elsewhere every n'th name

was taken from the list of registered students in their second year

of undergraduate study. Each participant was written to

individually and invited to take part.

In practice, the sample size obtained from each university

was determined by a number of factors. In some cases the

careers service was prepared to write repeatedly to the same

individuals; in others a larger sample than requested was invited

to participate and no follow-up was attempted. In the

circumstances, such matters had to be left to the discretion of

the collaborators. One university (Manchester) had no difficulty

Page 66: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

66

in recruiting most of its original sample without follow-up, and had

to turn away uninvited volunteers, whilst at the other extreme

Sussex had a very poor response. In an attempt to offset these

differences in response rate, the universities had been stratified

by location, using arbitrarily defined regions of the UK, and by a

hybrid size/type classification (London, Oxbridge, provincial large,

provincial medium and provincial small). Shortfall in one

university was partially remedied by persuading fieldworkers in

universities in the same stratum to increase their sample.

The sample consists entirely of undergraduates in their

second (i. e. not necessarily penultimate) year of study, tested

between March and June 1973. Second year students were chosen

as being less under examination pressure than finalists, more

accultured to university life than freshmen, and likely in any case

to be about to make contact with the careers service. The sample

was randomly divided in two within each university, half the students

completing forms A and B of the 16PF, half forms C and D. All

students also completed the APU Occupational Interest Guide

(Advanced Version), and those taking the shorter forms C and D

of the 16PF also took two short Cattell scales (IPAT Anxiety Scale

and the Neuroticism Scale Questionnaire), plus form A of the

Eysenck Personality Inventory. A further balancing was introduced

by varying the administration order of forms of the 16PF, so that

Page 67: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

67

half of each subsample completed them in the order B-A (D-C)

rather than A-B (C-D). This was achieved by manipulation of the

page order in the specially printed booklets before stitching. The

original print of the 1968 Anglicised versions of the 16PF was

reproduced lithographically in the booklets used: answer sheets

were dispensed with by instructing participants to ring their

answers in the booklet - a procedure which speeds completion and

eliminates transcription errors. Reports from the fieldworkers

suggested that about 90 minutes had been adequate time for the

majority of the participants, a figure suggested on the basis of a

small pilot study conducted at Glamorgan Polytechnic.

2. Size and match of sample

The sample quotas negotiated with each university were

aimed at providing a minimum total sample of 2,000, split equally

by sex and across pairs of forms of the 16PF; the print run

allowed for a maximum total sample of 3,000. Table 1 gives the

actual numbers of returned protocols in each subgroup. The total

of 2584 represents approximately 3% of the total undergraduate

intake in British universities for the year in question, although

clearly and designedly females are overrepresented. For the

purposes of the analyses reported later, only complete protocols,

including all biographical information, were included, to ensure

Page 68: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

68

Forms A and B

Order A-B B-A

Forms C and D

C-D D-C

Male 309 304 305 300

Female 339 339 3)44 344

Total 648 643 649 644

Total A+B 1291 Total C+D 1293

Grand total 2584

Table 1: Numbers of protocols returned

% Pop.

Region

Males %

Sample

8

5

10

9

9

Females

% % Pop. Sample

7 9

6 6

12 9

12 11

4 5

% Pop

6

5

13

13

6

Total %

Sample

8

6

10

10

7

Wales 6

W. England 5 S.E. England 14

London 12

E.Anglia 7

Midlands 11 11 12 11 11 11

N.England 26 25 26 28 26 26

Scotland 15 17 17 15 16 16

N.Ireland 4 6 4 6 4 6

Size and type

Large Prov 32 33 37 34 33 33

Med. Prov. 29 31 29 31 29 31

Small Prov 18 21 18 22 18 21

London University 8 11 8 12 8 12

Oxbridge 10 9 4 6 8 8

Table 2: Percentages of university population of second-

year undergraduates and of sample by region

and size/type classification

Page 69: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

69

that all analyses were based on identical samples. This reduced

the sample for forms A and B of the 16PF to 545 males, 599 females

and hence a total of 1144, a move which had no appreciable effect

either on raw score distributions or on the match of the sample to

published statistics of university entrants for the intake from which

the same was drawn (University Grants Committee 1975). Table 2

gives percentage figures for population and sample classified

according to the sampling frame mentioned above for the forms

A and B sample.

Counting Oxford, Cambridge and London universities as one

each and the colleges of the University of Wales as separate

institutions, forty-three universities participated. Of the remainder,

three refused, one did not reply and three were prevented from taking

part as a result of their materials being strikebound at Heathrow

airport. Table 3 gives the results of chi-squared goodness of fit

tests between the sample and population figures of Table 2; these

show no evidence of a significant mismatch in distribution of

respondents across these (admittedly arbitrary) sampling

categories.

3. Internal evidence of quality

All protocols were inspected individually to identify those

with missing data and to ensure that incorrectly completed documents

were not processed. In addition each script was checked for signs of

Page 70: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

70

(8 by region

.2

by size/type

R

degrees of freedom)

,.2 A—

(4 degrees of freedom)

,-)(...2

Males 4.53 >0.80 1.85 >0.70

Females 3.12 70.90 3.60 >0.30

Total 3.41 >0.90 2.00 >0.70

Critical value for significance at 0.05 level 15.51 9. 49

Table 3: Chi-squared tests for fit of sample to population

Page 71: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

71

blatant frivolity. In the event, only a dozen or so scripts were

rejected, all but one because certain identifying detail, e. g. sex,

had not been recorded - the exception being one respondent who had

systematically added extra response categories to certain test

items. As mentioned above, other scripts were later rejected for

present purposes due to incompleteness in other respects.

Since participation in the fieldwork was voluntary, there

was a possibility that the project would attract respondents of a

particular persoia.lity type, e. g. more anxious, more dependent

or whatever. This might be expected to result in restriction of

range of scores on one or more of the personality scales. Since

population statistics are not available the crucial comparison is of

course not possible. However, comparison of scale variances

from this sample with those from Saville's general population

sample and from the American college student standardisation

sample suggests that if such an effect is present it is shared with

other studies, in the case of Saville's with a study employing

superior sampling techniques. Table 4 gives the standard

deviations of scale scores for forms A and B combined for the

three studies, divided by sex. With the notable exception of

factor B (intelligence), there is an overall tendency for the standard

deviations of scores for the present sample to be at least as large

as for the other samples. Thus it does not seem that the problems

Page 72: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

72

This study 0

0 -1

.. H co cd al 0 to is -P a) ;-1 1-1 rd -r-I 04 a) H F4 a) o

ci P-1 fl 8 6..3

-*a This study

Factor Males Females

A 6.92 5.85 6.92 6.58 5.02 6.04

B 2.94 3.47 2.94 2.75 3.44 2.94

C 7.60 6.99 7.19 7.11 6.67 7.10

E 7.58 7.30 7.11 7.72 6.69 7.30

F 9.39 9.23 8.28 8.56 8.61 7.96

G 6.69 5.90 '6.25 6.51 5.46 5.99

H 11.95 10.30 10.96 11.39 10.36 11.07

I 6.87 5.76 6.46 5.89 4.58 5.18

5.40 4.95 5.19 5.30 4.94 5.22

M 6.54 5.63 6.79 6.09 5.92 6.49

N 4.52 4.80 4.26 4.65 4.71 4.32

0 8.69 7.79 7.91 7.94 7.58 7.67

Q1 5.58 4.97 5.26 5.91 4.71 5.11

Q2 6.22 5.76 6.26 6.08 5.59 5.98

Q3

6.69 6.00 5.6o 6.58 5.73 5.96

Q4

9.59 9.20 8.63 9.32 8.16 8.65

Table 4: Standard deviations of raw scale scores by sex

in three standardisation samples, for forms A

and B of the 16-IF combined.

Page 73: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

73

involved in recruiting the sample in this case had any consequences

for the range of scores on individual scales that might lead one to

suppose that self-selection had been a more serious problem than

elsewhere.

In the circumstances, willing cooperation of participants

was perhaps more important than obtaining a perfectly random

sample by obliging selected individuals to take part in the project,

which might have led to poor quality data simply because responses

are entirely within the control of the individual. Had random

responding taken place on a large scale in protest against enforced

participation, the value of the exercise would have been considerably

reduced.

The only explicit internal checks on the quality of the data

were in the lie scale of the Eysenck Personality Inventory, and the

Motivational Distortion Scales of forms C and D of the 16PF (i. e

in tests which were not administered to the half of the sample with

which this thesis is principally concerned). These scales were

designed to detect deliberate distortion of responses. The mean

scale scores for this study were almost exactly identical with those

from Eysenck's standardisation in the case of the EPI, and about

half a standard deviation lower than those from Cattell's college

student standardisation of the 16PF. It is debatable to what extent

these scales should be taken seriously as indicating level of faking,

Page 74: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

74

since the items in question can often be identified by the suspicious

student. However, where the present author actually administered

the questionnaires, the only suspicions voiced about the detection of

faking and the like were when some participants suspected that

items were repeated verbatim as a check on consistency (which is

not the case).

The author's own observations of participants, in sessions

at Brunel University and several of the London colleges, left him

with the impression that there was considerable enthusiasm for the

project amongst those who actually took part. Each of these

sessions was, at the request of participants, followed by an

extended discussion of the aims of the project in relation to the

problems of assessing non-cognitive individual differences.

Reports from fieldworkers in other universities corroborated the

impression that motivation and interest was high.

No doubt the enthusiasm was in part due to the promise of

feedback of scores through the careers services, demand for which

continued throughout the following year. In practice, all

participants' scores on the occupational interest guide were made

available to those who requested them, and scores on the

personality tests were sent where an appropriately qualified

psychologist was available to interpret them. Anonymity was

maintained by keeping records of participants names against index

numbers only in the offices of the careers services.

Page 75: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

4. Initial analyses

Saville and Blinkhorn (1976) report a large number of

univariate and bivariate statistical analyses, with the aim of

clarifying the degree and nature of variation within the samples.

Relevant to this thesis in particular are the following findings.

(1) Sex differences in mean scale scores on the 16PF for

several factors are both statistically significant and in some cases

large in absolute terms. Furthermore there is overall con-

sistency, with only minor exceptions, both across forms and in

comparison with corresponding findings in the general population.

(Saville and Blinkhorn 1976, tables 3. 1 to 3. 6).

(2) Mean scale score differences are statistically significant

also in several cases across discipline categories, as coded in

terms of the nine groupings used by the University Grants

Committee. In particular there are large differences on Factor I,

which also shows large sex differences. These differences

remain when discipline groups are divided by sex. The size of

these differences had not been anticipated - indeed one researcher

with considerable experience in dealing with large undergraduate

samples advised that there was little point in even collecting

discipline information, since in his experience no groupings had

ever been found to yield significant differences. In the event,

75

Page 76: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

76

discipline differences appeared to be about as large as sex

differences overall, although mostly they were concentrated in

the division between Arts and Social Science students on the one

hand and Science and Engineering students on the other, rather

than between groups within these categories. In comparison with

the approximately comparable age group in Saville's general

population sample (mean age 20, range 16-24 years, of whom about

10% were university students) the undergraduates (mean age 20,

96% being in the range 17-24 years) show a complex pattern of

differences. Table 5 gives means and standard deviations for the

first-order scales of forms A and B combined of the 16PF for the

undergraduate sample divided by sex and discipline, and the general

population young adult group divided by sex. On some of the

scales undergraduates as a whole tend to differ in the same direction

from their general population coevals, whereas on others the young

adult mean scores are in between undergraduate discipline groups.

(3) Reliability, as estimated by alternate form correlations,

was generally higher than reported by Cattell, Eber and Tatsuoka

(1970), but in general not impressive. For forms A and B

coefficients ranged from .23 to .81, with median approximately

. 58. Fuller details and a comparison with other estimates of

reliability are given in the next Chapter.

Page 77: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

77

ril a) a)

cd

• a)

a) 0 0

1-1

E c13 r- (1) a g 0 0 cd 0 rd a) 0 7:1

*z r=4

A mean 21.12 15.57 21.39 18.19 18.04 21.88

s.d. 6.57 6.20 6.23 6.74 5.70 5.16

B mean 18.00 18.40 18.62 18.56 15.02 14.68

s.d. 3.04 2.84 2.70 2.84 3.02 3.27

C mean 30.19 31.83 28.95 31-17 30.48 27.82

s.d. 8.04 7.14 7.13 6.85 7.13 6.65

E mean 28.28 26.44 22.23 21.30 26.77 20.87

s.d. 7.28 7.70 7.87 7.39 7.35 6.50

F mean 31.06 28.56 30.24 29.93 32.67 31-07

s.d. 9.83 8.93 8.39 8.90 8.43 8.18

G mean 21.51 21.71 21.63 22.94 21.07 21.98

s.d. 6.72 6.68 6.44 6.59 6.14 5.47

H mean 26.58 22.82 22.81 22.66 28.56 23.14

s.d. 12.26 11.45 11.46 11.29 10.21 10.49

I mean 23.35 18.10 28.64 23.22 17.02 24.14

s.d. 6.37 6.39 5.01 5.86 6.11 4.49

L mean 18.72 16.96 15.96 15.02 18.89 17.29

s.d. 5.28 5.39 5.41 5.04 5.32 5.11

M mean 28.36 26.90 28.10 25.93 23.32 21.34

s.d. 6.63 6.41 6.09 5.86 5.83 6.04

N mean 17.34 17.82 18.86 18.54 18.78 20.69

s.d. 4.85 4.24 4.69 4.59 4.63 4.66

0 mean 21.34 20.91 26.17 25.04 20.47 26.34

s.d. 8.92 8.49 8.03 7.73 8.32 7.09

Qi mean 22.32 20.99 20.27 19.29 21-57 18-13

s.d. 5.61 5.49 6.03 5.6o 5.03 4.81

Q2 mean 20.18 20.92 20.09 19.33 18.85 18.97

s.d. 6.61 5.89 6.14 5.92 5.84 5.40

23 mean 21.37 22.82 19.98 21.50 21.57 20.02

s.d. 6.97 6.42 6.77 6.09 6.09 5.67

Q4 mean 25.11 23.67 29.25 27.85 25.54 30.71

s.d. 9.98 9.28 9.41 9.10 9.24 7.38

Table 5: Means and standard deviations of forms A-f-B scale scores

Page 78: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

78

(4) Factors C, 0, and Q4 on all forms and for all groups had

inter correlations approximately equal to their reliabilities, and

factors L and N for some groups and forms had correlations with

other scales equal to or greater than their reliabilities,

suggesting that their factorial independence was seriously in doubt.

(5) Eysenck's Extraversion and Neuroticism scales, followed

by Cattell's scales representing factors B, I and Q1 were

successively partialled out of a correlation matrix including the

two Eysenck scales and the scales of forms C and D of the 16PF,

calculated both for the total sample and for the sexes separately.

Results indicated that Eysenck's (1972) claim that the reliable

variance in Cattell's first order scales is almost totally

accounted for by Neuroticism and Extraversion is exaggerated,

and suggested a minimum of seven factors to account for these

scales.

The results summarised above form the background and

source of the present study. As far as can be ascertained, the

discovery of large discipline differences in personality scale

scores has been made elsewhere only by Barton and Cattell (1972)

in a smaller study conducted in a single college. Certainly there

appears to be a tacit assumption underlying both the conduct of

psychological research on undergraduates and criticism of such

Page 79: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

79

research that undergraduates form a relatively homogeneous group

selected from a heterogeneous population. In the case of the

personality scales under discussion, this does not appear to be the

case. It may well be that in the course of development of the 16PF,

the use of preponderately undergraduate subjects resulted in its

being peculiarly applicable to undergraduates, but on the present

evidence it seems only prudent, in the light of the discussion in the

last chapter, to be prepared to treat sex x discipline groups

separately for the purposes of factor analysis.

The monograph containing the various initial analyses

(Saville and Blinkhorn. 1976) deliberately did not set out to tackle in

detail the points of difference between Cattell and his critics,

however certain of the results suggested that the problems of the

reliability, homogeneity and factor structure of Cattell's scales had

been to some extent exaggerated in the critical literature. The

treatment of the question of homogeneity in particular seemed

inadequate and yet there are simple widely used procedures for

arriving at some kind of an agreed conclusion, which topic provides

the material for the next Chapter.

Page 80: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

80

CHAPTER 6

Several authors have criticised the 16PF on account of the

apparently low homogeneity of the scales it contains. The use of

homogeneity indices as a measure of the reliability of a test has,

of course, become well established in the development of measures

of attainment and ability. Commonly split-half coefficients,

estimates from the Kuder-Richardson formulae and Cronbach's

coefficient alpha (Cronbach 1951) are the indices adopted.

Perhaps the two most influential papers tackling this

question are by Levonian (1961) and Howarth, Browne and Marceau

(1971). Neither reports any overall homogeneity index for the

scales of the 16PF, rather inter-item correlations are presented

in summary form, the criterion used in each case being the

statistical significance of correlations between items within and

across scales. Howarth et al. report that of 3,267 significant

correlations, only 348 were intra-factor correlations. Further-

more, on average an item correlated significantly with 1.89 items

within its own (10- 13 item) scale, and with 15.86 items of the

remaining 171-174. Levonian's results were similar, although

as his sample was considerably smaller, absolute numbers of

significant correlations were correspondingly reduced.

Page 81: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

81

Presentation of results in this form is highly misleading:

let us consider the same data in another form. A single form of

the 16PF has 184 scored items, implying a total of 16,836 unique

off-diagonal entries in a correlation matrix, of which 984 are

intra-factor and 15,852 extra-factor. Thus we may rewrite the

Howarth et al. results as follows: rather more than one in three

of all intra-factor correlations are significant whilst only about

one in five of all extra-factor correlations meet the criterion, and

on average an item correlates significantly with one in five of the

items in its own scale, but with one in eleven of the items outside

its scale. Given that the scales are designed to be correlated, it

is of course hardly surprising that items in different scales should

themselves be correlated.

It is in any case not at all clear that the approach adopted

by Levonian and Howarth et al. is particularly appropriate to the

problem. In Chapter 4 it was pointed out that items validly

measuring the same attribute might have low intercorrelations on

account of differences in their 'difficulty' levels leading to

different univariate and bivariate score distributions. Since the

product moment correlation coefficient essentially measures the

linear component of the relationship, it is to be expected that items

in a scale will show varying intercorrelations partly on account of

their difficulty levels.

Page 82: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

82

For instance, items 76 and 176 in form A of the 16PF

correlate, in the total group for this study, 0.043. Univariate

score distributions for these items, expressed as percentages,

are as follows:

Score 0 1 2

Item 76 73.01 6.67 20.32

Item 176 17.67 19.44 62.89

Despite the apparently low intercorrelation of the items,

correlations between them and a composite formed of the

remaining 19 items in factor A for forms A and B combined, equally

weighted, are 0.394 for item 76 and 0.230 for item 176, very much

the order of item-partial scale correlation one expects in a fairly

homogeneous scale.

Since the instructions for the test suggest use of the middle

'uncertain' response category for no more than one item in five,

one would expect item score distributions to tend to be bimodal.

In fact, although overall this instruction appears to have been

followed, 'uncertain' responses vary widely from item to item,

from as few as 2% to over 50%, with the result that item score

distributions vary enormously. The effects of this are not

handled with any great sophistication by Levonian, who simply

combines middle response categories with the less frequently

chosen extreme response, or by Howarth, Browne and Marceau,

Page 83: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

83

who report only the distribution of frequency of use of the middle

response without drawing attention to possible implications for

item intercorrelations.

Cattell, in response to critics on the matter of homogeneity,

has consistently agreed with them that his scales are heterogeneous,

but equally consistently has argued that this is a desirable property

(e. g. Tsujioka and Cattell 1964, Cattell 1972, Cattell 1973). Items

which have low intercorrelations but which correlate well with the

factor to be measured will, he claims, yield scales which are

factor-true, in that their error variances will be a combination of

unique variances and small components of many unwanted factors,

rather than large components of a few.

However, actual indices of scale homogeneity are almost

impossible to find in the literature where Cattell's scales are

concerned. Cattell confines himself to presenting samples of

interitem correlations, together with multiple correlations

between items and factors in a factor analysis (e. g. Cattell 1972).

In the case of the multiple correlations it is not clear whether

these are based on equal item weighting, or on the usual

maximisation procedure for multiple correlation (in which case

the inference from items and their validities to scales and their

relationship to factors is not necessarily reasonable). It seems

likely that the latter is the method adopted, since such multiple

Page 84: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

84

correlations are easily obtainable from the matrices used in

computing factor scores, and the presetting of weights to be equal

is sufficiently unusual to warrant specific mention. If so, then

these multiple correlations probably represent an overestimate of

the validity of the questionnaire scales, which are scored by

applying equal integer weights to items.

Cattell adopts (Cattell, Eber and Tatsuoka 1970) the notion

of 'structured homogeneity', by which is meant that scales should

be internally heterogeneous, but correlate highly with alternate

forms and with the factor to be measured. Such scales are, he

suggests, the most useful and generalisable given likely variations

in item characteristics across social and cultural groups. Thus

one might expect to find homogeneity coefficients for each of the

scales of the 16PF rather lower than the corresponding alternate

form reliabilities.

Coefficient alpha was therefore computed for each of forms

A and B, and for forms A and B combined, for each sex and for the

total group. Since variations across sex groups were very small,

Table 6 presents these results for the whole group, together with

the corresponding alternate form correlations and values for

coefficient alpha taken from Burdsal and Vaughn (1974). These

last are, to the best of the present author's knowledge, the only

published values for the homogeneity of the 16PF scales.

Page 85: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

Cro

nbac

h's

Alp

ha

0

co

eff

icie

nts

A

ltern

ate

C

ronb

ac

h's

A

lpha

7:1

C14-6

E-I

Cn • v-I

rd

0 M 7711

gq

.10

to

cn • r•I

,ct4 C-

P:1

CD

0

0

0

0

P:1

0

N if

0 0

'70 •0 cn

• - 0 0

a, .0 In cn

0 0

0 714 N

0 0

N 0

in .14 •

0 0

ce) •7t4 N N CO

0 0 0

'70 ON 0 U1 tf1 N

• • • 0 0 0

co cn '0 t11 Ln

0 0 0

C, .0 ,0 If1 Ln

0 0 0

N N IC) .0 '0

• • • 0 0 0

0 N C'-

0 0

0 co

• • 0 0

.0 Ln Ln co

0 0

CO 7i4 CO

0 0

ON rg

Lf) CO

. • 0 0

'70 N

0

0 .0

• 0

N 'C

0

to

0

<NI

• 0

0

en •

0

N •cti

0

•cti

0

•14

• 0

rg L(

0

cn N .

0

N .cti

0

N d

0

O N

• 0

C'

0

• 0

CO 0

0

N

0

N N .

0

N

0

If)

. 0

,ct, in

0

0

CO Ifl .

0

N

0

N

• C

M eel

0

NI

0

.0 di

• 0

re)

0

CO Tt,

• 0

N

0

0 in

0

0 7r1

• 0

N 0

CO

• 0

M

0

.0 U1

0

0 Lc)

• 0

CO 0

0

• 0

N

0

(NI N

0

di N

• 0

a) 7:1U

1

Cr) ).1 0 4-a U

F-1 073

;-4 1:14 a)

-6a

0 4-4

›.• 4.4 • f--1 • I-I

Cd

a)

a) .-4 cr)

0

;-4

4.4 0 cn

85

N <12:1 (.)4zi01--4 4 zoactaa

O 0 U

Page 86: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

86

Clearly, these results lend no support to Cattell's claim

for 'structured homogeneity'. On the contrary, the close

correspondence between alternate form and homogeneity

coefficients suggests that items have not been combined into scales

on any principle of balanced heterogeneity, but rather that the

scales are collections of items from moderately homogeneous

domains. Since coefficient alpha may be interpreted as the

average of all possible split-half correlations, corrected for

test length, there is really nothing to choose between alpha and

alternate form correlations as a measure of scale reliability.

Lest it be thought that the close correspondence of these

two sets of indices results from cultural homogeneity in the sample

of undergraduates on which they are based, Saville (personal

communication) has found equally close correspondence between

them for his general population sample using the same scales.

Variations in the value of alpha for different forms, as, e. g.

in the cases of factors B, M and 0, are traceable to specific items

having negative or very low positive correlations with the rest of the

items formed into a composite, i. e. item-partial scale correlations.

Whilst these results fail to support Cattell's specific claim

for structured homogeneity, however, nor do they show the

disastrous lack of homogeneity that Levonian and Howarth, Browne

Page 87: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

87

and Marceau seem to suggest. Coefficients alpha for the

Extraversion and Neuroticism scales of the Eysenck Personality

Inventory form A, based on the half of the sample which completed

forms C and D of the 16PF, which is in all respects comparable

with the forms A and B sample, are respectively 0.77 and 0.84.

Since the EPI has 24 items per scale, these figures are comparable

with the 20-26 item scales of forms A and B combined in the 16PF,

that is to say with the last column in Table 6. On this basis,

Cattell's scales for factors A, C, E, F, G, H, I, 0, Q3 and Q4 are of

approximately the same order of homogeneity as Eysenck's scales,

with Q1 and Q2 not far adrift.

In the context of ability and attainment tests, Nunnally

(1967) states that about 20 items are typically needed to reach a

value for coefficient alpha of about 0.8. Although several of

Cattell's scales fall below this level, even for two-form length, it

would appear that the discrepancy is not nearly as great as both

Cattell and his critics would have us believe. Dotnain-sampling

theory (Nunnally 1967) suggests that the 'domain validity' for a

scale should approach the square-root of coefficient alpha. In

terms of factor analysis, this may be translated into an expected

value of the factor-structure coefficient for any scale and the factor

it represents, i. e. its factor validity. Comparison of the square-

roots of the coefficients in Table 6 with factor validities quoted in

Page 88: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

88

Tables 5.4 and 5.5 of Cattell, Eber and Tatsuoka (1970) supports

this interpretation: domain validities approach, but are in general

somewhat lower than, Cattell's claimed factor validities.

Of course within-scale statistics reveal nothing of the

dimensionality of the 16PF as a whole. It is quite conceivable that

several of the scales are drawn from the same domain of items.

However as a check on the adequacy of a factor analysis, co-

efficient alpha provides an independent source of information. As

will become clear, the results of the factor analyses which follow

bear out the value of internal consistency as a measure of scale

adequacy in the case of Cattell's scales.

Page 89: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

89

CHAPTER 7

INTRODUCTION TO FACTOR ANALYSES

The remaining Chapters are concerned with a description

and interpretation of factor analyses conducted on forms A and B

of the 16PF. Most of these analyses are summarised rather than

described in detail, for two reasons: first, many of the analyses

gave interpretatively similar or identical results, and second

presentation of the tables associated with all the analyses would

add greatly to the bulk, but not the substance, of the thesis.

A number of strategic decisions were made in advance, in

the light of the considerations discussed in Chapter 4. Firstly, it

was decided to work with scale scores rather than items; apart

from the technical advantages of this decision already discussed,

the problems of factoring a square matrix of order 368 by any but

the crudest methods were insuperable from a data-processing

point of view. Secondly, the sex and discipline differences

mentioned in Chaper 5 seemed large enough to demand at least

some analyses based on within-group or pooled within-group

matrices. The question of variable and factor variances was

settled by standardising all scores on a common metric, viz.

n-sten normed scores, with a mean of 5.5 and a nominal standard

deviation of 2, which had been calculated on the present sample,

Page 90: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

90

imposing not only a common metric but also as close an approxima-

tion to a normal distribution as is possible. For convenience in

interpretation the scale was further changed to a mean of 0 and

standard deviation of 1, again on the total group.

Thirdly, the choice of computational methods used was

deliberately eclectic. A variety of factoring methods, involving

different definitions of and means of estimating communalities,

and different rotational procedures, some automatic, some under

human control, were adopted by way of exploratory analysis, in

an order partly determined by availability of computer software

and time at any given moment. The utility of such an exploratory

sequence was seen as in either the partial or total replication of

Cattell's model by means of traditional factoring methods as a

prelude to the use of more expensive confirmatory methods, or

alternatively in the absence of such replication, the identification

of a likely alternative model for testing.

Certain desirable features of analysis proved not to be

possible for lack of appropriate software. Most notably, off the

shelf programs for unrestricted analysis of covariance matrices

and Harris-Kaiser orthoblique rotation could not be implemented

successfully within available resources, but in the event their lack

did not detract from the emergence of a relatively stable and highly

interpretable solution, linking Cattell's model to previous psycho-

Page 91: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

91

metric studies, the homogeneity analysis reported in Chapter 6

and even psychobiological notions concerning the roots of

individual differences in personality.

UNRESTRICTED FACTOR ANALYSES

Perhaps the most obvious and most widely used first step

in factor analysis is quite simply to correlate all variables in the

total sample, iterate for communalities and rotate principal axes

corresponding to principal components associated with eigenvalues

greater than 1. This was accordingly done for the 32 scales in

forms A and B of the 16PF. Seven factors were rotated using

direct oblimin with the delta parameter set at zero. The resultant

factor pattern matrix is presented in Table 7, and'a plot of eigen-

values in Figure 1. This solution is rather close to Cattell's

second-order factor model:

Factor 1 is clearly very close to the second-order Anxiety

factor, with major loadings on scales C, L, 0, Q3, Q4.

Factors 2 and 3 together involve most of the scales

contributing to the second-order Exvia factor, A, E, F, H

and Q2 , with an additional loading on N.

Factor 4 bears a strong resemblance to the second order

'prodigal subjectivity' factor, with loadings on I, M and

possibly L.

Factor 5 loads G and Q3, i. e. the second order superego

factor.

Page 92: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

, 1,3 L.) U ) .1

92

0 • 1-4

/-I 0

co N Lfi ^O +- N 'IT e4 CO N ro Co co m 0, 'AD CO N m 'AD 4' C4 C4 r. CO PI N VD 0

LID V, C'- CA -40 N) N) CN N N) Co hD +- Co lca VD OD M 04, CO ..cr Co N .qr N +- CA M

IN Co Co Co v Co Co Co m Co Co N Co m 14-71 Co Co Co C4 lr N LID VD +- • • . • • • • • • • • • • • •••••••••••••••• u.

MA Co tn M Mt M tai Co Co Co Co Co Co Ma Co Co Leto Co nmtslmmmmmmmmmm tt-

! 1 1 I I I ! ! 1111

1'0 147I -.r. ..ci -0 -- -ot ro ...1. "0 cl- N ro Co -- "a' N) 0, N) 0, -0 h, OD ..r %!' -0 VI V, -0 Co CA Lil 0.4

PO N) ro ..4o CO Co h, Co Co PIO PO h7 h0 -.0 c.4 -- Co ,r V7 h., M h0 CA PO V7 .- CA la' +- LO C4 CA D.

-40 SA '0 M Co Co Co Co M +- Co Co Co Co tO Co tO Co 47 Co Co +- tO Co o Co Co M M +- Cl Mt Mt CI • ••••• ••••••• • • • • • • •• • •••••••• ••• 13I

M M tO MI M Mt Cr tO Co Co to to MI to tar Co tO Co to to to to to Ma MA to to M M to n Co 1 I I I 1 1 1 I 1 ! I I I I I 1 I 1 a,

-17

to r.... O. CA 00 -0 CO -.1- -./- r, -4o n to to CO h., No 0, ,- 0, ,r 0, to in 1.,7 to m 0, ro c4 C to 4, N M LIO tai CO 0, r... to -- IN c, -0 to -- N 1.17 M 4- -.0 4.-- ....- ,r 01 ...- Lo r, 40 ,r r-, c.4 0, ,r

L7 LTA to SA to +- N. .,.. M M +- SA Co CA in LO M SA to MA M CA NO +- +- ,-- M +- Co +- M4 V7 +- S- • • • • • • • • • • • a • a • • a a • • • • • • • • •••••• 0

to Co to M to to M Co Co to MI M M to MI Co MA to to to 'Si to tn Co Sa M to SA Co n MA Co 4-

1 I I 1 1 1 1 ! I I 1 1111 1 1 ul

CA +- .4" 'r to -- N '‘i- OD N) V7 ro co OD NO CN LID LID N m tO cr CO 4-- +- M 'C LID CO CO VD LID -t- '

4Cr ...... IN Co .1. SO N N. ..0 CO to +- CO CO CID ..- Kt 'sr 1", to h, +4" LI h-, to -47 CO to ,0 -Lr -a to 6=

..qr to i to m m to M NO MI m tO Co Co Co to Co -- Co 'Si MI to +- M NO N LID to +- +- M 'Si Co al • ••••• ■ ••• ••••• • •• a • I. •• ••••••••• E

MI 'Si to 'Si tar M to M M MI '0 tar 'Si to 'Si Co Co to MA to 'Si M Co 'Si 'Si Co 'Si tO M to MA Co w S. I 1 1 I 1 1 ! 1 1 ! 1 ! 1 1 1 ! 1 r-I O al

..r., !Jo -o -- CA N. sr.) -4- ...r -- r, -- -o "0 1'0 .cr Co cr. cr. ..- -- CA Ul LID N 01". 0, CA N 0, *AD M +- O) N +- M MA M VA V, to cl CO r, n -o to ro ro in m CO +- 0, CO LO sO h0 0, h, hl -0 •Kt- -4- .Lr

m N1 V, Co M +- CA M +- CA M sn +.- to to h7 M M h, M4 M +- +- M +- M MA CA +- M M PO M M f- 11 • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • 0.1

MI M Co n M M M Co '0 MI M M tO tO Co to to to Co M 'Si to M to 'Si M Co to Co 'Si Si tO +1 I 1 1 I ! 1 I 1 I 1 I I

61

-ro IN CO CO +- 07 +- 'slr +4' LO +- -ID h, NO 1.0 CA M !Kr CO PO Lil CO +- to ro cr co -.1:1 -a LID M III Ci

141 M CA N 00 CA CA h, 4.-- hI7 +- CA +- 1.0 ...o cq sO N) ro co ..1- M-- r!...1 n CA -sr '4" CA sO +- hD I

04 isl L., ...- In ...0 I a N M N Ma .4- +- SA h7 'Si 'Si M MA 'Si CA -O '0 -0 to M N h0 +- to -.1- to -- r- • ••••••••••• ••• • •••• • •••••••• • •• 0

Co to 'Si MI Co Co M MA '0 Co to MI tO M tO to tO 'Si to to 'Si 'Si 'Si to MA '0 M to Co to '0 Co 1-, I 1 1 1 1 1 I 1 1 I 1 1 1 I U

f45

+- -0 CO -.0 0, h, ON CO +- Ni CO sO +- +:1" V7 Os +- OD N +- f.0 IN N. ...... 0-. VI -0 LID CO MI +- M L.L.

Co +- N CA +- .4- CO 'Si N to CO +- 0, 4- f"0 CA CA N) Os 0, to '47 1+1 N. CO R. .- ..tr Si OD N. CO

'Si Co r, to 'Si Co 4,- N .4' CA tO h, tO tO M CO Co M -Cr CA tO t5, M ..-- CO tO N IN to '0 M r, .. • ••1•16••••• •• • • • • • • ••• ■ ••• • • • • • • . • r,.

Co Co Co 'Si tO M Co tae MI to Co 'Si M to MA Co SA M 'Si M Co 'Si Co '0 to M Co to MI 'Si 'Si IZA

I I I I I I I I I I I I I I I 0.1 1-....i -In 173 I--

CC'

+- VD "qr +- CA VI ..tr E 13.4 C.) 1.4.1 1.1. CD DC 4-4 -J 2= 2.** CD CD CD at r iC 04 (...1 L.L.. CD DC -J 1: CD CD CD cn cn 0

Page 93: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

(c) c;11:.7. r- T.: C. iii c in

0 CO

;3

CJ

d nos

D ivioi

aqg zoo void enTeAuebT2

Page 94: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

94

Factor 6 loads B, i. e. the first and second order

intelligence factor.

Factor 7 is not unlike the second order Independence

factor, with major loadings on E and Q1.

However, there is relatively little dispute about second-

order factors, our main interest is in reproducing the first-

order structure, i. e. with single factors loading more or less

exclusively pairs of scales. The eigenvalue plot of Figure 1

was therefore inspected, as suggested by Cattell (1966b) for his

'scree' test for number of factors. The two most likely choices

are for 8 and 18 factors, implying extraction of 9 and 19

respectively to allow for one 'junk' factor into which odd scraps

of correlated error may be rotated to allow for a cleaner solution.

However, eight factors would give little scope for a closer

approximation to Cattell's model, and the choice of 18 looks on

the face of it fairly tenuous. It seemed more sensible to try other

factoring methods involving different definitions of communalities,

namely alpha factoring and image factoring. The first of these

resulted in a solution virtually indistinguishable from principal

axes, but image factoring gave a quite different result.

Thirteen factors were indicated, and again rotated by direct

oblimin. ., From this procedure a much clearer pattern emerged.

Single factors loaded A, B, F, H, I, Q1 and Q2 in both forms, another

Page 95: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

95

G and Q3, but loadings on C, E, L, M, N, 0 and Q4 were complex

and uninterpretable. With hindsight, it is possible to say that the

scales in this last group were the source of all difficulties in

subsequent analyses. At this stage, given the quite different

results suggested by different methods, it seemed that the wisest

course was to experiment with different factoring and rotational

methods, and to look at matrices computed within sex and

discipline groups.

Initially, attempts were made to factor within-group

covariance matrices, but the proprietary software used turned out

to have traps for values over unity in the matrix, and since source

code was unavilable could not be rewritten. By way of experiment,

therefore, within group correlation matrices were factored both by

principal axes and the image method. Eigenvalue plots for the

iterated principal axes solutions are presented in Figures 2-5, but

here the scree test is of very little help: there are no obvious

choices for the number of factors. Image factoring of the same

matrices gives 18 factors for Male Arts, 18 for Male Science,

16 for Female Arts and 17 for Female Science.

Rotation of the image factors by direct oblimin yields

similar results overall. Clear factors corresponding to A, B, I, Q1

and Q2 appear in every analysis, F and H separate sometimes, E

only in the Female Science group, G and Q3 are loaded by the same

Page 96: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

conar.D.cznini-im

0 13 1) W .th 4. 01 I

t.•03 (JR I\) Cli (4

96

Page 97: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

cr DC 2: I/101A Uri

W 4). 4 CA 01 G1 IV 0) 0 Ch so 0 • ti,i'sliKIMMIlan.1.--mi.t,..,pn,vkvstr..z...-2^. Q •

97

•O

• © O

Page 98: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

M

4-0

CJ

•P• ••

tj

(1)

P.)

CD

0

Fh0

CD

lii

CD

1:10

ct In

iR 0

tti

H-" co • tr)

0

IL) •• 0

1 C-)

0

4

9 8

ce) m zr- rn rti

1).• Po% I') (..A) f.,) 41% CA G'1

to co 1.7-1 co 0

• Cr1

(.1

Page 99: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

9 9

co I'll r- z m n c-1 rn

O ad.

N W 4.1 .. cji

.• 01N C0 C9 N

Ei genval ue pl o

t f or th e f emal e sci ence grou p

33N3139 3-1t4W33

4.

eft

0

Page 100: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

100

factor in every analysis, C, 0 and Q4 are invariably loaded by the

same factor or factors, and the remainder show no clear pattern.

Interpretatively similar results emerge from Varimax rotation of

16-factor principal axes analyses of the same matrices.

Thus a pattern begins to emerge. Factor B, as might be

expected since it is not a personality factor in the same sense as

the others, is invariably located; factors I and Q i, which Saville

and Blinkhorn (1976) showed to be clearly independent of extra-

version and neuroticism in forms C and D, also appear clearly.

Of the extraversion factors, A and Q2 are most easily located,

followed by F and H, whilst E is difficult to place. G and Q3 are

consistent but virtually inseparable, and when C, 0 and Q4 split it

is not along the lines suggested by their pairing across forms.

This emerging consistency suggested that perhaps with

different rotational procedures, allowing for deliberate targeting

of factors, Cattell's solution might be obtained. Analytic rotation

programs, after all, are known to have idiosyncracies. Varimax

for instance tends not to rotate much variance into the last few

factors when relatively many are extracted. Furthermore, based

as they are on particular operational definitions of simple structure,

they are not well-equipped to cope with the situation where one or

more variables may be complex, and not amenable to treatment in

terms of simple structure.

Page 101: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

101

Attention was thus directed to the search for a procedure

which could rotate to a specified target. Graphical rotation was one

obvious candidate, and work was set in hand to produce an on-line

interactive graphical rotation program. Procrustean rotations of

one kind or another had undergone development to the point where

only a partly specified target was needed (Browne 1972 a and b), but

nonetheless required specification of fixed numerical values of at

least some elements in the factor pattern.

Hakstian's (1972) Optimal Resolution method seemed to offer

a better prospect. This involves specification of which factors have

high loadings on which variables, which low loadings, and which are

to remain unspecified. The difference between high and low loadings

as specified is then maximised, in the oblique case via a reference-

factor solution, thus avoiding the need for a numerically defined

target matrix.

The sixteen-factor principal axes factor patterns were

analysed by this means, with target matrices set up in three ways:

1. With the pair of scales hypothesised to be loaded by a

single factor specified as high, all others specified

as low.

Z. With only high loadings specified.

3. With only low loadings specified.

Page 102: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

102

For all conditions, values of 1, 50, 100 and 1000 were tried for

Hakstian's weighting constant 'r'. In every case the reference-

factor correlation matrix proved to be singular, i. e. uninvertable,

and the reference-factor structure matrix could not be interpreted

with confidence.

However, a general interpretation of the failure of this

procedure is possible. For one or more pairs of Cattell's scales,

either one or both of a pair is loaded by another factor to the extent

that two factors collapse into each other in the analysis, or one or

more factors can be expressed as a linear composite of the others.

Factors C, 0 and Q4 were the most likely culprits, as noted

earlier, their intercorrelations being more or less identical with

their re/L' 61,6ie;,t,ei , and since analytic rotation had failed to

separate them. Fourteen-factor matrices were therefore entered

into optimal resolution rotation with these scales all targeted on one

factor, but again the result was the collapse of the factor inter-

correlation matrix.

At this stage, through the courtesy of Professor R.

MacDonald at the University of Alberta, a program for unrestricted

maximum likelihood factor analysis (COFA) was obtained, and at

the same time work on the author's own graphical rotation program

(ROTOG) was completed. COFA proved to have two major

Page 103: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

103

disadvantages: firstly it aborted consistently when covariance

rather than correlation matrices were entered, and hence was not

ideal for providing comparable results across groups; secondly

it consumed large amounts of machine time, requiring a separate

run for each hypothesised number of factors rather than

successively estimating factors until an acceptable solution was

obtained. For these reasons, only two runs were performed

separately for each of the four sub-groups, one for seven or eight

factors as suggested by the Kaiser-Guttman criterion, one for

sixteen as suggested by Cattell's model. Analyses for 13-18

factors were run on the pooled within-group correlation matrix.

The Kaiser-Guttman model proved unacceptable for all

except the Female Science group, whilst the 16-factor model was

acceptable for all groups. For the pooled within-group matrix,

the 13-factor model was just acceptable. The number of factors

was then increased until the difference between chi-squared values

for successive numbers of factors was no longer significant at the

.05 level, which occurred after 17 factors had been extracted.

Test statistics are presented in Tables 8 and 9.

Graphical rotation of the 17-factor solution was attempted

with a view to targeting factors on to pairs of scales as before.

For some factors this was very straightforward. Factors

B, F, H, I, Q i and Q2 were rapidly isolated. Factors G and Q3

Page 104: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

104

Group No. of Chi-squared Degrees of 2 factors freedom

Male Arts 7 395.14 293 0.00

Male Science 7 377.01 293 0.00

Female Arts 7 450.05 293 0.00

Female Science 8 280.24 268 0.29

Male Arts 16 75.34 104 0.98

Male Science 16 72.45 104 0.99

Female Arts 16 86.05 104 0.91

Female Science 16 59.80 104 1.00

Table 8: Test statistics for maximum likelihood factor analyses of data for four sex x discipline groups

Page 105: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

105

No. of Chi- squared Degrees of E factors freedom

13 193.26 158 0.04

14 159.61 139 0.11

15 135.31 121 0.18

16 102.82 104 0.53

17 72.66 88 0.88

18 60.22 73 0.86

Table 9: Test statistics for maximum likelihood factoring of the pooled within-groups correlation matrix

Page 106: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

106

r--4 0 1.11

V7 117 n ,-.. CD 'St- ,- '..0 Cr-, -o er co c, CO ro -- ro 0, CO Ci. T.) t'0 0 !"0 er ro cl u) n er (-4 v.)

u- r, co (-1 ill C4 ro C4 b7 V7 !", ,-- VO !", er n ,-• u-) b-) --t. n L....! Ls) lin L-1 -- C..1 t::zt ;7.1 'kr 171 ,-1- .....

)... CS! 1.,,-4 1,74 ei4 iSI C4 n e.'ii eiI C4 C4 tS! rs. n n r, n e, c) • • . ••••••• • • •• • 71 la •• ■ •• le....,...• .1., n n n n n n

r t n r V' n n. St CA eA e;:i i:!,:l n C, C, n n n tS4 tn n t.S.1 C, n n n LI

1 ! 1 I I I 1 ! I ! 1 01 q-

Cl N.. b7 0" 127 0- CA , - CD 1.-, b7 *44 0" ,0 ..-

CO i'l i-, CA KA "0 SI n er -- ro ro n c, er bl ts) ro -1) f,.. b7 CA ,-- VD V7 SI b7 VO n N) tS4 er r-:

co ts! n - St C, n f4 *-- C, =- ,- *-- n CA vS: C, C, M CA SA M CA VS: S4 =-- VIZi Ci Ci Ci ,..- "-- v- ru • • • • .... • • .1 II • Is • • • • • • • V. a. • • • • ••••• e)

m Ls) m! n n n ts! ts! ts! n ts. n m m n r-,1 n CS! ,-- n M S4 '<A SI n SA M n S4 ES! SA SI .4-.) 1 I I I ! I I I I 1 I I I I I I I 5:: •

C.:

IN N) V7 OD 0, VO 0" 1.-, "CI CA b7 M VO U7

n n es) CD SI c,:::" ,-- VO CA ,---. CA CA "0 "0 CA PT1 0

r'w Ci• ei) e:I eili e'.1 '5I q.:: C, St V) eA e=4 er m n C, n ts! S4 C, M r, n co - n ts) r, n Cr, er) m ul

vs • ■ ••■■ •• Is • • • ft Is •• • • • • • • 0 • • • • • • • • •

n n n n C, n n "Si C, n C, n n n n n C, n n n'n Ci S4 *A S4 Ci n Ci =- SI n SI 0

! 1 ! ! I I I II! I ! 1 I

,- ,- l'", .:!' n =- VD er CO SA U7 N' =1" b7 ._!-) ,-- +- Ci b7 CO b7 r, C IN -- n .0 -0 "0 CA 0" ..,.e.

C4 ,-- n U7 "0 "0 !.', V7 -CI "0 =- C. ,-- CA -- ,- .4- CA V7 . - C4 CA 0" *.1- <A ,--- "0 S ,--- . - OD -0 !"-- -40 =-- S4 +- mt co C n n =- C, C, ts! Ci ts: r.:;,! C, ts! if, •-• ts-I 0.- n - !C, Ls! ks! 'N riz, n in • • • •••••• ■ ••• • • • .4.....

Ci S4 Si n n St SI VS! r' V KA tii,I eil e> el) C, n n M n n n n n n KA S4 SI n n S4 St ! I I I I I I . I I I I I I U)

,c) OD IN ,D V7 CO V) I'', VO -0 CO CA CO "0 ,D .C.: e)

b7 SI S4 +- SI S! n MA n ...- MA n 0.4 MA MA n U,I ,.-::f MA MA t'.4 n ,..... n ..- 1'2! n tn n n C,- 4::

•••11•••• • ■ ••%, 'Ma • • ••••••••• •• ss••

Ci- Ci

Co Ki4 e:,) Ci e'il e.-4i ei) eiI tr... eil C, tSi eti C! e1 eg.:. C, tEl e.: Ci elI LSI tS) eS) CS) Co-: ..- M C, n SA ,--1

1 ! I I I I I I I I I I I 0

er ct- 1.-) ro -o r.,•r"-! =- SI -CP =- -40 =-- V7 f". CO CO CA i', CA KA ^0 C4 C4 n ^0 n er oo -- CD cq C 0, CO ,0 Er.1 ,0 0, e,i t!!;:i t", r', ...- 17'.: ..-- r, er

5- er v-... C, n tel es. r, - et) te cl -- ro - CH:, CO -- et. CS) n N) n n m n m CS) SA tn.) n m co r'" Ili 0 . • • •• ■ •• • • • • • A • • • • ■ • • ••••••••••• 'I..'

1-, C! n MA n CS) SI n n CS) MA n MA M n M n '"! n n n n CS) n n n M M M CS) CS) CS) n -t-, LI 1 1 I I ! 1 . I ! I I ! 03 fi, ea. L.L. CO r4 =- r--... m Cq -0 .-- 0" CA 0, N7 ,- V7 ,- )-- tn ,0 N. 00 0.1 N U7 KA t-, 0" ,-- .1: b7 "0 er cA I

CO ,-- ..-- =- CO .1- 0.'- l', ,-- b7 "7 0- )- 1.1 CA IN co n cq er OD V7 Ci •cl- n 0.0) er

N) CS) n SI b7 .- C) +- n S4 ^0 ,-- SI 'N <A KA +- CU S., St SI SI C, n clA n .-- SI n n CS) 15) SI ,D 0

... . . • . • . ••• • • •••• • • is••••••••• • s e• ,...1

• IS) n MA M CS) M M ,ts! n to CS) Se C5) SA SA v54 n vs! n Ci n n n n CS) n CS) n C5) QA St U ! I I I I ! I I I I I I I I I tr.,

1.2...

41' C.0 r-, N. 1.0 Ct.- "4" tn +- in 0' `.0 0.• Cl r, - r, n e- co ,D IfI. U7 CA ,-- CO CA er ts1 -o - rq uo ro - r, CA c-t ro 1:7 "0 er er ro ,5.4 .0 (-.4 __- er ez! cr. Ls! Ls! - es! uo un Ls. IN - er ,--- -

cq in CS) CS) n n n es! n CS) CA CA St S:, S SI flA CS) c. 15) ,- C.::: CS) c5) n ..-- SI n CS) M ,- ml Cl 0 ••• ••••• %V •••• • • • • fals••• •••••■ •• •• ,

CS) C. in MA n n tn t.,A n St IS) CSI ES) e:.) CSC C, n S4 SI M M n CS) M n ..- MA n .i) Cs! CS) C5:, Ci I I I I I I I I I I I I I I ! I

.1-1

If) 140 ro ^0 IN ^0 V, CO CA N7 n n .1' b7 VO N, U7 U7 V) ,-- .1- CS) SA 1.17 M ,-- ^0 CO "1" to 0. C-

r, r, 44- ^0 er c.-4 - IN If) -,-/- .4- ^0 f'0 N. N1 -•:!. ..qr r.... C5) cg ro IN - CA CA cq 'C CS) CS) CS) IN Cl 671

..- NI MA MA n m n t... Ls) es PS) n Lsz: to m ts) n C, n n - CS) Ls:: es! n PS) e1 C" n Ls! CS) n est • • . • • • . • • • • . • . • • • . • . • • • 4•••••• ••

St ,SI el..? CS) MA n CS) M MI MIA MA MA CS) MA Co, ..,- MA n t5) MA MA n MA MA to n n in CS) n C, M 1 I 1 I 1 1 I 1 1 I I

- er

N) er cl ro e- r 0) Ci.)Lt.! CD 2 !--4 -I 1: 2: CD CO CO CO CO 1.174 Ci 1.4.1 CL. CO I-I .qr. C) CO cr. Ci Ci

0 LL CL

Page 107: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

107

0

r.... CI 1-, !...1 ra

c!--

r, ro -o :--.1 uo ,- c- IN r, C4 NO N, .-- 0, ,0 !", CO m! NI N) a CA Cl "1" ND .,-- C4 C4 •-• C4 ,-- C4 a !--.

. . • . • • • • • • • I: al... • a et • a • It II 4 n • a 0

to t *4 n t' n IS! n n n *4 *4 to

I I I I I ! I I I ! I ! 1 I != ID

C4 tr) "{) *-1-. 0... TN n o% — co r, -a c-%! a ND NI I", h-.I a a !"-, NO ..,- CA CO CA tn r•, CD NO 0, In :: .

NO -.0 k!-.1 '4" -0 c...! -o c..! c%! co r% r, — -4- ''.- '0 N7, n In tn •,?' OD F") h", n ..- •-- IN r',..! !,--. ,-- 1.1) tS! si

II a • •

••••••• • • •••••• ••• • • • • •••

n *4 *4 IS! t) n n m! *4 n *4 m! n n n *4 IS! *4 *4 n 0

! I I ! ! 1 I I ! I ELI .1..)

uo ro

r, -.0 co o- P... r•O Cl 0, CO ,- ,-- -,-- IN Cl

m! 0,4 -- PO tn 1'0 'Kt' tn tn tn n n tn =- ,-- tn tn .-- n n tS1 tn.; t • • • • • . • . . • • • .t • el • • • • • C!....

t'2. ,.. E.... 174 ES! m. ts! m! m! *4 *4 m! S! *.! n ISA C.4 n n n n m! m-4 ES! ES! r.,! ES! ES! L I I I I I I ! II ! I' ! I ! ! 01

t-1 ',I" t0 a!

• - • • • • . • . • . • • • • • • • • • • 0 n K. *4 n n n n n n n n *4 n n n m4 n m!

1 ! ! ! ! ! i I 1 ! ! 1 I ! III ,---

vo r, Pr, 0, V) N;3 0....z! ro C4 N, ,- 1.!') ro . -- 1:"..1 ro ro llo ts4 !--o — crzz! 0, co 5-

:- ,--- ES2 t'.1.A .. - En "4:1 tiz! ES! *-- C..! ..:1 CA tn trn n n ,-- tn n t'Ir:4 n 0,-1. ES! MI ,•-• tS! ES! ..:1 1S1 ,- •!-- al 0 • ! • • • . • • . . " • . . • 4 •• la • • • w • • -!...'

n n n 0,4 m! 0,4 n n n n n n n n n n 0.-4 014 04 04 n n 0,4 m! 0,4 %:-,! n mo -,!--,, LI 1 I 1 ! ! 1 1 i ! ! 1 ! ! I It. 65 cl„ D.- CN ,- IA? OD -4) CO C..! h71 c4 0,4 o% ro -ID -o 0- m! I

0.34 n ro n m4 ro 04 n n m! n 04 ro ro uo

— ml cr- n n n 04 m! n n n n n n n n t n n• n n m! n m! n n n n n n n 0.74 0!:-.1 o . . . • • • • • • • 1. . 4 a •• '".

•I■ •• •• • .1.-:,

*4 n *4 n m) *4 n IS4 mA n IS) n n *4 I-I

I

I I I I I ! ! ! II I It. ...1-1

n v.:I ,- m. cl !!,:.4 n c!!. ,- *A ,- ,- *4 *4 to n tS! tEZ! ,- m! n t''' m4 m! n n m.4 *4 M. n m! C!

. . . • • 11••16 •••••• am 4111•• ft 44 • • •• • 3

1 ! 1 I I I ! I I I I I I ! ! I .P r-.

n ea ,- n -- -- n n Si n oo n ci -- n cq n m! n 0,4 n n -- 0.4 n 014 0,4 — mi n n n • • . aa II •• ■•••el•Yli ••••• Is If•••••••••

I I I ! I I I ! I I I I I I I I I ,-

Pc! (.._) DJ Lt- CD

0 U-

y2-4 !ET:

c 0.1 01 01 r•-• 0

cq

U-

r) PCA LLI U. CD _J 1: CI ci cn co ct!

Page 108: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

108

split into one triplet and one singlet factor, with factor G form B

loaded by the singlet. C, L, 0 and Q4 split into a 'general' factor

and three singlets. Factor A scales were loaded jointly by two

factors, and the form A scale by a singlet in addition. Factor M

form B is loaded by a singlet, and the rest of the scales have

complex loadings. The same matrix was also rotated by direct

oblimin, with such strikingly similar results that the factor pattern

in Table 10 is representative of both solutions, being in fact the

oblimin results. But once again the optimal resolution method

resulted in a collapsed factor space.

14-, 15- and 16-factor solutions for this matrix were also

rotated graphically: factor A consolidated into a single factor, but

otherwise little improvement resulted. However, in the long and

complex process of performing these graphical rotations, certain

features not immediately apparent from rotated factor pattern

matrices concerning the configuration of projections of test

variables onto the factor space became clear. Firstly in each

COFA solution the communality of one variable, different in each

run, tended to be estimated as noticeably higher than the others

with the result that a corresponding singlet factor was found.

Secondly, it proved quite impossible to steer factors to load

separately the C, L, 0 and Q4 scales according to Cattell's model.

Thirdly, the only way to separate N from E was to allow the E

scales to be loaded by factors corresponding to H and the anxiety

Page 109: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

109

scales. In passing, it is worth commenting that since many hours

were spent trying to force Cattell's structure unsuccessfully onto

these results, corresponding perhaps to many months of labour

without computer assistance, either the possibilities of rigging

results by this means are exaggerated by its critics (e. g. Eysenck

and Eysenck 1969 p. 327), or the structure of the present data is

uncommonly well marked.

Essentially the use of maximum likelihood factoring and

graphical rotation have provided no information over and above the

13-factor image solution for the total group. The unrestricted

analyses here reported fall essentially into two groups: those using

the Kaiser-Guttman criterion for number of factors, including

principal axes, alpha factoring and COFA 7 and 8 factor solutions,

all of which yielded similar approximations to the Cattell second-

order model, and the remainder, using either the image-factor

criterion, the goodness-of-fit test in COFA or simply an arbitrary

choice of 16 factors, all of which agree more or less on which pairs

of scales are clearly loaded by single factors and which are not.

The factor patterns of Tables 7 and 10 respectively represent these

two types of solution.

Thus there appears to be partial support from these analyses

for Cattell's model. It is not suggested that any of the solutions

discussed should be regarded as definitive, as their purpose was

Page 110: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

110

simply to facilitate a decision as to whether the expense of

computer resources involved in a more adequate test of the model

was worthwhile and to gain some idea of whether splitting the sample

by sex and discipline would have any serious consequences. While

it is difficult to demonstrate concisely, the impression was gained

from a study of all the exploratory analyses conducted, of which

there were over fifty, that splitting by subgroups tended to result in

less correlated, more identifiable factors, without noticeable

shrinking of the size of factors associated with scales on which there

were large mean differences across the groups. However, this

tendency was not so strong as to prevent the 13-factor image

solution for the total group giving results very similar indeed to

subgroup analyses.

We now turn to what was intended as the definitive procedure

for determing the fit of Cattell's model to the data.

Page 111: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

111

CHAPTER 8

Use of unrestricted methods of factor analysis in hypothesis

testing may, as has been indicated, lead to the confirmation of an

hypothesis, but it is difficult to make an incontrovertible case for

disconfirmation, as the history of critical studies of Cattell's work

indicates. It is always possible to claim that particular matrices

are not suitable for a particular method of factor extraction and/or

rotation, since communalities can be estimated but not defined,

criteria for rotation are to some extent arbitrary, and there is no

way of arranging computational routines to ensure that correlated

measurement error is not extracted along with common-factor

variance. What is needed is a method whereby an hypothesised

model may be specified beforehand and formally tested against the

data.

Such a method has been provided by JOreskog (1970) in the

form of the technique of analysis of covariance structures (ACOVS).

Recalling the matrix formulation of the common factor model in

Chapter 4:

= A cPA'-i-W2

Page 112: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

112

where A is the usual matrix of factor-pattern coefficients, (1)

is the matrix of factor covariances and is the (diagonal)

matrix of unique-factor loadings, elements of any or all of these

matrices are directly estimated by maximum-likelihood, subject

to whatever constraints are imposed on the model. The ACOVS

model, which specifies only the structure of the covariance

matrix, may be extended in the usual way to impose a structure on

the means, and in particular to the testing of a model in more than

one population simultaneously. The analyses reported in this

Chapter were carried out using Jbreskog and Van Thillo's (1970)

programs for simultaneous factor analysis in several populations

(SIFASP), which are simply revised versions of the original ACOVS

programs allowing easier specification of certain parameters.

The ACOVS method satisfies almost entirely the require-

ments for a decisive analysis. Firstly, covariance matrices are

acceptable (indeed preferable), thus allowing for different scale

variances across groups. Secondly, Cattell's model may be

expressed to varying degrees of strictness in terms of the para-

meters of the model. Thirdly, a test of the goodness of fit of the

hypothesised structure is provided, allowing at least a comparison

betWeen alternatives. Fourthly, a structure may be specified in

such a way as to exclude correlated error fromthe factor pattern

matrix. However, indeterminancy still remains to the extent that

Page 113: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

113

a well-fitting factor model is not necessarily the only model that

will fit.

The SIFASP program was obtained and modified to suit the

size of the problem. Since a 16-factor solution for 32 scales

implies 512 independent elements in the factor pattern matrix, 136

in the factor covariance matrix and 32 unique factor variances,

there are 680 elements altogether for a single population. However,

most of the factor pattern coefficients are implied to be zero, and

the major problem was to expand the program to actually allow

simultaneous analysis of more than one matrix. Expansion up to

the estimation of 300 parameters proved possible, at which size the

program occupies 88K words of core storage, rather more than

half of which is accounted for by the matrix of approximations to

second derivatives in the minimisation procedure used. Since

this matrix grows approximately as the square of the number of

elements to be estimated, the modified Fletcher-Powell algorithm

used in SIFASP is clearly a constraint on the usefulness of the

program for large problems.

Variance-covariance matrices were computed for ten groups

and combinations of groups as shown in Table 11. These matrices

are to be found as Tables Al to A10 in the Appendix.

Page 114: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

114

N

Male Arts (MA) 233

Male Science (MS) 312

Female Arts (FA) 396

Female Science (FS) 203

Male Total (MT) 545

Female Total (FT) 599

Arts Total (AT) 629

Science Total (ST) 515

Pooled Within Groups (WG) 1144

Total Group (TG) 1144

Table 11: The ten groups and combinations of groups, with sample sizes, for the restricted analyses

Page 115: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

115

Initially, a strict form of Cattell's model was tested. The

four basic sex x discipline groups were analysed in a single run,

with each scale specified to be loaded by a single factor, the factor

pattern coefficients being constrained to equality across groups but

not across forms. The factor covariance matrices were free, but

constrained to equality across groups, and unique variances were

left entirely free. Starting values for the minimisation were:

Factor pattern coefficients 0. 6

Factor covariance matrix: identity matrix

Unique variances: 0. 5

There were thus 32 elements to be estimated in a common

factor pattern for all groups, and 136 in the factor covariance

matrix, again common to all groups; estimated separately for each

group were 32 unique variances. The total number of parameters

estimated for any group was therefore 200, of which 168 were jointly

estimated over all groups.

Since the Fletcher-Powell procedure converges approximately

quadratically, it is possible to judge whether a given model is likely

to prove satisfactory before the minimisation procedure has actually

converged. In the case of this analysis, it was clear that an

acceptable value of chi-squared was very unlikely to result, since

at the point at which the difference between successive-function

evaluations was 5% the corresponding chi-squared value was still

approximately six times the number of degrees of freedom.

Page 116: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

116

The ideal procedure to follow at this stage would have been

to estimate a unique but unrestricted model, and then successively

impose constraints as recommended by Mreskog (1971), however

the size of such a problem exceeds core storage on any computer

available to the author in this country, and furthermore it involves

the estimation of many more parameters than there are subjects in

each subgroup. Therefore the next step was to test the model out-

lined above on each of the ten groups in Table 11, again specifying

two scales per factor and leaving factor covariance matrices and

unique variances free, since pilot runs suggested that removal of

constraints across groups would lead to a substantial reduction in

chi-squared relative to degrees of freedom.

Accordingly, when sufficient computer time became

available, this analysis was run to convergence on each of the 10

matrices mentioned above, each run requiring 2-3 hours of

processor time on the Hatfield DEC-system 10. The results for

each group were strikingly similar, and will be discussed in detail

shortly. However, the goodness-of-fit test for the model as

estimated still gave highly significant results on each run, and

furthermore in every case the program converged to a solution in

which the factor covariance matrix was not positive definite.

A number of experimental runs were then tried, changing

starting values for the minimisation, and gradually combining

Page 117: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

117

specified loadings of factors on scales towards Cattell's second-

order factor model. Changes in starting values had no effect on

the solution obtained, although using the factor correlation matrices

in Table 10.2 of Cattell, Eber and Tatsuoka (1970) as starting values

for a sixteen-factor model speeded minimisation noticeably.

Reducing the number of factors to thirteen, by specifying loadings

of single factors on C, 0 and Q4 on the one hand and G and Q3 on

the other, whilst allowing L and N to be completely free, did not

improve the goodness of fit, whilst attempts to fit the complete

second-order factor model were abortive, resulting in negative

variance estimates for one or more factors. Allowing the number

of parameters to exceed the sample size did improve the goodness

of fit, but not surprisingly the results of such runs were entirely

uninterpretable.

It thus remains to interpret the ten separate analyses by

subgroups and groups with their apparently poor fit and singular

factor covariance matrices. Table 12 gives estimated factor

pattern values for each of the groups, all but two loadings per scale

having been set to zero by hypothesis - since each scale is loaded

only by a single factor, these are also factor-structure coefficients.

Table 13 gives an example of a factor covariance matrix (lower

triangle) with the off-diagonal elements in the upper triangle

rescaled to unit factor variance (i. e. correlations). Since these

Page 118: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

118

G4 0 .t.i

0 E-4

L.,

E-I CO

E-4 (si

E--i

CO

tip

z 2 o

N N

0

en N

0

0 N

0

CO N

711 N

6

s.o N

0

N

O

CO N

.0

.0

0

CO

CO N

0

,J0 CO

11.) N

0

00 N

CO co 0

.0 N

0;

Lt1

0

N co

in .0

0

-14 N

'11 N

0

TV N

CO .0

0

N N

'II N

Li-) N

0

tin .0

0

it) N

0 N

0

co N

ON N

0

■0 N

0 CO

0

C. N

N N .

t4 N

0

N CO

t

N N

O

.0 N

0

1 N

.--1 co

• O

.-I CO

0 CO

0

0 co

N N .

7t, CO

0

CO

0

.0 N

o

0 CO

0

•ti N

0

IC) N

ce) N

0

714 N

cn N

CO N

6

ON .0

0

CO N

o

LU CO

0

tn N

0

cil as

• O

NI ON

.-1 ON

0

'74H Cr.

--4 as

't 0",

0;'

oN

0

ON co o

co CO

0

N Crs

0

00000000000 00000

N co

• 0

tr) in

(;) N

0

.-i .0

N .0

•,J0 N

• 0

L.rt

0

.0 rn o

cn .0

0

N

• 0

0000000000 00000

..-I ...o 0

0 ..o

0. in

0

,..4 .0

,..-1 .0

0 ,0

• 0

•714 in

0

s-O .0

o

,--1 .0

0

714 t.11

C0

00000000000 000

.fo -14 0

rel .14

•11 7ti .

0

oft n61

,--1 .11

N L.C1

0

tf) en

0

Cr, on

o

c:r, 7ti

0

N

0

LU mot' 0

cil ,711

LC) cn .

0

N In

N 7t4

Qs (-11 0

Lf)

0

CO LU

o

N cr)

0

cr.)

CD

in N

• 0

N N

N N

C:;

.0 N

.0

.0

N N o

11-1

0

ON .0

LI') N

0

N CO

O

°do

cr, ,..co 0

I-C1 .0

as in

N N

cil .0

,S) .0 .

0

CO cr

0

cr. .0

co if)

0

if) N

0

0000

a- in

0

0 .0

r...1 .0

CO LU

N in

'70 •-.0

0

CO if)

0

co Lf1

■D .0

0

M •.0

O

on N

• 0

N N

.0

.0

CO N

,-.4 N

in N.

0

N

0

co N

o

0 N

0

N 00

O

----co

• 0

CO N

ON N

cn co

.0 N

0 co .

N

0

o co o

0 CO

0

•zt4 CO

0

N re)

r:C1 w w 0 x aZ 0 a aa (r)

Esti

mate

d fa

cto

r

C)

›-• cd 0

Table

12

part

1

:

Page 119: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

119

N 0 O N u1 CO .0 in .0 e•••4

N -44 N N co .0 co N to N Ln CO

s-SO

&.4 •

CO CO CO 0 CO CI CD CO CD CI (0) 0 0 C

CD .0 Cs in d, ON lf1 N N cf) dl .0 CO .0 d4 cn .0 .0 co .0 co .0 u1 N ul N N r-

0

0 0 0 0 0 0 0 0 0 0 0 0 0 0

0

't CO 714 CY' dt dt • r-4 4-1

-

Cn

▪ N c.r) .0 .0 co N co .0 in N u1 N .0 .0 N E-

o o o o O O

o c0 O cr O c0

o

•::14 N dt tf1 In as ON CO d, (41 N N ce) •::14 N N co .0 CO .0 Lr) N in co .0 co N co

. . . . . . . . . • 0 0 0 o 0 0 0 0 0 0 0 0 0 0 0 0

.0 -44 N 0" CO en ce) L.r) M cc) o co cn (-€) .0 N .0 co .0 7r1 .0 In N co N N N

{1:1 LT-t co

o° do o o o oo

oc000

C:4 0 E-I co ,--I In in ,--I Cr, as N N .0 In o co -14 -14 N N -71-1 N .0 oN .0 CO .0 in N in co ul N N r-

C.1.1 O d o o o o o o o o o o o o o o I

0

Cn co (NI N cr, N 0 71-i ce) N C' CO 't Ce.) In ri -i-I

Cr) CO Cr/ in N N CO .0 "714 .0 Lc) .0 CO N N N (..) ct3

• 0 0 0 o 0 0 o c; o 0 0 0 0 0 0 0 7:3 U

N --I co NI N -t Li1 c■I cr) in co co -7ti on I.-I Cd

...• JD ,Z44 ....0 N N NO CO ,-.0 t11 N LSI N N CO N CO ri-i

C. -70 co co in o's •ti ,--1 0 .71, •--4 0 in o --I N

CrA .0 d, .0 .0 co .0 co .0 .0 N in co ul .0 N r-- c•1

0

C o 0 0 0 0 0 0 0 0 0 0 0 0 0 .. +4 F-■

Saf cd O P-4 O ...? CO 0 •0 c's in ON '14 N 'TO CO N d, 0 N 0 d, P --, in in CO u-.) 0" sr) 0" ',CO to CO `-.0 CO •-0 ON CO CO

N

000

000000 00000• 00 CU

• cn as u C=1 f.T. "..71

-

1-; z oa

ctaa U

e te

n SIF

ASP

att

ern

coeff

icie

nts

fo

r t

0 0 0 0 0 0 0 0 O 0 O 0 0 0 0 O. +4

cc

Page 120: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

120

NO a r. -.0 T... r4 h, MM a MI OD ,0 CA 47 4 a uo -c1 r I M IN to cr. co a cc r '7 a cr. u, CN 'Si Cl- 'Si 'Si CA .t.4- 'Si 'Si L-..1 No 0.. • • • • • • • • • • • • • • • • m CP M CP CP CP CA CP CP M CA CP CP CP CP

I 1 I 1 I 1 1

CO CA NO CA AT 47 CP CA CA a- co op pa op ,4-

rn -40 a m IN .,c1 m r, N. or a- -,c1 - N7 NI NO P0 G.

CN CA CP NO -- -- 00 CA CA CA CA .tr .0 ..- -- p., ,_ .7..i • • . • • • • •••• ••••• CI

CD 'Si (Si'Si 'Si CA CD 'Si Si 'Si 'Si CP 'Si CP ..-

I 1 1 I I 1 1 1 I 1 CM

or oo a to m oo or -- CA CP PM MI 1.0 47 P., ui

CJ -- 4 CO r, -- or AT ,O P, CO CA CA ,0 4= CN 47 CP N7 -- 43 ,0 C,1 M1 M

••• • • •• • •••• n••• tT m rsi m mmm m mmmmmmmmm

11111 1 al

• m r. CA 0, 47 47 OD 0, Pa. CO 47 47 M a M.1 MI C4 CO 0, P0 m o- o- -- co r, r,

CV CA PM CA ro to a CA -- CP CP • . • • • •• ••••• • • ••

CP CA 'Si (Si 'Si 'Si 'Si CA 'Si 'Si WA M 'Si CP WO 0 1 I I I

0, PM PM Cr- N ND PM .qr WM CA M- NO N. r4 OD 0, VO r4 N At CP 00 Or. NO CA 0, cc 0, 00 41

O CJ m 'Si M N- ,0 NI a - m 'IZr CA CA OD WM • • • • • . • • • • • • • • 43

MMMM M M M 1Z 1 I I I 1 1

NO r., ON V, I', CO Cr` CO NO OD a m PI NI C..1 cc .... Cr- a- or oo -- ro - m r. -- N. ro or a 0-

r. at

m CP CP CP 00 r, .v ,r ._ ,r is, ,_ 'Si ,r __ b, (...4 r•-• • • • • • • • • • ••••••• 7.r.,

I 1 I I 1 1 I i IS •••/

a PM CA 47 Pa AT P0 47 NO op CJ pri m ..- m 1..,

a 0, CA 0, 'Si CN OD CA 40 -0 Cr- N. r, r uo to 4= 31: CA .-- CP .4- -- ro -- -4- m is, .4- -- uo m CA CP

. • • . • • • . • • 1 b • • • • • r_

WA (Si 'Si CA M 'Si CP CP Si (Si CP CP M M Si (Si al 1 I 1 i i I 0.

is WA CO 47 ,0 CA AT NM ND -a - 47 43 CP CP N 1.17

Pa 0, 41 M -0 ...- CA .-- Pa 47 PM CA 47 h, 47 0, -- J CP CA 43 ,0 PM .-- CP 'Si CP CP 41 AD .4 -- ro o, • • • • • • . • • • • • • • • • I.-

CP m CA CP CP CP CP CA .- CP CP CA M CP CP MI 11 I I I I I I 1 1 ..4

C..1 ...... CN 10 T... era '0 47 N. 0, 43 .- CO -.0 'Si M

CA CP AT CO 0• CP 0, 4D ..-- a- m P0 .4- CP 0, .- ...-4

1-4 .- NM a- CA CA .-. CA OD CP NM '..- AT CA .-- CA 47 a, • •• • •• •••• • • •• • 5-

CP M M.! (Si 'Si CP CP CA CP CP CP CP M M M M S. I 11111 1 1 I 0

U CA 0, N N 47 NO 03 CO 03 ND N. UM M CO, NO CA ,-- PM 0- CA NM OD a- - r4 co r4 to N. 47 ,O N)

7C 47 CP 47 Pa P, M PM P0 CP .-- 40 CO .-- 47 P0 N. = • . . • . . • . • • . • • . • • M

CP M CP CP CP CA .-- WA CP CP CP CA M CP CA CP

I 1 I I I I 1 cu

.....

4 CO CO •-- CA r, 0t] 0, -0 47 vo o, ts,r000 oN ,-.4

-- -- No uo (NI -- -- -- .4. CA CO .-- PM OD 47 CO CM

CO PM -- - CA N 41 .- .- ,-- 4 47 CA 47 -- .- MM C

. • • • . . • • .. • • .. • . M

M CP MI CA CP .- M CP CP VA qA MI Mt M .-- M '1'4

1 1 1 I 1 1 I I I I 5-

CP 0, At 47 ,0 N. MM 47 0, CA .-- OD h, C, CP -

47 PM 47 AV VI CA r4 N a- MI 0, r, -o MO 40 5-

U. MI - CA P, a- ro m NM a- - o- ro ro uo CJ 'Si Cr

. . • • ••••••••• ■ II • 3

CP FA CP m- M ,-- M CA CP M CP CP CP CP CP 0

1 1 1 I 1 1 I 1 .....!

N. N PO Pa PM .4- as uo cs c...! so N. as ro CA C4 MI Cr- a ro as cs CO ND AT a uo to uo Qr

uu m r4 r4 0- VM Cr- C..1 40 a m AT N CP • • • • • • • • • • • • •

M M M ■- MMMMM CP CP CP M CP M

1 1 1 1 i I I 17-

N. R. PM 0, NO CA OD MI PM to a 471 Cr, -41 cr m (5, -40 CCI to a 'Si Cr' cr r4 r4 a- co a- C,1

t.,J P0 CP PM CA P0 CA 00 a N. m a m P0 03 47 CI • • •••••••• • • • •

M CP CA CP M CP CP CP CP -- 1 I

a -o -.0 a a- OD 4 Cr- M '0 CO (5! -0 OD N. -0 4= a- to m CA CP NI CP P0 CA N) MM CA MO 47

04 CP 47 'Si CA 'Si CA CA IS, M CP 'Si (Si CP 01 . • ••• • ••• •11••••• Li

'Si CP 'Si MCPCPCP CP CP CP 'Si 'Si CP M CA 'Si 1 1111 1 1 1 • •

MM 0, CA N. P, P, WM ND 'Si Cp p, oq m 00 N1 PM AT VI m r, r, N. m 'Si to a cr. m

.Cr 0, CP PM ro uo m CA M7 m m • • • • • • • • • • • • &IONS

CP CP 'Si 'Si W CD 'Si CA CP (Si CP 'Si 'Si m 'SiM rs i1

1 I I I 1 I I 1 IV I-

CJND a CC 04 CD UJ U. G0 VC r..r J3= 2C CD CV CV CV CS

Page 121: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

121

matrices vary only slightly from group to group, the rest are

presented in the Appendix.

Table 13 suggests several sources of the collapse of the

factor space: note particularly the correlations between factors

corresponding to C, 0 and Q4, between E and N and between G and

Q3. To locate the source of the problem, the matrix was first

tested for rank by the method of triangular decomposition, with the

result that it appeared to be of full rank. Next the eigenstructure

of the matrix was investigated, first by the Jacobi method. This

suggested eight positive and eight negative eigenvalues, of

approximately matching magnitudes. Since this seemed rather

implausible other methods were tried, notably the Householder-

Wilkinson tridiagonalisation approach. These indicated 13 positive

and three small negative (= -0. 1) eigenvalues. The triangular

decomposition analysis was then repeated with a higher tolerance

criterion, and gave the rank of the matrix as 13. Finally the

matrix was reconstituted from the 13 positive eigenvalues and their

associated eigenvectors from the Householder-Wilkinson analysis,

and found to be identical with the original to two decimal places.

Since in general with ill-conditioned or near-singular

matrices numerical analysis methods are particularly sensitive to

the numerical accuracy of the computer used, different methods

may give different results. However, corroborative evidence is

Page 122: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

122

available for a rank of 13 for this matrix. The triangular

decomposition analysis proceeds by exhausting sequentially the row

and column with the largest residual entries at any stage. Rank is

determined by a present criterion level, such that when no entries

remain above this value, the number of rows (and columns)

exhausted is the rank, and the remaining rows and columns are

considered 'non-basic', i. e. they may be expressed as linear

composites of the 'basic' vectors. In this case, the vectors

corresponding to factors L, N and 0 are non-basic, and the two

smallest (i. e. least independent) basic vectors correspond to Q3

and Q4.

In this respect, the results of the SIFASP runs are very

like the results of the more traditional factor analyses discussed

in the previous Chapter. The same scales consistently fail to be

separable into independent factors. However, as has been noted,

chi-square values for these analyses are highly significant,

suggesting a poor fit of model to data. Table 14 gives the results

of the goodness of fit tests. McGaw and J8reskog (1971), however,

suggest that with substantial real data bases, chi-square values may

be very much inflated by a combination of slight deviations from

multivariate normality and small amounts of correlated measure-

ment error. Since JUreskog's function for minimisation, whose

value at the minimum is the basis of the test of goodness-of-fit,

Page 123: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

123

Group Chi-squared Degrees of 2 freedom

MA 549.11 328 0.00

MS 619.43 328 0.00

FA 695.15 328 0.00

FS 475.05 328 0.00

MT 825.58 328 0.00

FT 839.33 328 0.00

AT 965.76 328 0.00

ST 778.87 328 0.00

WG 1248.85 328 0.00

TG 1269.48 328 0.00

Table 14: Test statistics for the ten SIFASP analyses

Page 124: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

124

involves the sample size as a multiplying factor, whereas the

degrees of freedom for the chi-squared test are determined by the

numbers of fixed and free parameters in the model estimated, taking

no account of sample size, large samples in combination with small

correlated errors may increase the value of chi-squared out of

proportion to the size of the absolute discrepancy.

To overcome this problem, Tucker and Lewis (1971) suggest

a 'reliability coefficient' for assessing the acceptability of a model

under such circumstances:

p M. mo —

- 1 0

where M. X.2

for an i-factor model

v. L

The rationale for this statistic lies in the fact that the ratio

of chi-squared to degrees of freedom is an estimate of the residual

variance under the null hypothesis, hence 15 is an estimate of the

proportion of, residual variance under a zero-common-factor model

accounted for by the k-factor model. Table 15 gives values of

for each of the ten analyses, together with values of chi-squared

for the zero-factor model. Note that with the exception of the FS

group (which was the only group for which an 8-factor unrestricted

model was acceptable, and whose sample size was close to the

number of parameters to be estimated) all values of "p are very

Page 125: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

125

Zero factor model 16 factor model x2

P MA 22541.3 0.9838

MS 28457.25 0.9832

MT 50330.92 0.9839

FA 36671.72 0.9836

FS 18846.62 0.9999762

FT 54622.68 0.9848

AT 58860.1 0.9824

ST 46692.0 0.9854

WG 103228.02 0.9854

TG 105097.7 0.9840

(528 degrees of freedom) (328 degrees of freedom)

Table 15: Tucker and Lewis "reliability" coefficients for the restricted analyses

Page 126: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

126

close to each other, and all values are high, suggesting an accept-

able fit; there is of course no guarantee that only the particular

model tested will yield such favourable results, but since the only

a priori model available - the second-order factor model - had

already been tried and proved abortive, there seems little point in

proceeding to a posteriori model-building at this stage.

Table 16 consists of the residual matrix for the WG run, as

an example of the size of the residual covariances remaining after

the fitting of Cattell's model; it is in all respects representative of

the other residual matrices. Note that residual covariances between

corresponding scales in the two forms are effectively zero, that

most residuals are well below 0.1 (absolute values), and that 16 of

the 22 residuals above 0.1 are associated with factors L, M and N,

which had poor internal consistency as shown in Chapter 6.

Such a residual matrix is, in effect, an indication of the

correlated error between scales under the model fitted. Since

the factor covariance matrices were left free in the specification

of the model, the fact that certain factors were estimated as being

very highly correlated needs to be borne in mind, but with this

proviso it is clear that by and large Cattell's claim that the error

variance of each scale is made up of many small components rather

than a few large components is justified, or at least that such error

components as are present are in common to both scales in a pair

to reasonably close limits.

Page 127: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

12.7

CA CO CA A- A- A, A, Ar A- Lil AD AO A, CD r, CO N CA A- CO AO Co CA N) uo so ro C'- co in 0, CO

a — — Al Al r, ro -- — -- — sr N) DA +- Cl CO AI AA A- r, CO 111 CA AS Al CO a CO CA N) ,- CO

cn CO CO CO CO CO CO CO rn CO — — CO CO rn CO CO rn CO m rn CO CO — CO rn CO rn CO CO CO CO CO • • . • • . • • • • . • • . • • • ••••••• ••••••••

CO M CO CO CO CO CO CO CO CO CO CO CO AA CO CO CO AD AA 'CO AA AA CO CO AA CO CO AA AA AA CO AA 1 I I 1 1 I I 1 1 I

CO Cl CO Cl ro ro 4 r- 1,0 '-Cl sr co (...4 r, CO N -- Cl 1.0 t..-4 a 03 so ro CO CO AO A- "0 A- CO CO

Al CA A- Ul CO A- AO 1M MO AO CO a — sr ro CO CA a ro CO rl ro — CO a- Cl ro co ro N) CA AA A-

CV CO CO CO AA CO CO CO CO CO CO CO CA 'CO AA CO CO CO CO CO CO CO CO CO CO AA CO rn CO *4 CO CO CO

••••••••• •••••• • • •••••••••• •••••

CO CO CO Cl CO CO CO CO AA CO CO CO AA CO CO CO CO AA CO AA CO CO CO AA AA CO CO CO CO CO AA CO

1 1 1 I I 1 1 1 1 1 1

AO CO CD Cl N) a '0 CO CO CO WO -O CO CO A, CO A, Ar CA DA CA A- Cl CO Cl N. to N. cr. 'CO so cr.

CA NA CO U7 Ul a CO CO ro N. -- so N) AA CO AO A- Al AA 03 1M CA CO 41 CO AA A- CO sr — CO N im

CA AA CO CO CO CO CO CO CO AA CO AA AA CO AA CO CO CO CO CO AA CO AA CO CO CO AA CO AA CO AA AA CO

. • . . . • . . • . • • • . • . • • • • • • • . • • • . • • • • 111

CO CO CO CO AA CO CO to CO AA CO CO CO CO CO CO CO CO CO to CO AA CA CO CO CO CO CO CO AA AA CO .A1 1 I I I I I I I I I I I I U)

DA sr AO AA CO sr cr.. r, — ,-- CA CO AA CO CA A, CO In vo IN os o- co r, AA CO A, -CI CA sr — DA AA

A- Ill ro so — DO A- AD A- AO A- a a m CA Ar DA PO Al CO AO C..1 F.1 CO A- CO 47, Cl CA sr -- sr ro m

CI CO CO CO CO AA CO CO AA CO CO CO CO CO CO CO CO CO AA AA CO AA AA CO A- CO AA AA CO A- CO CO CO 5.: • . • ••••••••• • ••• • ••••• • ••• •••• • • ill

M CO CO CO CO CO CO CO CO AA AA AA CO CO AA CO AA AA to CO AA CO CO CO CO AA CO CO CO CO CO AA 1 I I I I I I I I I I I I I -r,

at

1...N r.r.I AI DA r, .-- ro CO AV A, A- A- CO AO CO sO OA CA A- 0, AA CA DA -0 A, CO DA A- ro to IN CA 4,-.'

..... Cl CO , - N Al er CA ,-- •-- •-• CO Ar Al .-- PO CA N) CO -,- Al pA C4 r0 LAI ..--. A„ CO C-.1 CA .1- .1- LI

CD CA CA CO CO CO CO CO CO CO A- A- CO CO CO AA CO AA CO CO AA CO AA AA CO AA CO AA AA CO CO CO AA AA • • . •••••••• • • •• • • ••• •• • ••••••••• ....■

MI CO CO CO M rn CO CO CO CO CO (CO CO CO Cl CO CO CO CO CO CO M CO to M CO 'CO CO CO CO AA AA .1A

1 1 I I I I I I I I I I I I IP N

ro -O AO CA AA A- A, CO AO A- CO A- CA DA sr so a ro CO CO CA A, AO a CO sr CO cis sr a r-t sr r•

AACA DA ro CO Al A- Ar AA A- AA A- sr '-0 sr sr r, -- rn — a rn Cr. Zr to- ro CO — — sr CO r,

2.: m rn -- rn m m -- m rn *4 m — *4 rn rn .--. rn rn ..- rn CA S4 M AA tn n M ,-- M CA ..,-. n I:2 • • • • • • • • • • • • • • • • • • • • . • • • 0 N• II • • • • 7:i

CO AA CO CO AA CO AA AA AA AA CO AA CO AA CO CA CO AA AA AA AA CO CO CO AA CO CO CO AA CO CA AA Al

1 1 1 1 I 1 1 I 1 I I I I r-- Cr,

M M 141 N. CO MO CO N 1...... M ...... N. A- OD AO LA CO CA OA sr CA DA OA N ro (5) sr — CO -- ro co I

so ro ,-- DA Al CA (-4 sr PI AA A- A- A- A- AA A- CA AA r, ,-- rn r-. CA r4 rn CA 1-, DA Cr. -0 a -0 ,-.. • AA CO A- AA CO m CO CO CO CO CO ' m rn CO — — rn CO m CO CO *4 CO -- CO CO *4 (5) m CO rn .,-4

. . • •••• •••••••• • • ••••••••••••• ■ • .0

M CO CO CO CO AA AA CO AA S4 CO (Si CO CO M CO CO SSA CO M M n CO CO to SSA AA CO rn CO CO CO .1.....

1 1 1 I I 1 I I I 1 t 1 1 I I • Al

N) Al- Al CO CO N) so -0 rn r, --Cl a CO AA A- AA CA Cl LA A, AO Os "0 CO N) AA A- N) r, so A-AA CO CO AA AA CO AA CO CO CO AA CO AA AA AA CO • • • • • • •••••• I•• •

CO COCA M CO t5A MIMnnn SA to CO CO CA 1 1 I I I I I I I

N. ..0 CO r, -- r, — os tn uo rn CO DA CO A- ro so CA A- rn cr. un CA CA AA CO -0 DA AA DA Al un -,71 AA CA AA CO AA AA AA CA AA CA AA CO CO m rn (5) cu • • • • . • • • • • . • • ■■ •• F-4

CO CO CO M M M AA CO CA AA CO rn CO CO CO CO 0 I I I I I 0

Al < rn Cr so CO AO AA CO AA AO CA AA CO A, AA A- a a OD VI CA A- A- A- AA A- ,0 sr a co a cl sr-

CA A4 AA CA DA CO CA AA AO sr sr rl — ro ro — -a ro to n sr CA AA AA A- A- a to -- -- ro uo r... £ 1.4 M CO CO AA CO AA CO CO AA CO AA AA (Si <A AA AA AA CO CA CO CO AA AA CO (Si AA AA CO A- CO AA CO CI A. . • • • • • . • • . • . • . • • .• • • . • • • . . . • • . . • 5_ O CO CO CO rn rn CO CO *4 rn CO *4 CO rn CO AA AA CAA KA AA (5) CO AA CO AA AA CO CO AA AA AA AA AA A- LL I 1 1 1 I I I ! I I I ! I I I I I I

IP

to sr CO -- CO In CO CO AO CO N. ro CI., so a r, DO '0 DO DA N. A- AA A- CA AI Ar CA Cr, CA A- A, 0)

C4 A- A- sr CA Al AA AA CA C.4 ,- sr ro tn vo — r4 -- AD A- AA PA AA CI Ar A- Ul ro m — cut m 1..1

:a rn rn CO AA AA AA AA CO CO AA A- CA CO CA CO '5) AA AA CO AA CO to CO CO CO AA CO CO CO AA AA A- !-- . • • • • . • . • • • • . • • • • • • . . . • • • • n • It • • • .15

CO M CO CO (Si M CO AA CO AS CO CO CO CO CO CO CO AA CO AA CO AA AA CO AA CO CO 'Si CO AA CO CO .A1 ! ! I I I I ! I I I ! ! I I I I r.-

m

uo uo r,.. CA AO AA An AA A? PA A- A- AS sr ro r, r, LC) ro 17-4 so m to sr ro c4 -- rs. sr IN cl R. :..-

sr -- DA CA AO CO VI CO AO CA AA N) A- AA N) ro so (Si CO AO A- CA AA CO sO a — -- -- CO rn uo 0

CO CO CO AA AA CO AA AA AA AA AA CO CO CO 'Si CO CO AA CO AA (Si AA 'Si CO AA CO AA CO CO AA AA AA CO L1 • • • . • • ••••••••• • • ••• •• ■ •••• •• •••

AA CO AA AA AA AA AA AA CO AA AA CO AA CO CO CO AA AA CO AA AA AA CO CI AA KA CO CO CO 'Si 'CO CO 1-.1 I i 1 I I i I i 1 i I 1 I I I III

1

CI, rn CO ,... CO ,0 CO AD CO CO CO N. CO ro ro -- -4- sr 01 — 'Si los CO CO AA PO AD (Si AA OD DA e- AD

CA AA DA Al AA ro CA DA N AA CO CA DA Ar A- PA AA CA AD 00 AA A- AA Al A- AA A, AA P71 IN r.-.1 0, .,-4

LL CO CO AA AA CO CO AA CO CO AA AA CO CO CO CO AA CO CO CO CO CO CO 'CO CO A- CO AA AA SiA (5) 'Si AA A i . • • • • • ••• •• • • • • • •• II • • • • • • • • • • II a ID

M M CO AA CO CO AA AA AA AA AA AA AA CO AA CO CO AA CO AA AA CO AA CO AA AA AA CO CO 'CO 'CO AA CA: 1 1 I IIII I I I I I !

AS AA CA A- e- CA A- OD CO A, CA DA Ul Al CA A- AO -0 A, e- ,0 r.., CO MI CO to C..4 T... 'T. n a n ct

N? CA A- AA Al CA AS CA IM Ul MI A- e- DA CO Al 1...... -- ro CO uo 4 c.1 CO '-0 — CO -- so if) sr n r......

LU CO CA CO CO CO AA CO AA AA AA CO CO 'Si CO (5) CO CO CO AA CA CO AA CO AA AA MI CO CO CO CI CA A- • • • • . • • •••••••• • • • ••••• •••••••••1

CA AA CO CO '$1 AA CA CO CO AA AA AA CO CO MI AA CI (Si CO AA AA AA to AA AA to CO CA CO CO CO 'Si -1-) I I I I I I I I I I I I I r.-

ro

N A- ..-- OA AA Is, n so ro 1.,7 so ro so co tst CR cr- co — ro co CA CA to AD -.0 sr co n ,0 A- DA Q.

CA sr rn — hl Ul A- AA CA A- 1.)1 CA sO DA LA Al CA AA CA AO DA DA OD A- CA AA AO A- AA Al rl a

A..) AA AA AA CO AA AA CA CO CO A- e- CO CO CA m m CO 'CO CO AA AA AA AA CO CO CO *4 *4 CO CO 'CO M .40 • • • • ••••• •••.• • • • • • • • • ••••• 1.114 • • .......

M CO M CO CO CA AA CO (Si CO CO AA CO CO CA CO M MI CA CO AA AA CO AA AA CO An CO AA CO CO M I I 1 I I I I I I I I I I el

1.4

sr AA A- VA AO Ul AS CA AS CO -Cl PI sr Co c-.4 CO a '— 'Si Cr- OD AA sr — uo a (Si cr. N) 4 — CA iA

A- (5) a rn rn — ,- CO Cl 1A) CA IN PO AA e- A- A- CO er CO AA CA (N CA A) CO CA A- ro n CO — M

PA CA CO AA AA AA CO CA CI 15 CO CO CO AA AA CA CA AA AA CO CO AA CO CA CO CO 'CO CO CA AA (Si AA AA A- • • • • • • • . • • . • . • . . . • • . • . • • • • • • . • n •

Ilii CO AA CO CO CO AP CA AA CO 'CO CA CO AA AA AA CA CO AA n M CO CO CO CO AA CO AA AA CO AA AA I I I 11 1 1 1 1 IIIIII

CO 4 IN a Os 1M Al CO Al Al AA N DA sO CA los CO ro a N n N DA AO -0 n co a cr- co tr, a

CO A- N AI (-1 a rq CO ro -0 *4 — uo ro CA A- AA CA CA CO CA a ,--- sr so co a AA sr CO uo -- ‹ CO CO CO CO AA AA CO MA CO AA AA AA CA AA AA CO AA AA CA CO CA CO CO CO to M4 CO AA CO CO CO M

1 • 4 • ••••••••• ••• • ••••• 0•••11 •• ••11

CO CA CO CA CA CO CO CA CO 15) CO CA CO MA CO CA CA CO CO CO CO CO MA CO CO CO CA MA MA CA CA CO I I I I I I I I I I I I I

AL 'Si

A- VA AA er A- cl AA er AL OR L) LU LL CO 7C 1.4 -J AC 2t 1=t 1=3 CA CA CA C'L OR LI LW LL. CO MC H -J 3C .11C CD rI cn CV Cb

O 0

Page 128: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

128

a r4 1.19 Co ,- 0" r, a ro co -a cq in or- co rn co Co CO cq No 4- CO r, CO sO sO if ,- MI 0, Co .gr ,- -- Co4- cr un Co uo uo ..0 r, a to If) '- Co 4- -- Co Co Co uo Co Co -- N. N) -- r, Co ro Co

CO Co Co CA ,- Co Co ,- Co Co CA Co Co Co Co Co CA Co Co Co CA ,- CA Co CA CA CA '- CA Co Co Co CA • ••••••••••••••• •••••••••0 ••••••

M Co Co CA M Co Co CA CA CA Co Co Co Co Co CA Co Co Co Co MA Co Co Co Co MI Co Co Co Co Co Co

I I / I I I I I I / I I / I I

Ill ..-- ..--. a Lin r4 ,- CA ,- ro CA CA ,- sO CA 0, PO un 4- a 0, 0, ,- Co -CI O. rl (N r, Co Co 0,

M uo en rA a ro Co r4 ro (....1 a n a a CJ to .,-- 4- ro sO OD CA MI ,0 ,-- r, ,- cl -- ro ,- in ro

Ca Co Co Co Co Co Co M Co Co Co 1.- Co Co M Co Co M Co Co Co Co Co Co Co Co M M Co Co CA (Si Co • • • • . . • • • . • . . • . • 11•••••••••••••••

Co M n CA CA CA MA CA CA CA Co Si Co Co Co Co Co Co M Co t CA Co Co Co Co Co CA Co Co 'Si Co

I I 1111 1 1 1 1 1111 I 1 1 I I 1

co a ,0 In CO rq c.4 a co --- a MI a n -- wo ro ,- cq -o Co rq -- co r, c.4 cr 0, rl Co CA MI

CA Co M rl 149 CA Co ,- ...- U9 .,0 a rl -- Co r.4 N) r4 n a Co a Co ,-. CA Co PO 1.19 ,- ,-. CA =-. Co

Cl si Co Co Co Co 'Si Co n Co Co Co Co Co Co Co Co Co Co 'Si .- Co 'Si Co M ,-. M Co Co MA Co Co MA

e • a ft v. • • I so • • • • • . • • • • • • e . • • . . • • ■ • in

Co Co M Co Co CA Co CA Co Co CA Co CA CA CA CA M Co Co Co Co CA CZ Co CA Co Co Co Co CA M CA •1-1

1 1 I 1 I I I 1 1 1 I I 1 I IP >.

0, N, 1'0 .-4- PC1 a 0, 00 Lil CA 4-4- PO CA 0, ...0 CO 129 ,- CO 03 In PO LICI 0-. ..q. CO bc4 PO CA CA r, -- -1

...- 4- ro n sO MI ,- CA ,- CA 0, ,-. rq a -- ro rA 1..... r4 1.19 ,- uo ro 4- c.4 ro No -- a- m -- ro r, RI

C2ScA (Si Co Co M Co Co •-••• CoESI MISSI •"•• Co Co Co Co Co tsinnmslmits! Co MI Co (54 (Si (St (5) r

••••••1••••••••• •••••••••••••••• III

M Co Co Co Co Co CA CA Co CA 'Si CA CA Co Co CA Co Co Co Co CA Co Co CA Co CA CA CA CA Co Co Co I I I I I I I I I I I I I 1 I I I "0

a)

a o- co -- Cu r, cr a co -- cr -- No r, -- lo- cl -- uo co -- uo sO ID ,0 CA rn -- ro LT- cl uo -1-,,

CA -- -- -- n -- to ro b9 Li) ,-. CA CA '1' MI CA ,-. ..-- ,- ,-. '1' P9 Cr, =- ,- bc, 0, CA 41- ,- ...... ,-- LI

C) Co 'Si CA Co Co Co CA Si CA CA ..-- Co Co Co Co Si Co CA Co Co Co Co Co Co Co Co M Co Co Co M CA •r-1

•••••••••••••••• •••••••••••••• •• l:•-•

Co CA CA Co Co CA CA CA CA CA CA CA CA CA Co CA Co CA CA CA M Co Co Co M M CA CoCo Co Co (5) C:1 4-'

IIII 1 1 I 1 I I ! 1 1 I 1 1 0 a,

CO CA 4- c.4 "0 ,- 4-4-' a n a- Co uo r, ro -0 N9 CA ,- CA r, PO ,0 U9 rA '4' rl Co MI uo 4- rA ,0 5- •a CA SO CA r, ,- Ul a No N. ts/ r, c..4 Co co 4- Co Co to PO ,- r4 r4 CA 'Si CA CA CP, ,- U9 r4 N)

Z Co Co CA Co Co CA Co Co CA Co CA Co Co Co Co CA Co Co ,- CA CA CA ,- Co Co CA CA CA CA CA CA ,-. CL!. • . • • • . • . . . • . • • • • • . • • • • • •••••••••• ..71

M M (5) CA CA CA CA Co CA CA CA CA CA CA CA CA CA Co CA CA CA CA CA CA CA Co Co CA CA CA Co Co CI I I I f I 1 I 1 I IIIIII 1 1

Cr, Cr,

41.1 4- No m ro CA h0 -40 In CA 'q- CA CO 1'... CA -o a r, co IN CO •-0 rA sO 149 CA CA rl OD rl CY, ,C)

CO CA CA =- CA a- -- -- CO Co VI ,- 0, ,- PO Co CA P9 '43 149 CA b9 Co ,- 19 CA Co b9 -0 V) -- r, .- • Co m Co CA Co CA CA CA CA Co CA CA CA CA Co Co CA CA CA SiC CA CA Co CA CA CA CA CA Cy Co m (5) ..-1

• ••••••••••• • ••• •••••••••••••••• !::

Co CSI Co Co M Co M Co M Co CA CA Co (Si CA Si CA CA CA Co CA 'Si CA CA CA CA CA Si CA Co Co MI 1-.-, I I 1 I I I I I ! I I • -1

-4:

401( bl sO CO VI MO CA ,- CA MO CO r, PO rl CO U9 MO ',CI cl- CA MI CO =... b9 CA b9 .1" .,0 .1- tN -cl co

sO PO rl sO ...... sO a -- m n or un n (...4 CA r4 U9 CA CA M4 0, CA CA 1" CA bl Co ,-. N9 CA r, ,- 7,

..J Co Co CA Co ,- CA CA Co (Si ....- CA Co CA Co CA CA Co Co Co CA CA CA ,- Co Co CA Co (Si CA ,- Si CA III • • li• •• • ••• 110 •••• 111111111 • 10%11 1111• • ion.. rl

Co Co Co Co Co Co rA Co Co Co Co CA Co Co Co M Co Co Co CA 'Si CA Co CA Co CA Co CA CA CA CA CA CI 1 1 11111 I ! I I I 0

PG In =- CA r0 CO *1- ...- CA 0, f`, a No 1.**N CO 1'0 I'l rl T.. 4- uo cr a t., -- uo -0 IN b9 0, CO CA r,

Tr Co ,- (Si MI CA Co CA CA r4 1- MI ,- m a a .-- -- a to vo co 45) n a --- cl -- r..4 cl -- in 1-.. E I-I Co CA CA C:4 Co Co CA Co Cr CA CA CA ,-. Co CA Co Co Co Co Co 151 M M4 M4 Si '5) MI Co M MI (Si Co C)

5-• . • • • • • • • • . . • . . • . • • . • • • • • . • • • • • . r O m Cl Co m Co Co m Co Co CA MA Co (Si CA 'Si CA Co Co Co Co rA Co Co Co Co CA CA Co CA CI Co CA .'1-

141.. I 1 I 1 1 1 1 I 1 ! I u)

L) '1" r4 CO SA MI Co ,- =-. Cr- ,0 U9 CO r4 -0 0, 0, Co ro No 0- cl -- r, -- c...1 uo -0 uo --- -- co al

-- CA 03 CA MI CA CA CA CA CA 0, cl m uo n n -- 1:-..4 a -4 -- cq ts/ n m 45/ c.., o- a -- -47 CA LI

=C MA CA CA SiC Co MA CA CA Co CA CA CA 'Si (Si CA ....- CA Co Co Co CA CA CA CA ..-- Co ..-- CA Co Co t5. CA • • • • • • • • . • . • • . • • • • •••••••• ■ ••••• 17.1

Co (Si Co Co Co M Co Co Co (5) CA Co CA Co 'Si MA (Si Co (Si m mi MA CA Co Co CA CA 'Si M4 M to MI ,-.! I I 1 1 I 1 1 I I 1:-

ITI

cl ro rA N. 0- 'M T-. T.. N ID N. 0... 01' .1-- OD CO CO MI (Si r, =- Co EN 1 CO ,0 -(1 In MI CA Cc .1- >

'Cc r4 bl a -- n vo CA U9 R. CA PI MI CA ,-. U9 ---- =-. ,-- PO r., Co cl CO CA U9 IN to ro in P-.1 In C)

CO Co Co Co m Co m m Co Co n Co CA Co. (Si Co CSI CA Co vA (Ci Co U,1 CA CA Co CA Co cA Co rA rA CA U • • • • • • • . • • • • • • • • • • • • • • ••••••••••

M rA CA CA CA CA CA Co CA Co CA CA Co Co Co CA Co Co CA Co 'Si (Si CA CA CZ CA CA CA CA 'Si 'Si CA ...A

I 1 I I 1 I 1 1 1 1 1 1 1 I I) :1

CA CO CO ,0 CA ,0 IN 4..... , ''... CA CA Co Cc Cc .1- sD ,-. =- U9 =.- CA ,- 0, 0, ro co ro -- !Jo n 0, -o -r,

CA Co In In Co ,- 'Si -4- 0, CA ..,,M PO CA rl PO CO r4 Co a -0 in N. -- PO 0, CA =- -4- In '4- Co CSI .-.1

LL Co Co m M (Si M Co Co M Co M Co M CA (A CA Co (Si (Si CA 'Si CA CA Co Co Co CA M4 (Si Co M4 .,..- 1.11

. • . • • • a • • • • • • • • • • • •••••••••••••• Ill

MM MI Co Co Co CI (54 (Si Co Co C., Co Co t.Si (51 MI (54 to (Si Co Co (Si (Si (Si (Si (Si CA 'Si Co ,SI 1:51 CC:: 1 1 1 11111 I 1 1 1 1 1 ! 1

a a

r..1 0" In ...-.. T.. Cl Ul Cl- N .1.4. CO 0.. Cl U9 rA CO Si CA ,0 =-. T... N. '0 uo cl CA r, CO CO ,0 '1" CA 0

CO CA sr) CA OD PI ,-. CA CP ,-. ,- =- "0 U9 ri r, NI CA PO CA ,0 N9 4- ro n bI NI ,- ,- CA CO CA I: W CZ CA Co CA CA CA Co CA 'Si m 45-4 Co ts! Co 'Si Co 'Si Co Co Co Co CA in Co 'Si Co 43/ Co Co -- Co m -1-)

. ••••••••••••••11 *•••••••• • • • • N114

(Si CA Co CA CA CA CZ CA (Si CA CA rA CA CZ CA CA Co CA CA CA CA 'Si 'Si CA Co Co CA 'Si 'Si Co Co CA I I I I I 1 1 1 1 1 1 1 1 1 r....

IV

1' MA =- I-, CA ro In In CO 0, CO =-. MI CA In ..-- IN N. .,..- -o uo in ro 1- .4- co m ul co CA a co a

rl 4. in ro No co ,o ro ,.- r, to m m co m -- ,- c...1 ts., to a -- a a r4 SD PO *-- b71 '4" "0 CA

(-1 CA (Si M t54 Co M (Si Co Co Co ..-- Co (Si 'Si Co Co Co (Si CA 'Si Co CA CA M STA Si .1.... M4 STA r5:. it1:4 Co ...0 • • . • • • • • 1.11•••••• •••••••••1111 • • • a •

Co (Si CA Co M Co rA CA CA Co M M MI M M4 Co M) Co Si Co rA ISA Co Si Si Co CA M CA CA CA Co

1 1 I 1 - I I 1 ! I I i 1 1 1 I I C.I

MI ,-. CO No a uo -47 CO ,0 Cr. NI CA if) a CA rl '4' SA r, rl ,- MI CA ,- No r, -- -- -- -- uo 'Si

CA Co cq ..-- CJ 'Si ,-- N1 CA CA ..-- ro r4 CA ro n ro Co cl in m -- cl -- Co ro Co ..-- r4 (Si No -- P4 Co CA (Si CA 'Si Co M CI Co n Co (Si CA CA 'Si 'Si CA M4 (Si Co M MI In Si M Si 'Si to '51 m CD CA

• %IV •••• •1111 /1111%1 •••••• ■ ••••••• • .

Co CA Co Co CA Co Co Co CA Co Co M M Co CA Co CA 'Si CA CA Co CA (5) CA CA 'Si '5) CA CA (Si Co Co 1 1 1 1 1 1 1 1 1 1

CA 4- 0- -0 a r, uo 4- r, 07 1* 0, CO R. .- CA CA 4- cq Co ..-- co ON rq V) t (N rq 1.0 1'0 N) CO

CA .- Co IN h-4 -0 CA '1- -0 CA IN Co to (-1 4- ro n ro -- ro rq --- -- -- L') C4 Co ".•.. IN rq 4- a

,X CA CA (Si Co CA Co Co Co Co =.- CA MA Co M M M Co MI M Co (Si Co CA Co Co Co Co CA CA CA CA 'Si

. • • • • • • • . • 11••••• ••••••••••••••• •

CA Co Co CA Co Co CA CA 5) CA CA CA 64 CA MA (5) (Si Co Co Co rs4 CA CA CA MI CM (Si Co rA MA (Si CA I I I mi I I I I I I i I I 1 I

P4

4'4 M .cr r4 Mi .=1-

C <Ci7O C.) W LI. =C ...I 1= Z cn cn CL CI CJ C Q OP G.) 14./ 4. CO = 0-1 -J 3= 2C CD cn Cl Cl CN 0 0

••••4

lu 1-

Page 129: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

129

Table 17 gives the unique factor loadings for each group:

note that there is a tendency for differences in these values between

paired scales to reflect differences in homogeneity coefficients

(Table 6), the more homogeneous of a pair tending to have a lower

unique factor loading, the tendency being more marked in the case

of larger differences (e. g. factors B and M).

The results obtained from SIFASP are generally consistent

both with analyses discussed in previous chapters and with certain

of Cattell's own published findings. For instance, Cattell, Eber and

Delhees (1973) reported a factor analysis of scale scores from four

forms of the 16PF, on a sample of about 600. This analysis showed

rather poor identification of factors E,M,N and 0. It has already

been remarked that factors E and N were difficult if not impossible

to separate by rotation of unrestricted solutions; further in the

restricted analyses the factor corresponding to N was not linearly

independent of the others, and elements in the residual matrices

tended to be larger for N than for other scales. In both forms,

alpha coefficients for these scales are very low. Factor M also

proved troublesome in rotation, perhaps partly due to the very low

homogeneity of this scale in form A, and also has some of the larger

residual covariances. The problem of separating C, O, Q4 and L,

the major components of the second-order Anxiety factor, has been

constant, and reappears in the restricted analyses; the problem

with L has no parallel in Cattell's own work, but Cattell, Eber and

Page 130: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

130

di

0

N N 0 CO 7t1 C7, N • N ON M N M CO M N • .0 N L l Lf) Ul t*, N CO CO N N ,S) LC) • . .

0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0

O 1--I O CO t-r) CO N N N CO M N M CO ce) N

Lc) N Ul tn in 711 N. 00 co .0 N N Lc-)

• . . . 0 0 0 0 0 0 0 0 0 0 0 • 0 0 0 0 0

▪ N re) ON N M L.C1 N .14 N ON ON ,C) N Ui in in .70 t.r) N CO CO .0 N N Lt1

. . . . 0 C: 0 0 0 0 0 0 0 0 0 0 0 0 0

rn 0 ON LC) r•■4 .0 0 O N .4D co -1 en Ul s..0 N is) Ul No N co as co .0 N u-)

oo o o o o o o o o o o o o o o

CO Ul c.) as as as as o o cn CO N N E-1 N Ul .14 in cel N N ON CO s.0 N N S.0 Ul

CLI•

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 4-1

E-4 N N N .0 co N 7t4 o co N in cn .0 N d4 cn „.„ .0 .0 No in in in d4 .0 r- co co .0 N N .0 in

0 0 0 0 O 0 0 0 0 0 0 • 0 0 0 0 0 Ti rzt 0 ,-,

N 0 N .0 r- 0 0, .0 cn c0 N c0 (/) .0 --I r-- In .7f1 .0 cil In N 00 ON .0 r-- N .0 in 0

O o

o o o o o o o o o o

o c o o u ai

't CO ON r'r) 7t1 C' in CO 0 0 •14 en 0 Ul N a) ▪ in .0 .0 in .0 cc*, N CO N CO .0 In

▪ 0 0 0 O. 0 0 0 0 0 0 0 0 0 0 0 0

co co ,c) o co con co co In N co ,t o o N '4) u> s0 Ul .7r di N co co .0 co co r Ul -

O o o o

o o o oo o o o o o o o

ON co .0 c0 co 0 0 CO 714 N r- s0 in in

N CO CO C7, .sp Ul N Ul in

0 0 0 0 0 0 • 0 O 0 0 O 0 0 0 0 0

N N m71,

7 L.1 z o aaaa U

he t

en r

est

ric

ted

facto

r

CO

2

Page 131: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

NI M .c) cy, rs ce) O .0 Co -.0 1.11 N • • •

coo° cpc0 O

N CO C/) .0 ON 44 • • 0 0

N N in in N CO •zt, N Cs N to .0 in .0 co .0 CO in .0 .0 in in

. . . . . • O o

o o

o o

oo O o o oo o T

able

17 p

art

2:

131

N In

• O

N .0

• c0

0 co (0)

CO .0

coo

0 co

•11 in r-

c0

in .0

• O

co in

• o

;.4 NI in

ti .0

0 co

.0

.0 •

Os N

• Ln in

•41 N

-I .0

in .0

co in

0 4•J t.) 0 d o o o o o o 0 o yea

'0

in .0 co cOOM.00.000.0N.0

.0 N in N .0 on .0

ON in •i■,

U • . ..-1

0000000000 000000 4;-1.. cn N

,--4 .14 c. In N en .0 N N .0 F-I in .0

o N

• o .0

• cD co

• o in

o N o

in o

.0 o

in o cu -,-

O

ON •14 M ON 0 N O 7r4 co di cr• 7r1 r• in CO -10 in N Ln -0 co r- N in Ln Ln

. . • O o o o o o o o o o odo ocio

• 7,4 o •44 -0 0 •11

7t4 "44 NO cr. .0 co N it) .0 co .0 co in N .0

•.0 Ln

. • • oc o o o

ON N st) CO N as 714 o 0-- in co N • to co .0 .0 in N W in CC N co in .0 in .0 in

(

Soo° coo o

do do o 000

N CO CO ce) 111 .0 0 CO ON CO .0 0 .0 •--1 .0

cf. N In N in .0 co .0 N in N N .0

O do O o o O o o

oo o o O o

o p., 2 C\31---Crstf)N.0 ceIMONNNN.ONNN

• N N .0 .0 to N in -.0 N to CO in N in .0 in 0 . . .

O o o o o o o o o o o o d o o o

(1) N re) ,7t4 al° wrilox.4 zociacta

CC

facto

r lo

ath

Page 132: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

132

Delhees had very similar difficulties with C, 0 and Q4. They

rotated their axes in such a way that corresponding factors were

linearly independent but had low loadings on the appropriate scales.

Overall, the results do not unequivocally either confirm or

disconfirm Cattell's model, rather they lend partial support. In

Cattell's favour we have the following points:

1. The majority of the analyses support about 13 common

factors .

2. Many of these factors can be rotated, in the unrestricted

analyses, to positions where they load principally

nominally equivalent scales.

3. The restricted analyses give a tolerably good fit to

Cattell's model, although only, in effect, for 13 factors.

4. The residual variances in the restricted analyses tend to

reflect differences in the reliability of scales, estimated

either by alpha or by alternate form coefficients.

5. It is at least conceivable that the closeness of the C, 0 and

Q4 factors is sample specific (and the same argument

might also be used for L and N), either because the items

in these scales do not efficiently discriminate between

the traits with British undergraduates, or because in the

present sample responses reflect state rather than trait

aspects of anxiety, which may be less differentiated, as

argued by Cattell, Eber and Delhees (1973).

Page 133: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

133

Against Cattell we have these considerations:

1. There is a consistent finding that C, 0 and Q4 are virtually

inseparable, both in this study and in Eysenck's (1972)

reanalysis of some of Cattell's data, not to mention the

Cattell, Eber and Delhees study.

2. Factor N is so poorly represented, and has such low

internal consistency, as to be worthless.

3. The restricted analyses cannot provide a guarantee that

the model as fitted is the best or most parsimonious

possible.

4. Rotated solutions invariably collapsed when attempts were

made to force them to the complete Cattell model.

Page 134: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

134

CHAPTER 9

The results of the restricted analyses in the last chapter are

consistent both with the homogeneity analysis of Chapter 6 and with

the results of the Cattell, Eber and Delhees study. However, in

one respect they differ from many of Cattell's published results,

namely in the factor correlation matrices. There is a strong

tendency for the correlations between factors to be rather larger

than in Cattell's own studies, a feature which has been noted in

particular in the cases of factors C, 0 and Q4 but which is wide-

spread throughout the matrices. To take but a single instance, the

correlation between A and H in the restricted analysis for the male

total group was 0.556, whereas the corresponding figure in Table

10.2 of Cattell, Eber and Tatsuoka (1970) is only 0.26. Now one

does not expect to find identical factor correlation matrices in

separate studies, but a further feature is present in the results of

the restricted analyses which is novel, or at least previously

unnoticed, namely that if the rows and columns of the factor

correlation matrices are suitably reordered, a pattern appears.

Table 18 gives one possible reordering of the matrix for the total

group, but essentially the same pattern is to be found in all ten

analyses. By and large the size of a coefficient is inversely

related to its distance from the principal diagonal. Were this

pattern to hold throughout we should have here an example of what

Page 135: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

FOR THE TOTAL GROUP

CO

CO

0-4

CO CC LU

LU CC

CI cr

CJ cn • CD

I-

=1

1 g: REORDERED FACTOR CORRELATION MATRIX,

135

r. ro PI *- MA ON Oh Oh VO Oh M •■■ ,■ .0 Q CO cr ro m m Os CO CA *- PO in M Ps .0 Co MA

2 MA ,- cr ,- ,- cr un sO 1.11 CO ,- MA MA MA • • • • • • • • • • a • • •

14A MA MA MA CA MA MA CA MA MA Q MA MA MA CSi ,- 1 1 1 1 I I 1

Ps MA ...- CO 1.- of NI CO *- CV Q CA Ps N. Q CO ....1 CO CA CO .0 .0 ro -- CO cr ,to r4 *... ro

NI .Kr 1.0

UO UO .0 PI ..-- rA CO *- ON NO ,■ 04

• • • • •••••• It • • •

M M M MA MA MA MA MA MA MA Q MA MA MA MA 1 I 1111 1 1 I 1

cr to co cr NI .0 cr PO .0 CO cr m r. co cr r. ,o CO O. 0 cr co ,o un CO CO CO cr

CD *- CA 0, 0, Ul ISM NI CA CO CO PO • • • Is • • • • • •

MA MA MA MA MA MD MA CO CO MA MA Cal CO MA

C •4 CO Ps 0, NO CO Ul P0 cr un ro v m cr r. C4 0, ...- PI cr C4 ...- CO MA 0, CO MA PO

0114 MA Q MA MAMA MA MA **, MA ...... ... MA MA *- .. . • a • . . • • • • . •

MA Q MA MA MA MA MA MA MA MA MA Q *... MA MA 1 I I 1 I 1

ro M m cr Vra cr PO CA cr CO cr er PA *. CO CSy ro ct, m CO .7 CO *- N

h4 CA C4 C4 C4 MA PO PI MA rn r4 *- • • ••••••• ••••••

*A CA MA MA MA MA GA MA MA CA MA MA MA MA IA

11111 1 1 1 1 1

Ps *- cr CO CO CO CA .4" PO CO CO PO L7 CI. N1 Ps 471 .0 *- MA *A CP 1.0 CO N1

LI. .4" CO C4 PO *- N? sO CO **. r4 CA VD • • • • • It • • II • • • • • • •

Cal Cal 1,5;) Mi MA *A MA MA *A MA Cal *-4 IS/ rSi

1 1 I 1 1 I

CJ CO 1.0 - 0, N Ul Ps Ps CO CO r4 r4 O- 0- Li? *.zr LIMO, CO cr 4i -4c) Ul

UJ MA MA .0 Mk PO 1.0 MA sO PO PO .0 CO • • • • ••••••• • • • •

M CAM MA to MA MA MA MA MA MA CA MA MA MA

1 I 1 1 I

Ps CA LO .0 *- CO 0, CO MA Ps ON MI v "0 ■•■ NI -o CO MA MA P0 CA MA Os *- .0 CIJ CO 0, PI

Cr MA CO CA CO COCA cr ,o CO cr • • • • • • • • • • • • • *

MA MA MA MA MA MA MA MA MA MA MA Q MA MA MA 1 1111 I 1

N cr ro CO un CO co CO cr ro vo CO CI-

CJ PO -Cl od- r, LA C4 MA .0 0, cr *- AC C4 CO CO r4 NI CO .0 /JO ** CO .0

. • • • • • • • . • • • • • •

MA MA MA MA MA MA MA MA MA MA MA MA <A MA I I 1 1

N 0, cr CJ CO 0, CO PI If) cr ro 0, C4 ' N0 CO CO Ps PO U) r4 0, CO C4

CD *- CO *- CA N CO PO cr VO P0 CO CO CO *- Ul • • • •• • • •••• • •

MMMMM CO MA <A MA 4A MA MA MA MA

I III II 1

Cr` cr ,o cr CO -Cl CO Ps *- CO .0 LI 0,

PI CA VI CO SiM CO CO 00 Ps .D • CI. Zr C4 CO CN MA CO .0 .0 CO Ps r4 ** r4 CA MA 1.0 PO cr • • • . • • • • •• • II • •

M MA MA MA MA MA CSI MA MA MA MA MA 4A KA MA 1111

CA Ps CA cr m cr CA Cal 0, cr cr ,o ro m

cr 1.11 cr un cr CO co NI CO cr oN M CO 0, CN CA-- 0, SiM .0 CA CO CO CO CA CO CP -4) • • • • • • .1 • • • • • • • •

M 4.A MA MA MA MP MA MA MA MA MA MA MA MA

1 1 1 I

CA cr co m cr ,o cr ro ,o c. cr m -- CO -- co m cr m cr NO m -- ON ON 4... .0 PO MA

LA P0 cr m c, ,o MA MA *- C4MA 0, .0 *- • • • • • • . • . • • • • • •• •

MA MA MA MA MA MA MA MA MA MA MA MA MA MA 11A

1 I 1

,o cr CO co rq cr cr un L7 Ill CO Ps CO *- NI 0, .0 CO CO CO NI CO Ps NI P.0 ON N. .0 MA

mc cr m cr ro -- MA rA CA .0 Ps CO Ul CO cr • • • 0 • • • • • • • • • • • •

CO MA *- MA MA MA CO MA MA MA m MA MA MA MA MA

1 I 1 1

CV CO Sr cr N. -- Ps CA N 00 N NI 00 UO CO PO CA PO MA .0 .- cr In CA (N Ul Os MA .0 cr li.1 NI CD .0 MA in PO ....... CO ....- ....... CO un CO CO r4 CA

• • • • a • • • • • • a • • • MA ...* MA MA MA MA MA MA CLl MA IA MA MA MA MA MA

I I I I 1 1

MA CA .0 CO (N 0, cr cr r. cw r, In rA cr r, PS CO PI Os 00 In CA 6.0 Ps M ON U, CO CA CO UO a-

4 MA .0 ‘7" ...- MA MA *- CO MA cr CA CO ...- MA MA • • • • . • a • • • • • • • •

.... ID MA MD MA MR Mk MA Q MA MA MA MA MA MA CA

1 1 1

("4 cr ro 4 C3 2 CA CD CV CO AC CD LU li■ 04 CD

0.061 -0.086

0

Page 136: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

136

Guttman has termed a 'simplex', a pattern characteristic of

correlations between items arranged in order of difficulty, and tests

in order of complexity (Guttman 1966). The order of Table 18,

however, ignoring B and I which play no part in the pattern and

L, N and 0 which have been shown not to be linearly independent

factors, is not a perfect simplex, particularly at the lower left and

upper right corners of the matrix. Nor is the pattern a straight-

forward circumplex, i. e. with the coefficients first falling and then

rising with increasing distance from the diagonal. There is however

an approximate circumplex in the part of the matrix between H and F.

The novelty of this finding might lead one to suppose that the

pattern is an artefact either of the way the restricted model was

specified or indeed of the SIFASP program used. Inspection of the

correlations between scale scores reveals some aspects of the

pattern, but it is much more clearly apparent when the matrix is

corrected for attenuation. The bivariate correction of a correlation

for attenuation due to unreliability has been in use for many years,

and consists of simply dividing a correlation by the geometric mean

of the reliabilities of the variables. Bock and Petersen (1975) have

extended this correction to the multivariate case, showing that a

maximum likelihood estimate of the correlation matrix of true scores

may be obtained by first correcting the coefficients in the usual

bivariate way, and then reconstructing the matrix from the positive

Page 137: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

137

eigenvalues and associated eigenvectors of the matrix after

bivariate correction.

This procedure was applied to the correlations among scale

scores for the total group, using as estimates of reliability the alpha

coefficients discussed in Chapter 6. The resultant matrix is

presented in Table 19. Theoretically, this matrix ought to be

rather close to the factor inter correlation matrix, since both

procedures purport to remove error due to unreliability, and so it

proves to be. Moreover, it displays most of the features discussed

with regard to the factor analyses: note that L, M and N have much

the lowest corrected alternate form correlations - these were

troublesome in the rotation of unrestricted factor solutions, and had

the largest residuals in the restricted analyses; C, 0 and Q4 have

corrected intercorrelations almost identical with their corrected

alternate form correlations; correlations between E and N are close

to the alternate form correlation for N; G form A is nearly as

highly correlated with Q3 in both forms as with G form B; and

finally the pattern of correlations found in the factor correlation

matrices from the restricted analyses is present in very similar

form. The similarity of results obtained by these different

procedures suggests that we are dealing not with computational

artefacts, but with real features of the data. It also suggests that

the Bock and Petersen method would be a useful procedure to adopt

earlier in the sequence of analysis.

Page 138: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

138

h,.- N) if, ..-. No if) No •-• -O ON rn rn cr Q- m No cr No o. CS CO PI tr to N. en PC 41 N) 47N

cr r. CO CA P. CO CO Ul P. tr ..-- C4 .-- CA CA P. CA CA M .- OD CA P. sO ON M sO C4 CO CA CA rC r,

CO CA CA Cs CA CA ,-- NI C4 40 CA CA CO =- .- 47 CA =- CA CO .- .--- Co tr =- tr Co N) CO CA CA sO CO • • • • • ••••••••••• ••••••••••••••••

Co CA CA CA CA Co Co Co CA CA CA CA M Co Mi ..-- CA CA CA CA CA Co MI CA CA CA CA to CA CA CA MA

1111111 I I 1 1 1 1 1 1 I 1 1 I I I

Q' Os =.- PC r4 P. cr r, ON cr un OD Co ,-- CA Cs Cs .-- CO P. ,-- 01-. CA 47 1.- T- P. OD Ps CA Cs P.

rC P, N) P, -43 P. PO Co Co r4 ,-- Ps Ps sO .- Co P. 47 sO ill CA ,- CA -43 SO Os CA CO .- ON Os 143 N) =

CJ CA CA tr CA CA OD ..-- N) cr C4 CA .4. C4 Co MA un M M in ,- 1..., .0 ,-. ,- .- tr sO 43 CA Co ON NO 0 • a • • • a • a • ft • .1 • • • • Isasswas• • • • • • • •• 4.4

Co M M M M CA CA CA M4 CA CA CA CA CA =- CA Co CA CA CA CA CA CA M Ma CA CA CA CA CA CA M 1 1 I I 1 1 ! 1 1 1 1 1 1 1 1 1 1 1 M

=.1

=- 141 =-. Ul Ps P. CA CA ,- 'V ON ,0 h... M T.. ." M h, rs M N) N) CO ..0 tr C4 =4' 0, .-- h. Os MA =

CA CO CO -0 CO sO CA CO CO r4 PC Ps cr -- m .- m r, -- -- -4- -o -- P. cr (•- -- rn -- r, m PO CU

C5 40 CA =- =- Ul =- cr n m M Cl .-- M CA CA =- 143 CA =-, CA 'T =- rn an r4 ...... T-. TT. M 00 CA CA al.-,

.1 • ■ a • • a • • • ■ a a • • a a a • • • • • • a • • • • • •• .1-,

CA CA CA Co CA CA (Si CA Co Co CA CA CO .- CA CA CA CA CA MA CA CA Co (Si CA CA CA (Si CA to Co CA 17.1

I 11111 1 1 1 1 I 1 I 5-

CA NO h. sO CO -0 -40 CA 143 r0 CA N M r... Co PO rs CA tr CP P, CO Ill CA CA 0. P, =- CA 00 OD =- 0

=- Os =4" CA Os P, N) cr PO .- CD Cs cr to -- ,o C4 .- tr =- tr -40 Ps -40 CA N) N) P. PN PN M MA tr 's-

CV CA Co .- -0 N1 tr P0 CA 43 N1 U, CA CA CA CA =- .-- (Si CA P. N) cr N cr cr 147 cr 04 Co =- •-• CA • . • • • . • . . • . . • . . . . • • • • • • . • . • • . • • • r

to m m CA CA CA CA CA M CA Co Co ....... M CA (Si Co CA CA CA CA CA CA CA CA CA CA (Si CA Co WA M 0

I I I I I I I I I I I I I I •.-4 ..1•'

*40 MA CA CA sO ,.. CA CO CO Os PO CA rs. .40 CO N1 ,- PC PC ON Os P1 P. CO 43 un (.4 r, 43 CO Cis rs U

N1 N. =-. CO Ps 143 -40 CO CA ,0 -40 CA tr tr Ps w- CO PC CO C4 0, .- =- <1' N1 143 tr Ps CA sO Os Ps Q1

C3 =-. CA CO CA =- =- !kr C4 PO CA N) Co C4 ..-- tr CO .- to P, N) ...- CA 47 CA N1 ..- =- 00 =- C4 143 -40 f...

•••••••••••••••• ••••••••••• ■ ••••• S-

(50 M M Co Co CA CA CA CA CA CA ,- CA CA Co CA CA CA CA CA CA CA CA CA Co CA CA M4 CA CA CA (Si 0

1 1 1 I I I 1 1 1 I 1 1 1 1 1 1 1 I 1 1 U

0-, Os .0 =- -40 Ps Co CA tr Cs CA NI CA Cl- ID Os cr No si r, un rs 00 N1 CA C4 cr PI 1.0 sO plo, cr al

-- ..-- 0. r, -- siD OD Cs CO CN MA ,0 0, r-, N. CA -0 CA Os CA cr CO MA Os =- cr r. -- an C4 =- cr 4-,

at =- CA rA OD sO rO 1.0 C4 N) N Co NO VI IN rq c..4 n to -- r, -0 N) 47 r4 .-- -Cl 1", .17. 47 C4 .- (Si M • . • • . . • • • . . • . • • • • • • • • • • • • • • • • • • • .1.4

M CA CA CA Co CA CA to CA CA ...- CA CA CA CO CA Co (Si CA CA Co 'Si (Si CA CA CA CA CA CA CA Co (Si r- i i 1 1 I 1 i 1 1 1 1 1 1 1 1 M

>

CA Cls cr CA CO CA CA tr WI CA Os Os N) cr tr "0 tr N) CO 143 Ps VI N. =- =.-. Os N P. Cr- .4] ,-- N) "m

CO CA Os N) tr ,-- CA CA tr MA Os -0 CO NI ...- +- cr m N) r, to so Os CO N1 =- rn SD MA =- .- N) 4-, • M Og ...- r4 ..-- N) rl cr n si (N c-.4 rn n c..4 rl r4 CA .-- r4 =- (54 ..-- CA CA P. N .- -0 CA..- .-- ...-I

• • • • • • • • • • • • • • • • • • • • • • •••••••••• :7,

M CA (Si (Si MI M M M M ..-- M MI M M M (Si M MM (Si (Si MI M (0 M M M M M M M M M E

I 1 I I 1 ! I I I I I ! I

cr cr PN .-- Co CO sO P. CA 4? tr CO 111 ..-- 0, -- cr r, co cr n -0 N 0. Os 43 00 rs CA tr PO CA 13.1

(0 cr 04 sO C.Is r4 CA tP Co cr co IN -- c.4 cg -r -- un co -- 1'0 CO CA CO 4 f4,-.1 10 .0 Os PC 143 C4 .4-'

-J CA (N Ul -o N) CA CA t5 to CA Ni PC 143 Co cr un -- -- -r un tr CA CA =- N C4 sO C4 PC .- N) sO 4-

. • • • • • . • •••••••• •••••••••••••••• to

M (Si CA CA CA CA CA MA =- CA CA (0 M M (Si M M M M M M (0 M M MA M 151 M (FA CA (5) CA 1 1 1 1 1 ! ! 1 1 1 1 1 1 CI

...! CC CA sO 143 Os Co CO Ul CA Ps cr r4 CO CA CA P. sO .0 r4 1.0 143 CA --10 sO N) Os Co .-- CA Ps r. r, cs o

N. so cr- N) 'r in- so '0 cr rl Cs CO N) CO CA N. un No -- CO P., ..... ...... PO in sO tr cr cr n cr un 5-

t 1a4 IN Cl CA r4 CA =- =- CA CA cr cl c...! cq n rn IN rn m rn CA CA =- =- 0-. C4 f.0 Ma cr -- CO rn rte) t 5.. •• ■ ••••••••••••• •••••••••••••••• C1 (Si m n n Co n n -- m so 151 m n to m (0 n to 1 to MA Co CA CA CA CA CA CA 15, CA CA (Si -.4

1.1... I I 1 1 I 1 I I I i I I i I 1 1 Ivo -t-

' O CO 00 =- Ps CA CA 40 sO CA 'Si CA sp. CA cr un Qs -- cr N N) Co PO C4 ..-- .43 =- CA cr cr 43 cr o

un PN .4' ... tr CA CA (Si CO CA CO -Cl cr co m un cr ON '0 N. 40 VI N) N) .- 40 Si 123 47 NI CA CO ..1--

MC cr -- cr N. r, so so -- C4 C4 41 cr N) cr -- rn un m N) un N. CO os -- n C4 PC 43 CA sO CA =-

•••••••••••••••• •••••••••••••••• W

M CA CA CA CA CA =- CA M CA CA CA CA MA CA CA CA 'Si CA MA CA CA CA Co CA CA Co Co MA Co Co MA J.= 1 ! ! ! ! ! ! 1 1 1 1 1 I ! -s,

Cs N) tr ON N) Co Co Co CO Co P, .-- -0 Ps Ps sO .- CA Co Ps CO Ps r. -- cr cr qn OD =- CO -40 Co 5-

.-- .-- 00 M 0, CA rA CI-. Cl T.- .0 Ul N) MI N) CO N) w- 00 Ill tr sO CA 143 ,-- tr Os sO c.-4 N) .-- cr o

CD CA CA CA N) CA CA CA =- C4 N) N) .- cr -- co -- n n r4 CA N) 0) Co Co CA 43 VO =- tr =- CO V? 4- • • • • • • . • • • • . • • • • . • • . • • ••••••••••

M CA CA CA CA =- CA CA CA CA CA CA CA CA CA Co Co CA Co Co Co CA CA CA CA CA (Si CA CA (Si CA CA X I 1 1 1 ! 1 1 1 1 1 I 1 I 1 I I I 1 I ..-1

5-

tr CA N) =- Co N) N CA CA OD sO sO CO P. CA =- N) N) sill P. C4 N) N. M M Os rs CO ON cr s0 47 4J

en 0, ,0 cr to Os '1- tr Os tr .- Is, N. -0 5., CO CA cr co un un -- (N N. ... CA OD 43 N) CO =- tr M

U.... cr rq CA .0 CA CA r. n rn .- No -- N) un r4 CA 43 =- Co NI 0, N) sO =- CA CA tr CA N sO CA =- E

. . . • • • • • • • • . • . • • • . . • . . • • . • • . • • • •

m Co m m -- CA CA CA CA CA (Si CA CA CA CA CA CA (Si CA CA Co CA 64 CA CA CA CA CA CA CA CA CA 1 1 I 1 I I I 1 I I I I I I 0

•tA

rl Ps Os CA ,-- Cs .-- 0, ..-- CA .-- CA sO 40 NI 40 143 CA CA OD .-- -40 sO PN 43 OD C4 CA PI rs CA Ps 4.:.

N) -Cl =- CA tr Co .- N) sO N) Ps 00 Cs CO .40 rs ,-- =- tr CO =- CA CO (Si .- -0 Cl) cr un cr ON T.-. M

ILI CA =- =- CA sO rn r, CA -40 CA 00 CA sO =- CA CA CA -.- CA 03 N cr un rn cr un N. N) 47 =- CA =- .-4

•••••••••••••••• •••••••••••••••• al

CA to Co -- Co to n (Si 10 M M Co CO n M n M M M MI M M M M M M Co M CA Co CA (Si 5- 1 1 I I I I I I I 1 1 I 1 I I 5-

O

cr -0 m CA N) cr CO 141 N. cr No CJ N. -- .- N) -O m CA Ps Ul Os rl 0- CA 00 00 sO N) ON N. Ul 5....)

PI ID M ....... sp 00 cr cs rq cs o. -- n No PN M U3 sO 43 sO SO CA sO cr cr cr No co CA h. CA sO

C-71 =- Co CA =- C4 CA tr CA Ul =- C4 CO =- .-- cr as -- CA OD =- CA CA 47 C4 Ul CA CA 00 CA N) li? PN ••

•••••••••••••••• •••••••••••••••• al

CA CA .-- CA CA Co (Si CA CA CA CA CA CA (0 CA CA CA CA CA CA CA Co CA Co Co Co CA CA CA M M M I ! I I ! ! I I I I I I I I 0

..-- m --.0 h. CA N) CO sO tr CN CN Co N) in ON .- ON r. un os N. co N) N) -- cr OD 143 in CO N -- ,J

cs Co un No ON T.-. N. to cr CA =-. r. cr co N) co CA CA =- CA CO ON co cr cr. cs -- N) cr N) r. Ll f-

pca m CA cA ..-- 04 Co -- 04 c...1 rl CA CA M CA CA CA CA Os =- .- IN CA CA CA CA .- .-- Si (N .- CA Co M • • . • . • • • . • . . . • • • ••• ■ ..•••••••••• CL

CA =- Co CA CA CA CA CA CA CA CA CA CA CA to CA CA CA CA CA ,:.. -7.. ..•.. CA CA CA CA CA Co M M M

I 1111 I I I I I I I 1 I cr

(Si =- tr N) cr cs ..00 m cr 0.1 ON sO C4 =-. .4" N. l'n IN 43 ON un cr cr No r, CO rs ON r, Co WI =-

CA Cs N) N) CA ,-- 43 Ps CA 00 =- N) ON 00 N. N un CJ m co -- un -- No r, un N -0 N. .- CA =- 111

'Z Co CA ,--- CA tr CA cr CA CA CA =- =- CA 43 CA CA CO- CA CA CA tr r4 tr =-- .- N .- .- CA Ps .- Co .-1

. • • . • • . ••••1•••• •••••••••••••••• An

T.. M to M Co M M M M M M M M M M (Si CA CA CA CA CA CA 'Si CA Co CA CA Co CA CA CA CA M I I I 1 1 I I 1 I I I 1 I P-

CV 41. CV N1 cr

<C LJ CL.1 1..1_ CD 7C t~ -J 2.C. CO CN CO CN CN .ocr i74 GJ La Li_ CD MC F-4 -IM.7. :it CD CD CN CN CN

• L f- 5- 5- 5- f-. 5- 5- f- f- r- r- 5- 5- C.. b1:1 C- 5- C- Y C- L L L C- C- L C- t- C- C- 00000000000000 00 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

V .0 4., 40 .0 4., 4-, -I-, .4., .0 4-, 4-, 4-, .1.) .4., .) .4.., .0 4-, -0 4., .1-, 4., -0 4-, 4-, 4., 40 U U U U U U U U U U U Li Li LJ Li u U u U U U U Li U u U U U

ID CO It CO Ca CO CO CCI MI CO CO MI CO C1:1 Ct CO CU CI 11:1 CO CO CO CO CO It It CO jO MI CO It 03 CO 11:1 • LL L 1J- 1.4.- Li- U_ U- U- Li- Li_ LL L U.- UL U- L L U- LL Li_ Li- IL Li- LL Li- L. LI- Li- Li- LL

Page 139: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

139

.y.- 10 V, DO CA a- or. (-4 rn a r, -- m r, r4 No ri NCI NO 'Cr M T... in T... T... a c• co m N

a- -- un No .r- a Y41- CO U') (N VI ..q.. N. v- 1,0 mi N. 5-4 ro m un un rn C,4 N) ...- r ON el, rn rn R Co

cn Co Co r. ,- ,- rl -- rn No -- Co No Co Co -O OD Co Co OD Co r- r4 m N .7' ..-- a- r, -- -- m • . . • . . • • . • • . • • . • • .••••••••••• • • m m m m CA CA CA CA CA CA CA CA CA CA MA MA CA CA CA CA CA CA CA CA CA CA CA m m m

I 1 1 1 1 I I 1 I I 1 1

rn m N. (....1 ..-- (-4 a 10 Y- .- ON M M Ul in (Si In IM ID la (Si -10 Y.- a un No

cr. L1 r, N. N LO N. VI T.-. ON Cr, CO ON ON un o, r, r, -CI -o un Cl". ro PO

ON r- MA 10 CO CA VI ra -- ,- DO ....- CA 0, sO M M "0 (Si Oq sO CA rA LO ,0 Y- 00 ro 1'N

0

••• • • • • • • • ••• •••••••• 4 •• • ..-I

CA CA Co CA CA (Si CA (Si CA CA CA CA CA CA CA (Si CA CA CA (Si CA (5 CA CA CA Co CA CA ..P

1 I 1 1 I 1 1 I 1 1 1 M O

CA CO 0, CO V- r, ,,r -0 ,C1 CO Co r, m ra PO 'AD CO ,0 DO rp rn cr• N. m ct, m r-

CSA ?...2 !:.2

ii.

N -- VO N. VI VI CA PO r- N ,C1 (Si N. CN 0.4 f.., rl DO PO Cr' N) CO Yr rn -- rn

cn r, ,- rn -- No m ,- m CA 04 ..-- OD CA r4 -40 CA PI ....- DO un m m PO r-. 4=

••• • • • • • • • • • • hi • • • • IS • • • • • • , 4=

CA CA CA CA Cal CA (Si (Si CA CA Si CA CA CA CA CA CA (Si CA CA m CA CA CA Cal CSa CA CA CA Co CA m

1 I i 1 1 I 1 I I 1 1 1111 I 1 r-

V, bl ra rn 0'. -- a- N. CA ON 10 ,D r4 Y-- N. VI CA 1.11 0, 0, N') a- a a rp --- un m m No co o

-- r, a CA DO PO CA DO V.' Cf.. CA CA CA r. -- Cl- CA r, --• m cr.. CO CA a No CO ^O CA PO 1...., VI 4-

C) CA CA CA DO CA V' r4 .r. PO ',ID DO Yr CO CA r4 CA Y- CA ...... -.0 ....- CA Y.- CA C4 V, 'V- s1 CA Y- ...- Y-

•••• ••• • • • •••••• •••• ••• ••• • •• • • ,--

CA Co (Si CA (Si CA CA CA CA CA (Si Co CA CA CA CA CA CA CA CA CA CA CA MA CA Co (Si ....- CA Co Co Cl

I I I I I 1 1 1 I I I I I 1 .1-1 4=

In .0 NO:1M r4 CAN N PON .- CI, CO PO N) '•O to co -O ,o co ...- VI N CA 'AD DO 0, LI

PO CO a DO ,0 bO V- ,0 -0 r- N. r. rn -- co CSI ,r 0... n ...- in ON r, L1 oo -- rn cr. Q.

CD rMCOMIO4 ••-• 1.0 V' 04 •-• .7' CO IN .-- U1 O? r4 stalra ra CSI -o N) 04 ,- pp "0 5- • • ••• • • • • • •• ••• •••• ••• • • •• • r■

M M CSI M M M M M M CA CA CA CA CA CA CA CA MA MA CA CA CA (54 CA Cat CA CA CA CA 0

11111 I I I 1 I I 1 1 1 I I 1-1

CO CO r4 m -- co r4 a- 0.1 V... a- r, m co a- cl- o -a (.4 -o 0, TO r CA MA U1 N. a a cu

No Co Co C, CA 1.0 ro N "--1- N •-• CON Sif CO fM N IDNIO ..-- -.0 CCI CA CO m la co or. 4= = T.- C4 a- LO pa CS7 "0 IN r. -- a. ...... No rn CA CA a No un un -- Co rn m -- ill

• • • • • • •••••••. • • • • ••• • • • • C51! 10. a

m m CA (Si (Si Co Com ....I

m st m m Com (Sim m m m (Sim (Sim m m Com m m CA Y.-. CA 5-

I 1 1 I 1 1 1 1 1 I 1 I 1 1 I I I I I M >

Yr OD CO C, -.10 CS1 bl 0, CA DO 0, rl Y- V, a- a- a- ,-. ..40 DO CA V/ 111 CSI a N. ..- V) Y--. ,- •TI

ION at3. .0 Co DO VD .r. a- un N) -- N No m -- cr. N, .- ra N m -- Co DO 'AD N. CA CA 4= 1= T.. M DO r4 U1 CA ro r4 N -40 Y- UO ...- V• CA CA CA CA V) (.4 ra (..4 rn m 17 No m .-- ra -- --1

• • . • • . . • • • • • • • • • • • • • • • • •• :..-i

CA m Co m Cat (Si CSt Co Co m (Si m m Co m m m m m m m (Si m m Co m Co (Si so Co E 1 I I I I I I I I I 1

e....

r4 VI CA Y- 0, 0, ...... CA DO CA a- -- un -- CA DO 10 CA V... PO -4- m un rn DO VO CA CA r- CD

-J 1CA LO a- m m CA N. CA ..... VO a- (.4 -- a- -- m a- a- m ....- -- ra m m rn r4 (74 1..:1.-j 22 .7,T ,..-.

Cl'r. -4- ,- ,- -- in ,r m -- rn PO Co 0, CA VD CO r. Cr' co -- ra a (Si -- No 4=

• s • • • • 4 • s ■ • • • . a • • • • • • • • a a • • • • al

CA CA CA CA CA (Si CA CA Co CA CA CA CA CA CA (Si CA CA CA CA CA CA CA ...... CA Co Co CA CA (Si (Si

1 1 1 I 1 1 I 1 1 1 1 1 1 1 1 1

P4 LI PI Cl- Co r CA ND Y- VO Co CA ,10 DO a cq IN -.0 -- r4 Y- Co a to cr. ,- a C. ra rn ..:7 CI

a- a- CSt r, U1 ra N) C6 00 0, a- rq a- No cr. N CO MA '40 Y.- PI CA a- (Si ,- cr. CA ....- .- N) E ,--1 r4 rl M ...- CSi r- 0, r CA CA CA V' CS 4- ....- CA (Si CA N) CA Y-- CA PO VO CA VI CA CA IN r4 C1) .5- • • • • • • • • • • • • • • • • • • • • • • • •

O CSa Co M CSt M C5a (Si CA CSt (Si CA Co (Si Co CA CA Co Co CA CA CA CA r- CA (Si CA Co (Si (Si CA CA ..-4 U.. I I I I I I I I I I I ) I I I I Ill

a M ra r...... rl No r, r, DO OD CA NO DO 00 n ON Cr n CG ,- CI

..--- -0 CA N) ,- or. --- -o N. --1.1 ,C1 Cl. CA 0, OD DO (Si PI P: F*1 5; CO CO -.0 CA a- ro o-

..1..,

:a -a C5a un L1 -0 0, Yr ...... V) CA rte) Y-. a- a- (1.54 'Cr -a (Si m -- -- N --- -- un CA ro

. . . . • • • • • • • • • • • • • • • • • . • ti

Co C5a m CA m C51 m Co CG Co CSt (Si (Si (Si m m (Si (Si m m (Si Y.... CA (Si CA (Sito 1 1 I ! 1 I I 1 1

(Si CA Co (Si .--

1 1 1.-.'

.4- cr. -0 VO r, OD -0 D1 NO 00 PO 0, OD CO V' r..... -o -o CA m CA r-, un CA V' VO LO 0'11 r-

un m CA ....- 10 Y- .40 ..--- V, Y.- r4 r, -- a -- un m m bn -- -- Cat CA 0, PD m! VD C1

CD CA CSa CA a- rn CO m -- N m m a- -- No CSI ,- m --- la a- si m CA Y-. ra DO rl Y.- ,0 IN 4- • • • • • • . • • • • • • . . • • • • • • • • • •

CA CA Co Co CA CSa Co Co Co CA Co CA Co (Si MA CA Co (Si CA Y.-. t54 Co (Si Co (Si CA CA CA (Si (Si ›..:

I I I 1 1 1 I I I I I I 1 1 I • rf

VI r. un ,- N 00 VI CA CSi V, DO C, N. M Y- 0, a a- N) -- Co No CO a- rA ,0 4" ,0 h0 Ull •O .4"

...... -CO SO . If) '1" LM I", Co a Cr. No .- m -- -- (Si un (Si m 41 C, OD ...- 10 CA R. 0, VD U1 113

Li. a (.4 c4 r, cr. ra V, CA a ..-- -10 Y- VO a PO .,-- 10 ..-- Y.-. V- CA a- No -- m CA DO VD ..-- DO r..., -- T.- • • • • • • • . • • • • • • ••••••••••• 44 • ••

Co Co Co CA (Si Co Co CA CA (Si Co CA CA CSA CA CA CA CA Levi CA .r. CA Co CA CA Co CA (Si CA CA CA CA ..-

I I I I I I I I I 1 I I I I I 0 •1-1

0, 0, V, CO V, V, 0.1 1.0 a- un r, ON 00 Lit i'.. Cl. 00 n un n --- -0 0, Y.- 1.1 Y- -40 OD 0, -.0 -.40 .40 .1.,

CO fS! -4D CO LO 1.0 N. CO •--. N. (Si C.'S .0" CA CO 0, SifV- Co Li) 1.0 CO -0 0, N. rA N. (Si N) In In iri

LU Co ...... Y- CO NO N bn (....1 DO CA VI T.....Ca ....... ...... V,..! r- M (Si 'I" P.7 a- rn a un No ra Np -- Co CA ..-1 • . • • • • • • • • • • • • 1.• • • • • • • • • • • el

CA Co Co CA CA Co CA CA CA Co (Si (Si (Si CSI Co CA Si Co CA Y- Co Co CA CA CA CA CA CA CA (Si Co (Si 5- 1 1 1 I 1 I ! I 1 I 1 I 1 I -

ci

DO DO 04 N 'AD Co V" DO OD CO CA NO 'V T..... 00 ,0 ,- ON 14 IM l'O N. M -.0 10 a- cr. rp 0, CO -tp Li

CA Y- DO a CO oo No -- CO rn o... CO -- -- til -- CO un m V' CA Y- 0, CA V, ON n CN T- DO 10 ml

L) (N ..-- CO CA (Si 0.1 VO NI ..tr ,- ,- N. CA Y.- LO CO — (Si CA Co Y- ..-- Yr CA a- (Si a- co -- N) -.0 CO ... • • • • • . • • • • • • • • • •••• •••• • • • • • • in

Co CA CA CA (Si CA CA CA CA CA CA CA CA (Si Co Co CA CA ..-- (Si Co Co CA CA (Si CA Co Co CA CA CA Co 31 1 1 1 1 1 1 I 1 1 1 1 1 4=

rl V, Co CA VI CA ...... CA r. ND ,1 N) IN N -- a- ra m or. 7 a- a. CCI N m a- a- No un No r, ('I Si-'

Cq M 40 T- 'ne. T.- ON '40 U0 (Si (Si In a. -- No (Si a- m un Ca ,- a- CO Co Y.- OD Yr N r4 to ra 5-

OA M CI, M ..... r- (Si (Si M T- MI Co M M Co M M M M M T- M CAIS4 M (Si Co MI (Si (54 Co m RI • • • . • . . • • • • • • • • • ••• • • • • • •• • • •• CL

Co CA Co (Si CA Co Co Co CA Co CA CA Co Co CA Co Co YY CA C5a CA Co Co (Si CA CD CA CA CA CA (Si I 1 1 I 1 I 1 1 I I 1 I 1 ar

ND ON sO Ill C.) ,.... ON sO a- a- a- ,- V, Co Cr'. '40 CA VI r- CO a- CO IN -- a- co N.., (Si NI 0' SO

If) Co 10 ..... Co NI a- to ,- a No oo .r- r, un m Co a- CO IT -- ,- (-4 ra m (Si Co No N, Co rl

.0 ON CP ...... 0.1 UO CA 1.11 PI r- CA CA qr. Y- Ul CA ..... CA CA ...- 41 r- rA ....- (Si CA CA 4- ND CA Co ,....f W

• • a • • • • a . • • • • • •• •• • •• • • • • ••• •• ..0

M IA Co Co m Co M Co Co Co CA M M Co Co M ... (Si CA CoM Co CA Co Co M M Co M M M M M

I I I I I I I I I I I-

¢ OD L.) 4J 4- CD 1-4 Co CI CD CI ./Z C:0 C..) LLJ 4. T -J Cl

• = L L L L L L L L L L L L L C- il4 L L L = L = L L = 5- =

O 0 O 0 CP 0 0 0 0 0 CD 0 CI 0 0 CI CD 00 0 C) CI O 0 0 CI 0 00

/C • .,.a 4-, 4-4 4-, 4.2 .1-2 4-, 4-3 4-3 4.2 4..2 4-2 -I-2 4-, 4.2 4., 4-3 4-2 4.2 4.2 4_a .4.2 4.3 4.3 4-2 4.2 LS u U U U U L) U U U U C.I U u LJ U L U LJ LJ u U LS u U Li U U U U

0 Co (0 (0 RI (0 (0 CO CO (0 (U IL (I3 IV (0 (0 (0 0 03 03 An IV (0 (0 IV f0 IV CO (0 RI CO LL L.L. U. U. U. U- LL. U. U. L. U.. U. U. 1.1. U.. U. U. U.. U. U. Li. U. LL. Li.. L.L. 1.1. U. Li. Li. 1J- LL.

0

Fac

tor

n?

Page 140: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

140

Since in the analysis of scale scores we are dealing with

composites formed from scores on mutually exclusive sets of items,

let us pursue the questions raised by these analyses to the item

level. Remembering that Cattell has consistently claimed that

items loaded by a given factor are designedly hetergeneous, a

claim vigorously endorsed but otherwise interpreted by his critics,

the correlations between items and scales for the items making up

scales A, E, F, H, Q1 and Q2 were inspected in the light of the inter-

pretations proposed by Cattell for these scales. These six were

chosen partly because none of them had been called into question

in the analyses so far, and partly because of their consistent place

in the pattern of Table 18.

Firstly, an interpretation for each scale was formulated on

the basis of its most distinctive item content in the light of Cattellts

own interpretation. Then each item in these scales was checked to

compare its correlation with the composite formed from all other

items in both forms of the same scale with its correlation with all

other scales (again at two-form length), excluding L and N on

account of their uniformly poor showing throughout. The items

were then sorted into four categories:

1. Where the proposed interpretation clearly fitted item

content and the item-partial scale correlation was higher

than the correlation with any other scale.

Page 141: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

141

2. Where the item-partial scale correlation was higher

than the correlation with any other scale, but the

interpretation did not clearly match.

3. Where the interpretation matched, but the correlation

with another scale was within + 0.03 of the item-partial

correlation.

4. Where the interpretation did not match, and the item-

partial was exceeded by the correlation with another

scale.

In a small number of cases the interpretation matched, but the best

correlation was with another scale. These items are discussed

individually. In what follows, a typical 'matching' item is given

for each scale, and where an item correlates about as well or better

with another scale, this scale is given in brackets.

Factor A

"If I had to choose, I would rather be

a. a forester

b. uncertain

c. a secondary school teacher" Item 51 form A

Interpretation: preference for occupations and leisure activities

which essentially involve social contact.

Page 142: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

142

1. Form A:

Form B:

3,26,51,76,101,176

3,16,51,52,101,126,151,176

2. Form B: 76 (would prefer to speak to a stranger in a railway carriage rather than stare out of the window)

3. Form A: 126(I)

4. Form A: 27(H), 52(Q2)

Form B: 27(I)

Item 151 in form B correlates more highly with I: the choice

here is between being an artist and being a secretary running a

club. I and M typically show strongly when artistic or abstract

intellectual content is involved.

Factor E

"It is more important to:

a. get along smoothly with people

b. in between

c. get your own ideas put into practice"

Item 57 form B

Interpretation: socially assertive

1. Form A: 6,7,56,156,181

Form B: 6,31,57,106,131

2. Form B: 56

3. Form A: 131(F, Q3), 180(H)

Form B: 7(Q1)

Page 143: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

143

4. Form A: 31(Q1), 32(H), 57(H), 81(G), 155(F)

Form B: 32(0), 81(H), 155(Q1), 156(Q4), 180(I), 181(H)

Item 106 in form A fits the interpretation but correlates better

with H; the interpretation of items 180 and 181 in form B is

dubious, but on balance they have been placed under 4.

Factor F

"I am well described as a happy-go--lucky, nonchalant person

a. yes

b. in between

c. no "

Item 183 form B

Interpretation: Socially active, carefree and optimistic

1. Form A: 58,83,108,132,133,157,158,183

Form B: 8,33,58,83,132,158,183

2. Form A: 8

3. Form A: 33(H), 82(H, Q2), 107(H, Q2 )

Form B: 182(E, G, Q3)

4. Form B: 107(H), 108(H), 133(A, I),157(A)

Item 182 in form A, "I am considered a very enthusiastic person",

correlates better with H, but in terms of content is rather more

like the interpretation offered. Item 82 in form B is difficult to

place between F and H, and in fact correlates better with H.

Page 144: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

144

Factor H

"'Stage-fright' in various social situations is something I have

experienced:

a. quite often

b. occasionally

c. hardly ever "

Item 85 form A

Interpretation (for low score): feels threatened when the focus of

attention in social situations.

1. Form A: 10,35,61,85,110,161

Form B: 35,61,85,86,110,136,161,186

2. Form A: 86,111,135

Form B: 10,36,135

Note: these items are very typical general extraversion items, e. g.

item 135 form A: "I consider myself a very sociable, outgoing

person. "

3. Form A: 60(E), 186(Q3)

Form B: 60(0)

4. Form A: 36(F), 136(F)

Form B: 111(F)

Factor Q1

"I think society should let reason lead it to new customs and throw

away old habits or mere traditions:

Page 145: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

145

a. yes

b. in between

c. no"

Interpretation:

Item 169 form A

socially and politically radical

1. Form A: 46,120,169

Form B: 70,120,145

2. Form A: 70

3. Form B: 95(G), 169(E)

4. Form A: 20(I), 21(Q3), 95(E), 145(E), 170(G)

Form B: 20(H), 21(M), 45(Q3), 46(M)

Factor Q2

"To keep informed I like:

a. to discuss issues with people

b. in between

c. to rely on the actual news reports "

Item 96 form A

Interpretation:

and behaviour.

does not look,to social group for norms of opinion

1. Form A: 71,96,122,146,171

Form B: 72,121,171

2. Form B: 47

3. None

4. Form A: 47(H), 72(E), 97(A, H), 121(E, H, M)

Form B: 22(H), 71(H), 96(C), 97(H), 122(F), 146(Q3)

Page 146: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

146

Item 22 in form A: "Most people would be happier if they lived

more with their fellows and did the same things as others", has

very low (less than 0. 1) correlations throughout.

In general, when an item did not fit well the proposed inter-

pretation for the scale on which it is scored, its highest correlation

could be located by consideration of its content in the light of the

interpretations of the other scales. For instance, item 31 in

form A, which is scored on E:

"An outdated law should be changed:

a. only after considerable discussion

b. in between

c. promptly"

was clearly more like the typical items of Q1, and indeed proved to

be more highly correlated with Q1. When such a relationship was

not apparent, H tended to be the scale giving the highest correlation.

Items with an artistic content tended to correlate well with I, those

with abstract intellectual content with M, and those with moral or

ethical content with G and/or Q2. The majority of items

categorised under 4 above fit the interpretation proposed for the

scale with which they correlate best.

This, admittedly subjectively based, analysis allows the

drawing of finer distinctions amongst these scales, as follows:

Page 147: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

147

(1) A, E, F, H and Q2 all refer to the behaviour of the individual

in social contexts, whereas Q1 refers to attitudes to social and

political ideas and conventions.

(2) A, H and Q2 are in a sense "afferent" factors: they refer to

the response of an individual to or in a situation, casting him in a

relatively passive role, whereas E, F and are more "efferent",

referring to the extent that he initiates activity or change, i. e.

casting him in a more active role.

(3) A refers to a preference for situations involving social

contact, Q2 to a certain dependence on socially defined reality;

H contrasts confidence and shyness but does not imply a tendency

to initiate social contact or activity, whereas F implies a readiness

to initiate and innovate, but not necessarily to dominate or aggressively

maintain one's position, which is the major burden of E. Q1 refers

more to ideas and opinions than to actual behaviour.

Thus interpretations modified only in detail from those

provided by Cattell fit rather well 67 of the items scored on these

scales, only 11 items which correlate best within the scale on which

they are scored do not fit the interpretations, 13 fit the interpreta-

tions but correlate about as well with another scale, only 3 fit the

interpretation but correlate better with another scale, 3 are doubtful,

and the remaining 41 correlate better with other scales and by and

large fit the interpretation of the scale with which they correlate

best rather than that of the scale on which they are scored.

Page 148: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

148

Now it may be that what is at work here is Cattell's (1972)

claimed suppressor action, whereby items not strictly loaded by

a given factor are nonetheless scored on the corresponding scale

to balance out unwanted variance introduced by other items. On

the other hand Cattell has nowhere indicated just which items are

supposed to be suppressing what. It seems likely that rescoring

of the items in line with the kind of content analysis just attempted

could lead to a clearer structure, increase the statistical homo-

geneity of the scales, which as has been shown relates rather well

to their factor validity, reduce the intercorrelations of the scales

and maybe even improve the pattern of Table 18. However, such

a reanalysis would require a separate study to avoid the possibility

that the above results are adventitious.

What can be said is that such an approach to the scales via

content analysis and item-scale correlations strips them of any

mystery which may be implied by their origins in factor analytic

research. There is for each of these scales a core of similar

items which tend to be those with the highest correlations with the

scale. Eysenck's (1972) comment that Cattell's scales are complex

in content, again a point with which Cattell has often implied

agreement, is true only to some extent, and the extent to which a

scale seems complex is 'dependent on one's interpretation of item

content. For instance, the items of factor A are by and large

Page 149: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

149

concerned with occupational preferences, but the overall contrast

between each pair is clearly related to the amount of social contact

they involve.

For these six scales, then, there appears to be a clear but

imperfect link between item content, item-scale correlations and

slightly modified versions of the interpretations offered by Cattell.

The remaining scales all have similar clusters of core items closely

related to their interpretations, but their poorer homogeneity and/or

their high correlations with other scales make the comparison of

item-scale correlations less promising.

Beyond item content, there is the further question of item

format, and in particular Cattell's provision of three response

possibilities for each item. The second of these takes various

forms: often "in between" is used, or "uncertain". The instructions

for the test refer to these second alternatives as "middle, 'uncertain'

answers", and exhort the subject to choose, wherever possible,

either the first or third, falling back on the second only about once

every four or five items.

Percentage figures of responses to each item show that by and

large participants in this study did as they were asked, but there

were large differences in the use of the middle category. For

instance, 51.41% of answers to "I make clever, sarcastic remarks

to people if I think they deserve it" (item 7, form A) were

Page 150: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

150

"sometimes" rather than "generally" or "never", whereas only

9. 56% answered "I occasionally tell strangers things that seem

to me important, regardless of whether they ask about them"

(item 131, form A) with "in between" rather than "yes" or "no",

with the remainder about equally split. Yet the former has a

higher correlation with its scale than the latter.

One type of item seems particularly prone to large numbers

of middle responses, viz. those with alternatives "often; sometimes;

never", "always; occasionally; rarely" and the like. Twenty-

eight such items are distributed across the two forms: of these 6

are scored on factor C, 1 on E, 1 on G, 4 on H, 1 on I, 1 on N,

5 on 0, 2 on Q2 , 2 on Q3, and 5 on Q4. Twenty-three of the

twenty-eight have response percentages above 30 for the middle

answer, rising as high as 63. 39% for item 123 in form A. These

items are with only two exceptions the most heavily endorsed in the

middle category in each scale; item 124 in form B is one of these

exceptions, only 13.01 per cent using the second alternative, but

here the choice is "occasionally; hardly ever; never" and over

70% choose "never"; the other exception occurs in factor C form B

where item 104 (26. 59%) is beaten into third place by item 5 (which

itself has "occasionally" as the middle response), and item 4 (a

mere 20.96%) is fifth after an item with "in between" as the second

alternative.

Page 151: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

151

However, these items show no consistent deficit in their

item-scale correlations, nor do items with "often" or "occasionally"

in the body of the item show any special statistical features. Such

terms are of course eminently open to interpretation by the

respondent, and even "never" and "always" may well be subject to

fine distinctions: we are of course dealing with self report data,

which carries no warranty as to its accuracy, via multiple-choice

items which may not allow the individual to specify his most

preferred response.

This brings us to the troubled subject of the relevance of

questionnaire items to individual respondents. Item content which

touches on issues of central concern to one individual may seem

hopelessly irrelevant to another on account of its specificity, yet

if items are written at a more general level they may fail to capture

the distinctions which are most relevant in any given case by being

too all-embracing and thus impossible to answer decisively. In

other words, not only may "often", "rarely"and the like be

interpreted differently, whole items may be relevant in different

ways or not at all and be subject to different interpretations

accordingly. Bem and Allen (1974) have had some success in

improving the performance of personality test scores by the simple

means of asking subjects whether or not they regard themselves as

consistent in particular respects and whether they find various sorts

Page 152: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

152

of item content sufficiently central to their lives to be interpretable

as test material. One might read the middle response categories

in the 16PF items as providing something of the same kind of

information, although it is not treated separately in the scoring

procedure, which also does not make provision for the omission of

items.

If one accepts that the process of answering a questionnaire

item engages conscious processes of thought, whether they be

attempts to remember past behaviour, predict future behaviour or

articulate a self-concept within the framework provided by the item,

then scores must reflect not only patterns of overt behaviour (indeed

they may not even do so), but also the processes of interpretation,

recall and self-presentation. Cattell shows some appreciation of

this fact by referring to questionnaire scores as reflecting the

"mental interiors" of factors, and it is of course a major focus of

personal construct theory in the tradition begun by George Kelly.

Similarly, any attempt to detect faking of response and social

desirability response sets implicitly recognises the scope for

interpretation of items and self-presentation through responses.

If the involvement of conscious processes of interpretation and the

like is seen not as an aberration but as part of the normal business

of answering a questionnaire, then one would expect consistency in

the individual to emerge through the sorting of items into groups

having subjectively similar content. The form in which responses

Page 153: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

153

are recorded is far from ideal for the detection of such consistency,

but the evidence of the analysis of item-scale correlations above

suggests that the participants in this study overall sorted in a way

more consistent with the author's interpretations of the items than

with the scoring key provided by Cattell. Perhaps the fact that

many of the analyses reported herein are amongst the most favour-

able to Cattell in the history of research on his scales outside his

own laboratory reflects the homogeneity of the sample, a sample

with, one might assume, a fairly uniform life-style and considerable

commonality of outlook and experience.

To develop this point of view further, consider first of all

the fact that in scoring a questionnaire much of the specificity of

item responses is discarded, items being grouped according to the

author's notions of similarity, however derived. Next, assuming

for the moment the validity of such groupings, the further grouping

of scales into second-order factors implies a further loss of

specificity. Should this specificity be regarded as unreliability in

the sense of random error, or information which is valuable for

some purposes but irrelevant for others? Eysenck (1972) sees the

specificity of Cattell's first-order factors as unreliability but hints

that in the case of those scales which are loaded by the second-order

Exvia factor there may be reliable specificity, quoting Frenkel-

Brunswik (1942):

Page 154: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

154

"Different classes of behavioural expressions were often related to one drive as alternative manifestations of that drive ... One drive variable may circumscribe a family of alternative manifestations unrelated to each other: the meaning of the drive concept emerges in terms of families of divergent manifestations held together dynamically or genotypically, though often not pheno-typically. "

For comparison with Eysenck's Table 2 we have Table 19, which

shows rather more clearly that twelve of Cattell's scales have

sufficient specificity replicable across forms to suggest they may

have some utility.

However, if one is interested less in the prediction of

individual behaviour and more in the physiological correlates of

personality, it would seem reasonable to look to a level of greater

generality rather than a level which is sensitive to the cognitive

processes of the subject, and of course it has been at the level of

second and third order factors in Cattell's terms that the greatest

success in finding physiological correlates has been achieved

(Cattell and Warburton 1967).

What then is the psychometric status of Cattell's second

order factors? Table 18 shows rather little evidence of a cluster

structure amongst the first order factors, although this might

change after rescoring along content-analytic lines. The largest

steps between adjacent coefficients are between H and C and

between G and M, which might be read as marking the boundaries

between an extraversion-like and a neuroticism-like factor in the

Page 155: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

155

first case, and Cattell's second order superego and independence

factors in the second. However the evidence is far from compelling,

and attempts to rotate principal components of the matrix in Table 18

showed no clear hyperplanes using graphical techniques. In any case

the step sizes are not all that large and their presence could as well

indicate missing factors as distinct factors at a higher order.

More in line with the interpretation being developed would

be Gray's (1973) attempt at a reinterpretation of Eysenck's factors

in terms of three major emotional subsystems, represented by

distinct neuroanatomical pathways and emerging in conditioning

studies and social and psychiatric aspects of behaviour.

Particularly if the rescoring suggested above were carried out, one

might imagine that H would form the core of Gray's dimension of

"susceptibility to punishment", occupying as it does a place between

extraversion and neuroticism type factors. Certainly the central

item content of H reads very much as susceptibility to perceived

threat in social situations. Most likely candidates for his second

dimension of "susceptibility to reward" would be E and F, with

their connotations of approach, initiation of activity etc. For Gray's

third speculative dimension, which he tentatively identified with

Eysenck and Eysenck's (1968) dimension of psychoticism, the

obvious candidates by exclusion would be G and Q3, and indeed

Montag (personal communication) has carried out a factor analysis

of Form A of the 16 PF together with the three Eysenckian scales

Page 156: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

156

and found that these two are loaded by the same factor as

psychoticism.

It is not being suggested that Gray's three proposed

dimensions can be conclusively shown to have a clear place such

that they will naturally emerge from questionnaire data. Rather,

Table 18 suggests that second order factors could be placed more

or less at will. If at the physiological level there are three (or

however many) distinct subsystems which have an important role in

generating the variety of human personality, the evidence from

this study is that they do not appear in questionnaire data as the

dominant features. However, the item-scale correlations do

suggest that participants in this study effectively sorted the items

by content, and put for instance item 157 in form B ("It would be

more interesting to live the life of a master printer rather than that

of an advertising man and promoter") in factor A (correlation of

0. 352) rather than F (correlation of 0. 239) where it is scored.

In the item content of the proposed candidates amongst Cattell's

scales for Gray's dimensions there is a good deal of similarity

with his descriptions of behaviour under the control of the three

subsystems.

The similarity for the Flight/Fight subsystem is at best

slight, but many of the items of H and C read plausibly as referring

to behaviour in response to conditioned stimuli for punishment and

Page 157: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

157

nonreward (the Stop subsystem), and of F and E to behaviour in

response to conditioned stimuli for reward and nonpunishment, the

Approach subsystem.

Thus what is being suggested is that there is a role for both

the cognitive processes of the individual and biological variables in

a comprehensive account of those aspects of personality which are

captured by the scales of the 16PF, at the cost of denying Cattell's

primary factors their ontological primacy. The first order scales

may be viewed as attempts to measure interactions between

relatively primitive emotional mechanisms and hypothetical

situations, mediated by the cognitive processes of the individual to

a greater or lesser extent, on the assumption that a majority of

individuals in a given cultural or social group will construe each

item in a similar way. Such an account draws together aspects of

the determinism implicit in learning theory, physiological approaches

to psychology and the trait tradition in the study of personality on the

one hand, and the emphasis on individuality, purpose and the central

role of each person's cognitive apparatus and strategies

characteristic of the work of Kelly and his followers, and more

recently Harre, laying the ultimately inevitable imperfections of

psychometric techniques at the door of the irreducible individuality

of men and women.

Page 158: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

158

The question "how many factors?" then becomes not an

ontological question of great scientific significance, but a practical

question: how many factors are needed to preserve as far as

possible the greatest amount of information about individuals whilst

still preserving comparability of scores across individuals? The

answer to this will depend not on some fixed rule but rather on the

purpose for which a scale is employed and the homogeneity of the

subjects involved.

Such a scheme has at least a superficial similarity to

Guilford's (1956) structure of intellect model, with three major

classificatory factors: situations, interpretations and responses,

with the first a function of the forms of social life embedded in a

given culture, the second classifiable only insofar as individuals

share common perceptions of the possible range of significance of

situations within that culture, and the third possibly corresponding

to a scheme such as Gray's.

The greater utility which has been claimed for Cattell's

first order factors will then be a consequence not of their peculiarly

preeminent causal status but of the extent to which they embody trait

x situation interactions in their item content in a way which maps on

to the constructs of the subjects. More precisely, what is proposed

is a trait x situation x interpretation interaction model not unlike

Argyle's (1976) account and consonant with Cronbach's (1975)

discussion of aptitude-treatment interactions.

Page 159: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

159

The evidence of this study is that to a moderate extent

Cattell's scales capture sufficient commonality in such interactions

to make them potentially useful instruments at least with under-

graduate subjects. In such circumstances, it is not circular to

ascribe explanatory status to scores: to say that an individual avoids

public speaking because he is low on factor H may be no more than

shorthand for "he is rather more than averagely averse to perceived

threat in social situations, and he perceives public speaking as a

potentially threatening situation", but it is useful shorthand insofar

as it brings a variety of possible situations under a general rule, and

distinguishes this individual from others who avoid public speaking

out of lack of interest or a preference for asserting their importance

by way of political intrigue.

However, to the extent to which a person is idiosyncratic in

his perceptions and to which a given situation is novel, scores

derived from a questionnaire will lose both explanatory value and

predictive power, since ultimately the method depends on eliciting

self-report of typical behaviour in classes of similar situations.

The scope for change in an individuals score on such a test will

therefore be twofold: he may become more or less sensitive, say,

to threat, or he may change his ideas as to what counts as threatening.

Page 160: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

160

To summarise this speculative account, based tenuously as

it is on psychometric evidence from a survey, it has been suggested

that trait-situation-interpretation interactions are the stuff of

personality assessment. Accurate prediction will involve knowing

the situation, the general class into which an individual would place

such a situation, and his typical behaviour in situations belonging to

that class, and even then one cannot discount the possibility of

dynamic change in any of these once the situation is real and he

must act rather than predict what his action would be. The

remarkable thing about personality questionnaires is not that they

work well, which on occasion they have been known to do, but that

they work at all.

Page 161: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

161

CHAPTER 10

This study has yielded results some of which were not

anticipated at its inception. From a technical, psychometric

point of view, the computation of measures of internal consistency

and their comparison with alternate form correlations suggested

that the scales of the 16PF were a good deal more homogeneous

than either Cattell or his critics had led one to suppose. With a

small number of exceptions, they were quite as homogeneous as

one would expect for scales of the same length, and the close

comparability of coefficient alpha and alternate form correlations

goes contrary to Cattell's repeated claim that the items are arranged

in scales which are parallel across forms but internally hetero-

geneous. The equally close correspondence between coefficient

alpha and the factor pattern coefficients in the restricted factor

analysis (correlating 0.92 in the total group) adds to the evidence

for the value of internal consistency as a measure of scale adequacy.

In other words, both within-scale and between-forms measures

agree as to which scales are best. Given these results, the strong

similarity between the factor correlation matrices from the

restricted analyses and the scale inter correlation matrix after

multivariate correction for attenuation is perhaps more interesting

as confiimation of the utility of the latter procedure.

Page 162: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

162

The unrestricted factor analyses proved in the end rather

unproductive. As expected, splitting the sample by sex and

discipline to decrease the effects of correlated error made rotation

easier and yielded a somewhat better approximation to the model

under investigation, but the location of the source of collapse of the

factor intercorrelation matrices when targeted rotation was

attempted proved troublesome. However, the strong similarity of

results from a variety of procedures suggested that given relatively

reliable variables the controversy over factoring methods, criteria

for retention of factors and rotational techniques is sterile.

The use of restricted factor analysis was more informative,

and so far as is known this was the first time the method had been

applied to Cattell's scales. The extravagance in computer resources

of the program used prevented an ideal sequence of analysis, but

nonetheless a consistent pattern of results emerged from all ten

analyses. Investigation of the rank of the factor correlation

matrices from these analyses pointed up some of the problems of

the numerical accuracy and idiosyncracies of particular

computational algorithms for matrix manipulation, even at 72-bit

precision. Having settled on a rank of 13, however, reanalysis of

the results of unrestricted analyses using targeted rotational methods

gave a structure close to that obtained from the restricted analyses.

Page 163: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

163

The restricted analyses proved rather insensitive to the

presence of correlated error variance, which was so problematic

in the unrestricted analyses. By far the largest source of residual

variance was the unreliability of individual scales; off-diagonal

residual covariances were generally small, with a very slight

concentration in some of the least reliable scales. The results of

these analyses were consistent and consistently interpretable in

very similar terms to those of the other analyses.

Perhaps the least expected finding was the pattern

demonstrated in Table 18, suggesting that the correlations between

factors are systematic, but not necessarily in a sense that makes

higher-order factoring an apposite move. There is a possible link

here with the failure of attempts to fit Cattell's second-order factor

model to the scale covariance matrices using J8reskog's program.

This pattern calls for investigation and replication on another

sample.

The results with regard to individual scales are better

treated out of their conventional order.

L and N find no support in this study. Their homogeneity

is poor, their alternate form reliability equally so, and independent

factors to.represent them consistently failed to emerge. From the

factor correlation matrices, L seems to be a mixture of E and C,

Page 164: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

164

whilst N is largely E, with in each case a large component of error

variance.

B can be largely ignored: as a short intelligence scale it

performed badly, as might be expected on such a selected sample.

M is another scale which is poor in terms of reliability,

although the form B scale is better than the form A in terms of

internal consistency. The restricted factor analyses placed an

independent factor on this pair of scales, but clearly they are

unsatisfactory and need more development.

C, 0 and Q4 have been mentioned together time after time.

At best only two linearly independent common factors could be found,

and even they are very highly correlated. A number of possible

reasons for this are available. Firstly, it may simply be that in

this sample the subjects really did have closely matched levels

of three separate traits. Possibly the distinctions implied in the

differing item content were not relevant to them. Again, these

scales may find their place in the diagnosis of disorders not

represented in the sample, or in the interpretation of scores of a

minority of individuals who score inconsistently on the three.

Perhaps the scores in fact reflect state rather than trait aspects of

anxiety as a consequence of the conditions of testing. Finally, if

one is unwilling to relinquish Cattell's attractive distinction between

Page 165: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

165

ego-strength, guilt-proneness and ergic tension, one may look to

improvements in the item content to reinstate them as separate

factors. The evidence of this study is, however, that they are all

virtually measures of the same thing.

G and Q3 may also be treated as a pair. They are distinct,

but factor G form A is rather closer to factor Q3 , which made for

problems in the rotation of unrestricted factor solutions. Internally,

the scales are fairly homogeneous, and there is no real problem in

accepting them as potentially useful scales which doubtless could

benefit from more development.

I was the least troublesome of all the factors: it appeared

consistently in all the factor analyses and is adequately reliable

for its length. It has been little discussed because there is little

to say about it. From the item content one might suspect it to be

a measure of aesthetic interest, but otherwise it is unremarkable.

A is remarkable for the homogeneity of its content. All

but four of its items are similar in format, and a factor

corresponding to the scales is apparent in every factor analysis.

The distinction implicit in its content appears to be between seeking

and avoiding situations which necessitate social contact.

The isolation of E in the unrestricted factor analyses was

made difficult by the closeness of the N scales. Once this problem

Page 166: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

166

had been eliminated it emerged clearly. The scales are adequately

reliable, although a large number of their items correlate better

with other scales.

F is perhaps closest to traditional ideas of extraversion in

content. In the unrestricted analyses it was quite a small factor,

but this may well be due to the presence of many similar items in

H. At 26-item length its coefficient alpha is 0.81, which is far

from poor.

H must rank as the most remarkable scale of all. Its total

of 26 items reach an alpha of 0.90, which is in excess of many

cognitive tests of similar length, yet within the scale are two

subsets of items which are distinct in their content.

Q1 has only 7 items which clearly correlate best with their

own scale, of which 6 fit the notion of social and political

radicalism. Whilst not one of the most reliable of scales, it

figured consistently in all the analyses.

Qz was most nearly related to A, was only moderately

reliable and only half its items correlated best with their own scale.

Nonetheless again it had a consistent and predictable place in all

the correlational analyses.

Page 167: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

167

The title of this thesis refers to re-examination and

attempted replication of the structure of the 16PF. With regard to

replication, there has been partial success, but an examination of

internal scale statistics and of both factor correlation matrices and

the matrix of correlations between the scales after correction for

attenuation suggested that neither Cattell's claims for the internal

structure of his scales nor his account of their interrelationships

were well supported. Since in over thirty years development he

has failed to produce scales which are internally heterogeneous yet

reliable, and since the internal consistency of the scales proved to

be a good measure of their performance throughout, the way now

seems clear for the further development of the scales with the

emphasis on homogeneity of content. At the same time, there is

scope for the incorporation of more recent theoretical notions as

to the sources of differences in observed behaviour across and

within situations without discarding the insights Cattell has provided.

Page 168: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

168

APPENDIX

OF

TABLES

Page 169: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

169

...... Ln In T- '- NO Tr so r.., so rp - N. uo so CI-. ..-- CO r, ,- r... co co Tr r, bo co o- CN PO PI Cs

Tr rn -- r, so 07 PO CO CO s- M, sO PO rl -- so ro -- DO Cr. OD s- ro co co co Tr co so 'N -.4- r, CO

CD s- M sO Cr s- s- PI s- PO s- s- v0 CP s- sr CP s- CP sO CP s- CO sr s- CA CP s- v0 CP CA Tr r,

. • . • • • • • • • • . . . • . • • • • . • . • • • • . . . • • CA n n CP Cr Cr CA CP CA CP CP CA ISA MCA .0. MCA M M M M n M M M n In M CA M M

1111111 1 1 1 I 1 1 I 1 1 t I I I

CA sr 0- s- CA CO Tr N v3 CA "4" Cs Cs PO CP -C PO M, -0 M7 -0 '- Co 1"... D7 -C! CO MI 4 Tr O. OD

MI CO PO CA PO CO r.., Tr CO r, Tr -4- N. CA CA s- sO PO PO PO CP CP PO CO v0 CA CP sO s- CP CO Tr 'AD

cn -- CO N) .- CO til s- s- s- CP CO PO s- CO CP sr CO CO Tr -- r.4 Tr N CO s- CA N) Tr -- -- vD sr • • •• • . • • • • • . • • • • ••••••••••••••••

CA CP M M M CP CA CP CP CP CP CP CA CP .,- CA CP CP CA CP CA CP CP CO CO CO M CA CO CA M IS, I 1 1 I I 1 I 1 1 1 I I 1 1 1 I 1 I 1 I

CA v0 CO s- sO s- CO s- CO CO Tr CA -0 N 1.10 PO CO CD r, uo NO Cs CA PO MO s- v0 s- N. M .”44. ro

CA s- PO N. 0, CA CO CA MI sr PO C4 M CS! ND C.4 ..- M, sr OD CA -0 ,0 s- sr sO. St n Is.. 4 CO sr PO

CA PO CO CO CP Tr Cr ro -- m CO -- -- CO or CO -- CJ CO CO CO PO CO MO CP CP s- CO CO CP PO CP CO ••••••••..•.• • • • • • • • • • II • •

•• • • • • • •

M CO CO CO CO CA CP CO CO CO CO CO CA MI CP CO CO CA CA CO CO CO CO CO CO CO CO CO CO CO CP CO

1 11111 I 1 I I I I I I 1 I

r, PO N. PO v0 CO Cs s- IN N. CA s- Cs sO Cs M, CO CA co Tr rpPO Tr rP ro -- bo Tr cr. rq bo --

-- -- -- CO PO CA M, bl CA CN 'qt. M r, CO CP CA CA s- MO ,0 s- CA NO -0 PO v0 N. 0, Tr Tr co rp Tr

CA s- CO CO NI CA s- s- s- s- CA •- s- Os Cr s- (54 CP CO CO PO CA (N CP CA s- CA ,- s- Tr rp m CO • • • • . . • • • •• • 4 •• • •• • •••••••••• •••

M M M MI CA CP CA CA Cr CA M4 CO (5 M M CA CA n CP CP CP CP CA CP CO CP CO CA CO CP CO CP

I 1 I I I 1 I I I I I I I I

vD OD PO CA .-- DO N. Os PO Os v7 R. s- CA Cs s- M... PO N. 147 1.11 CO Cs Os PO v7 CO s- M, 4 Cs PO

PO CA CA sr CP PO PO CP CA s- 140 N. CO N Ln ON CO CA sO M, PO s- CO Tr Tr ,- 0, 01, IS! in 1,....

CD CA CP -0 CA CA s- sr s- CA c-.4 CA CA s- s- PO sO s- CO r, .-. -- CO 1,-.1 .._. c.4 .... „..- -O CO in ,r in • • . • • • • . • • • • 4 •• • • • • •••••••••••••

CO C4 n CP CP CP Cr CP CP CA CP s- CP Cr CP CP (54 CP Cr CO CO CP CO CO CO CO n Cr CO CO CO CO :I

1 1 1 1 I 1 1 1 I I 1 1 1 1 1 1 1 1 1 1 I CI ,

Cs PO CA C.1 M, Cs M, 4 CP Tr v7 vD CA Tr Tr rp so rq cr r, rp cr. -- -- rq ..-- r. cr. c, -- c, or. Cr'

r.4 CA CD Tr Tr bo so rP cr. Cr cr. -- CO rp Tr so so m ro co r, PO s- CP s- 00 "CI Cs CP r, Cr co

z m rP -- vo Tr rP PO s- CP CP Cs CA s- s- CP s- CP CP s- s- MO s- ro -- rp ,-- IN ..-- rP ..-- rp CS? u-1 • • • • • •• • • •• • • •• • ••• • • ••••••••••• I..)

M M M M M In en In n M M In M M M CO CO CO CO CO M CO IS, CP CO CO CP CO CO CO Cr CP

I I I I 1 1 I I I I I I I I 1 I Ili;

v0 CO sr vD CO s- -0 0- M1 sO 4 0, M, CP CA sO sO ,-- CO Cs IJO .- M, N. CO CO N. v0 sO CO N) Tr al

rp Ul CA M7 s- sr CO Cs (N Tr rP c..1 4 ro Tr N. -- -Cl co .,-- Tr bo co CO CO s- sr CA Tr -- so Cr- --1

2: CO CO CP CP CA CA CO CO s- .-- CO m .,--- Cyr m --- ..-- I:n Tr IS, -- ro -- rP rP !Ti • . • • • • • • • • • • • •• • •• 0 .1.111 a m • • , • • . • .

(St 16.4 03:1 CP CA n CO CP CO .- CP CA m (54 rp CO CSI Cr CA CO CO Cr CO m M M In CO CO CO IS, M

1 I 1 I I 1 I I 1 I I 1 1 5.. 0

CO CA MI PI r. rr m on uo m Cr NI r, co so IN -- uo ,-- 'IP os CA CP CA s- CP PO Cr sO CP M, sC1 4-

4 co ...- 1.0 PO Tr CO cr bo CA 0- rp 0, 4 r, -- CO CO CO Tr -- 0- uo -- cr. so -- cr CA C.7 PO CO

-.I CO .,- CA CA CA CO CO CO Cs CO CO CA s- CO s- PO M s- PO .- PI CO Cr CP Cl CO Cl ..-- .- CO ...-- ro :...; • • . • • • • • • . • . • • • • •• •• • ••••••••••• 4,4

CO CO CO CO MI CA CP CO n CP CP 15) CP CA CO SA (5) CO In MI CA CP CA CP CP CA m m CS 'Si rp CO r.- 1 I I i I 1 1 1 I 1 i I I I

al Sr sr r. so r, PO PO PO s- Cs Cs Tr os ..... ,--. CA vD 4 r, M, CO s- Cs v0 41 ro uo c, ro rP Tr cr ro r

co co cr. CA s- -0 N) ...- Os Cs CO N) CO CO co co -- so so CA N) sr s- s- '4' Tr CA s- PO Os Cs CA

Z 1-4 CP 0, CS! Is, CA pA ,- ,-- CO (N Tr CO CA CO CA s- '5, .-• CA W

I-. ••••••••••••••• •• • ••••••••••••• u 0 M C4 CP CP CP CP CP CP CP CP CP n n n m m m m M CA CP CP Sr CP CCP CP CA Cr n M M Lt. I 1 I 1 I 1 I I I I I 1 I I 1 I it

•...4

PI CA -.0 CP CO Ul CO PO CO sO N. N. Cs CO sr sr s- CO Cs M, -0 s- N bo Tr ro v0 CA s- N. Ols sr

0, sr PI sO M, Cs "4" NO CO CO sO Ul 121 CA '4' OD PO r, CP Tr cr. CA CP -AD PI PO r, ,-- -.0 M, cl Tr Ir., 2 NI CP • • r bl vD CP s • CP s- rte) 7 s- ro D N C T s v7 CP Cs - CP s- ' C PO A CA > • •• • •• • • •••••••• • • •ft 0

CO CA CO CO CP CA s- CP CO CO CO CO CP CO CP CO CP CO CP CO CA CO Si CO CO CO CO CO rP CO CO rp L.) I 1 I I I I 1 I 1 ! ! ! I

Cs Cs vD CP CA 141 UO ro r. -- cr. uo co -- rp so r, cr. ro r, -- rP -- vD CP CO PO CP M, Tr rp ro el

CN In t10 ..... CO CA Cs -01 4 sr PO CP M, CO CA PO v7 CO CO CO CA r.4 ro Tr -- cr. No -- -- bo co co

CD s- CP CO "- "- Cs CA ACP s- in s- s- CP L7— CO s- CA s- ,- 117 CP C4 CA CA CA s- c...! .,-. Tr -- 0 • . • • . • • • • • • • • •• • ••••••••••••••••

CO CO CO CO CO CO CO CO CO M m n Cr M CO CO CO Cr CO CO CP CA CO CI CO CO CO CO CP CP CO M .0 I 1 I 11 I I 1 1 I I 1 I 1 I I 1 1 I I

(!..15

0, r, ro L")--CS CO bo r. CO r. .-- ,C1 vD CA s- PI CA M, CP Cs PO s- OD Cs CA sr r, Tr CO so --- cl

so co Tr co r\ CO rs. s- PO s- Tr Tr c.4 CA CO 00 .- CA Cs s- -0 .- s- s- sr PO Ul PO CA Cs Tr N) GL. CA CO CA Tr -- -- -0 .- c..! -- Tr IN CA sr CO s- VO CO s- ro CO CA v0 CA CO CO CA CA s- PO Cr CO s-

••••••••••••• ••• •••••••••••••• • ■ el'

M Cl CO CA s- CO CA CP CO CO CO CO CO CO CP CO CO CO CO CO CO CO Cl CO CO CO CO CP CO CO CO CO I I ! I 1 I I 1 I 1 I 1 I 1 1 W

,-1

sr v0 v0 CO PO CP n M, PO v0 ro rp bo -- -- -- so rp ..-- uo CA OD PO PO OD N. CO CP% CA M, DO Tr z

r. rp Tr Tr co ..-- so CA PO PI Tr CA !JO Cs PO sO Cr- PO s- ro co uo Tr -- -- N 1:1°' 0". '40 MI 4'.. N 12

GU CO CO s- CO Tr ,..- uo -- CA -- ro r..1 ro CO -- rp CO m -r- Tr ;Jo --- Tr -- CA CA N) -- -- -- t54 rp I.- • • • • • • •• • . • • • •• • • • • •••••••••• • ••

CP CA CP CO CO CO CO CP CO CO CP CO CO CO CP CO CO CO CO CO CO CO CO CO CO CO CO CP CO CO CO CO I 1 ! I I I I ! I ! I I I I

(N sr Os v0 PI v0 v0 -0 PI Tr c..1 PO r, CO Cs Ul CO Tr co CO Os CA sr PO on 00 Tr r4 N. Tr uo CO

O- ro uo Tr Tr ro ro cr. -- (N co CA co r, CJ r, .,--- Cl- No 'C Cr- (N -.-- so -- -- r...! uo CA CO PO PI

Lj ,- CO CO ,- o...1 CO 4 ,-- N ,- ,- sO CO CO M7 sO ..-- CO -0 4,- s-- CO PI CA 1n CO -- -o CO ro r0 ,CI • • • a e • oe • • • • • a I • • • • I. a le v a • • • • a • ■ • •

CO CO .- CO CO CA CO CO CO CO CO CO CO Cr CP CP CA CO CO Cr CO CO CP CO CO CO CO CO CO M CO M 1 I 1 1 1 I I I I 1 1 I I I

CA s- sr -0 M, Cs CA M, CA CO PI OD PO v0 Tr bo so -- -- -- CA sO CN IN '10 IN rte) Tr Tr bo PO CO

IJO sr N) CO CO PO Tr co co uo CA PO ,-- PO N) s- CO M, CO s- PO CO s- CA -0 s- CA CA MI MI CA PO

04 CO CO Cr CO CP CO CO s- s- s- '5) CO Cl CP CA CO CO PO Cr CO s- CO CO s- CO s- CO CP CO CP CP CP • • •••• •• • • • • •• • •• • • • • •• ••• •• • ••

CO s- CO CO CO CP CO CO CO Cr CO CA CO CP CO CO CO CO CO CO CO CO CO CO CO CO Cr CP CP CP CO M

I 11111 1 I I I I I I

NI 04 CA sr Os ON N) Tr CU -0 Os -0 N CA CA s- CO '54 CO PO r. ro No CO CO CO NI CO s- 1.11 N. rte)

CO 1.11 Cs M, v0 Cs Cs CO sr CO CA s- s- ,-CO PI ON CO In 1..0 1'0 Tr rq co rp VO PO CP Cs v0 v0 CO

SE CO CO s- CO CA s- PO CO CO CA CO CA s- PO s- s- Tr CO r.4 CO c...1 r.4 Tr CO -- (N -- N CO ro (N Cr

. • • . • • • . • . . • • . . • . • . • • • • • . • • • • • • •

s- CA CO CO CA CA CA CO CO CO CO CO CO CO CO M CO CO CO CO CO CO CO CO CO CO MA CA CO CO CO CO I I 1 1 1 1 I I I I 1 1 I 1 I I

SE 04

1: s- CA PI Tr c CA ro sr 5- SE Co GA WU WI. CO AC .J AC At CA Cy cb CA CU G. .cr Co GA UW CO m ...I AM Mt CA CA CI CA CA 0 0 Li Ca. •

Page 170: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

170

MO CA CA 4- ,- MO 4- MI sO 0- If) ,- MO CO Os Tr -.0 r, co ro N Tr i co co Ch Ch N. N. TV .r....

Tr co ro ro r, ro co Tr m co co r, TV VO 'CI M C..1 CO il CA NO N N. NO TV TV TV MI r, r, ro

cn n ' so m m -- IN rq PO CA if) CA CA Tr r, is, el n -- ,- ro rl r.., is, CA sO CA +- Tr m • • • • • . . • . • • . . • . • 11 •••••1111110••••

M M M CP CA n CA CA CA CA CA CA CP CA CA CA CA CA CA CA CA CA CA CA CA CA CA CA CA ,-- I I I I I I I I I I I I 1

r, vo in In sO CA OS OS N. CS Ch In ,- OS l'•0 'CI NO M C-.1 1.0 N. Tr N. r, ,- N. Tr 0' Tr

PI sO CA In ,-. Tr OD CA OS 10 M In CA 4 4- M, MO ro to -- ,- 0, No CA In CA M, PI M, r,

a CA CA ro mi m Tr N -- ,- CA m Tr is, n ...4o Tr m n n --..Tr N -- IS, -- c...1 Tr n -- n Tr • . • • . . . . • • • • • • • le • • • • • • • • • • • la a

tS7 *A CA CA CA 'Si'SiCA CA CP CA CA CA CA CA CA 'Si CA CA CA 'Si CA S7 Cl CA CA Co-! ,- is, I 1 1 I 1 1 1 I 1 1 I I I I

in In Tr r, n Tr N. Tr CA -- Tr rq n Tr I17 so -- so 00 •-- cs .,-- rl m m ro m -- Tr r,

CA -.0 PI M ro cr. In r, los to r, n co OD CA Tr Tr m in co so cr. Tr OD -0 Tr Tr cl cq ro r,

CA m) is, mo ..- 10 .,- in is, in .-- mo CA In ,- c...1 Ni m N1 CA Tr m In CA SSA CA CA CA ..-- ..- ,--

. . • • . . . . • . • • . • • •••••••• 11••••••

M M M M is CS! M MI M CS M MI M M is, M MI MI MI M n MI is, MI iSA MI MI IS) MI .h. M MI I I I I I I I I I I I I I I I I

MO Tr CO 4- In Cs LID M, N. CO In s) CO CO Co CA U7 MO CA MO CA VI 10Os M -0 Cs CA M,

MI CA Cs In PO Tr so -- cr. n so co ,-- m CA CO CA CA CO ,- Os CA Tr -- so Tr in

z -- iA.4.- NO CA CA CA CA St CA v... ...- CA MO .,.... is, +- CA C4 4" CA C7 is, ,- PI - ..- CA rl (N

' • . . . • • • • • • • • • • • . . . • • . • • • • • • .

is, 5A is, Is, n Co rsl is, n n is, m is is, in is, Co Co n Co i is, is, is, is, is, -- m is, is, n i 1 1 I I 1 1 1 I I I I I I I I I I

r34 n. CO I''', r..1 in n co -- so -- -- so In CO CA sO Cr Co ro Cr- +- s0 CA Cr, LID MI CA N.

ro -- is, in Cr M Tr so -- co Tr r, m is, Tr so MO NI Cr- 0- ,r co n ,- -.0 is, mo (N so r..1

C CA is, CA CA CA CA Tr ,-- ,... (N .r.- N m CA is, CA CA CA .-- 15) CA CA 0- MCA Tr is . II • • ft• 0 • • • • • 0 ••••••••• •10 •• • •

,-

Co M M M Cal is, CP CA CP Co CA CA CA CA CA CA CA CA CA CA CA Co CP Co CA Co CA Co Co

I I I I I I is,

1 1 I I

CA sO Os CO Cr- toTr M—OOrAMMM In M, N U7 MO In In CO Cr- MI ,--- -40 141 MO NO Co CS CO

'Si NO ,... T... Tr ro Tr 0-. CO -- Tr so so N co so Tr CA U7 Cr- In sO 4- ,- ,--- Cr- LID In CA MO -0

....I ,- isl rel (MI CA n M N CA Co CA ,- CA *- CA in CA MO CA CA CA CA Cl Os CA ,- CA CA CA Co CA

. • . • • . • . • • • • • • • • . . • • . • ••••••••

M M n CA CA CA CA CA CA CA CA 's CA CA CA CA CA CA CA CA is, CA CA CA CA CA CP CP CA CA CA

I I I I I I I I I I I I I I 1 I

P4 'Si M, MO in CO in In CA =- Cr- is, ro r, Tr Tr 0) MO CA Cr. N N CA PO +- MO n ,- N Tr rt _1

co N so -- — so ...- -- CTt m 0) in Tr so co IN cr. N Tr Co OD CA MO Tr is, -- h.) T... CO NO N

V. hi M — CA ,- CA -,- Tr n T..... T. r..1 M M r.... M n r..1 .r. c..! m.,: rl co m; CA CA CA S4 S4 ,- CA 5- • • • • • . • • • • . . . 11••• " ••• si • 11

CI CP CA CA CA CA CSA CA CP CA CA S4 CP in CA in CA is, CA is, is,, CA CA in CA M is, CA in CA 54 CA CA 14. I I I I i I I I I I i I I I I I

-0 cr. Tr ro ,- ,- r, so m r, ,- cr. Tr n co co n so R. ,-. in r, Tr O. OS M CO ,... .1.--.. (N.. 4"

CA ,-- ,- Tr -- ro LT.4 -- ho co ...... -- so -- Co. CO CA CA CA -0 -- sO -0 CA sO CD CO -0 ,-- Tr os r, S Tr Cl If) 4 -O CA Cs C4 CA ,- N) in is ro IN Tr ro m U1 MD -0 CA ,-. N CA CA Co In Co In CA PO • • • • • • • • • • • • • • • • • • . • • • • . • • II • •

M M M M M M is, n CO M M In CA CA M n M S7 CA CA CA CA ,-- S4 In is, S7 n CA Cal M n

! I I I I I ! 1 1 1 ! 1 !

M sO CA CO In CP -- Cs CA -=-- Cr- C.74 MO Cs ,- 00 rq CO In n MO N. M, r.... co PO ro CO CA Cr- In CA

Tr is, rl in -- N rq Tr Cr- in ro ro so -CI ro ro -- so Cr- uo rl Tr -0 co 10 .1- N so Lo ....- ,0

CD CA CA Co ,-. CA in S4 CA CA CA .-- CA CA Co 4- CA .=.- CA Co ,-- CA CA CA CA CA ,- CA is, ,- is 4- =-

•••••••• • . • • . • • • • • . • • • . . . • . • • . •

CA 'Si CP CA CA CP CA n CA M CA CA CA M CA CA CA S4 CA r4 is, -r- m CP CA CA CA CA S4 CA CA In I I I I I I I I I I I I I I I I I I

N CA Cr- N 0, — -0 T... Cr- 1.0 MI 10 is, l'71 NO N. MI Tr Cl C..1 .■-• N') If) 0, In i54 N Cm. int ..-- CA MO

MI MO Cr- CO• sO CA Cr- MO — Tr N N. Cl '0 M IT Ir..1 TV NO MI N N T- rA Cr- Cl- CA CA CA sO ,- 04

1.1... CA ,- .=-- in CO ,- sO ,- ro n PO .-- CA ro N -- N n -- Tr -- N so CA CA CA 4- CA -- Tr -- ,- • . • • . . • . • • • • • 11 • •men• •••••• •••••

M M M M M n CP CA M CA CA M in CA CA SA CP CA CA CP +- CA CA CA CA CA CA CP CP MI 'Si Co I I I I I I I I I ! I I I 1 I

LID ,,- OD U7 is, N. r, co m Cr- r, ro Tr h7I rei T-. T.'. n Ch CO CA n — is, In 4- N) 0- 4 CO Co CO

MI -- sO MI ,- Co Tr rl Tr. -- Co sO ,- CA n CO C"- KA r, N Co LO '0 4- if) Cr- OD Tr uo co N-.1 In

La CA CA ...- Tr ro ,-- Tr ro Co -- M Cg, Co CA M, Tr — ro -- CA 04 CA ,-- CA Cc:! CA CA • • ••••••• • 1.• •all• ••••••••••••• I. • •

CA CA CA 15) CA CA CA CA CA M CA is, CP CA CA CA CA is, Co Co CA 15 CA CA CA Co is. Cn CA is, is, CV

I I I i I 1 I I I I I I !

M ...- CO ..- N VI Cr- M, .-- CO OS N 1.0 R. -0 r, r, ro n Cr- n in r..,.. ro ro so uo r, CO so Cr. r,

uo co so -- cr. Co Co -0 Co CO MI Cl sO CO In Cl- 4 CA CA N '0 0,.. CA CA CA N) C.4 Ig

If) 7.1E

(-1 CA CA sO ,- ,- CA 4 N ro n -- r, is, m Tr so -- Co c..1 Co .r- is, in N ro m C•.1 Tt

CO sO CA sO MO r, Tr so so r, CO ro ro m r, n Tr co co co uo so No Tr Cr- Co Cr- so Co +- sO Cs M, MO Tr OS is, CA NO NO ro Tr ro •• q" Co PI Co M CA CP +- Co 04 CA +- CO Co Co +- ro is, Co n m is, ro 'Si n •••••• •• • • • 11•• • • • • . • • • •

/Si MI 131 /S1 tSi CSl CA 'SI VA CSI ISI CS! CAC in ISt SI In f In in in in Cl in CA MI

I I 1 1 1 1 1

1.14

RA MO 4 C CA MI Tr 04 GJ CO MC hi -J MIC Mt CD CZ CM CM CM <r 04 WU CD MC I-4 -J MC NC CD CN CM CM CM

0 0 U- U-

6.0

48 -0

.249

• . • • ••• •• •••••• • • •••••••• •ft • ••

Co Co CP CA CA Co Co CA CA Co CA CA CA CA CA Co CA Co +- Co 'Si Co CP CA CA CA CA CA Co CA I I I I I I III I I

eA ,-. Tr Co N Cr- CA N LID ,- CA In N CO M, CA Co CA MI CA Tr CO -0 CO In CA CA ,-- ,0 ,ct

Co M, Cr- MO CA CA r, sO M sO CA *A M 4 In In +- .1- CA Co Tr -Cu M Cs 4 M) CA CA MO CO

04 CA MO Co Co Co ,- CG is. — m Co Co CA m is, it m CO m Co is. m m Co n el ,- n m n • • • • • • ■ . . . . . • • • • . • • ••• • • • • ••

M Co M M. M M M MI CA CP CA CA CA CP Co CA CD CA CA CP CA CA Co CA CA CP CA I 1 1 ! ! I 1 I I I I I I I I

- 4 M, CA 4- M, +- CA sO Os N. Cr- M, 4- Cs In Cs 0) Tr is, N ..-- -- so ro Cr. M uo m r, r,

-- Cr. ro cl sO CA +- s0 MI CA Tt S4 Cs Tr Tr m N ro N Tr In CA hD - .,-- In CA sO CA rl r, Co

CN CA CA CA ,- +- CA CA w- .- M CA MI 4 CP •- CA M Co CA CA — +- is, CP 'Si Tr CA Cr- CA M4 is, • . . . . • • . . • • • . . • . • • • • • • • . • . • • • •

CA CA CA CA CA CP CA CP CP Co CP CA M CP CP CA CA Co MI CA CA CA CP CA CA CA CA CA CA CA I I 1 I I I I I I I I I I

OD Tr 'N Cr- r, Co C4 MO CA -40 Cr- +- Tr -- vo cr. oo Tr r, cr. cr. co co m ro In sO MO MO =-- 0-.

CA CA U7 CS r.0 .,.. T... T-.. ID- CA Os Cs Tr r, -- so -- cr. Tr Tr n so so uo in ro -- Tr Tr r.., Tr

C) el CA -0 ..-- CA - Tr CA ,... ,- -- CA Tr so -- is r, -- r..1 n In N r...1 n -- Cn is, N Tr so • • • . . . • • • • • . • • . . . . • . • • . • • . • • • • • a

CA CA CA S4 CA CA CA CA CP CA CA CP CA CA CA CP CA 'S, CA CA Co Co CA CP CA M CA CA is, CA CA _, 1 I I I I I I I I I I I I I I I I I I 0

Cr,

Page 171: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

171

CA Lil -0 In 0- 1.11 SO 111 CA a- No ro No cs os n cl z In ..40 ,- ,- CN 04 ,r ,_ ,r ,_ -0 ,_ (N ..40

a co -- NO Cs ,- sO 4.- CS SO CI sO PI IN r, L....14 Do CO CA sO r, ,- CA CO M PO r4 ,- ,- ,- IN a a

CN M M 10 M 'Si +- NO 'Si CI .-- n In M 'Si M CO 'Si cc, "zr r- ,- ,-- SI M (-1 ,-. ,- Do n n P0 In

•••••••••• ■ •••.• • • • • . • • • . • . . • • . •

M 'Si M M 'Si M Si M M M CD CD CD CD CD CD CD cc. CD CD CD CD CD CD cc, CD 'Si CD CD CD CD CD

I I I I I I I I I I I 1IIIIIII I 1 I I

CO M ON ON PO ,- n CO r, a- -- so ,- co CN ON M *qt. ,....': CO 00 M N. In a r, PO M 0-, Cr.. ,-. ro

PI -0 NO CA CO3 Cs CO sO *- a- co ro m cs ,-- 'Si n Do r, 'Si CA N 'Si co so cs m ,- CA -0 In IN CA

cn n m r..1 m m a- .- ...- ,- 'Si m ro m m cis ro 'Si m CA cc. ,- 4 *- 'Si 'Si ,-- CA r.".1 n M *4 NI

1111•••••••••••••• •••• • • ••• • ••• • ft 1111

CA 'Si 'Si 'Si CD CD CD CD CD 'Si r CD 'Si CD CD CD CD CA CA CA M CA M CA MI CA M 'Si fA n 'Si 'Si I 1 1 i 1 I 1 I 1 1 i 1 i 1 i i I

-gr r, 0- a- c...1 a- co ro so m co co uo r, co cs a co r, 0- -0 ro a m NI a Cs a- In '- N) -0

CA CO 4 'Si CA C4 -0 M ,-. sO NI a- Si CA NI ..-- N. r, a a CA In 'i" 'Si r, 'Si 'Si a n CA CO CA ts0 Cr, ro m m ,- PI ,- PO M MI M ,- M M M M 'Si NI M cc, n CA ,- CA M M 'Si *- M M NO M n • • •• • • • • • ••• 11••• ••• • ••• ••••• ••••

M MI fA CA M 'Si M M CA M CA M M +- CA M M CA M M 'S M SI M M M CA M M M M M

I 1 I 1 I 1 1 1 I 1 1 1 i i 1 1 1

a- CA f', LO LO -0 -0 N +- +- a- n r, uo ,- -0 cs '0 so -0 NI 0- ro PO r4 ,-. N. n -0 cs. co ro

.-- m vo so so a In CA CO ON M r, -0 a- CJ cs r, 10 CA NO In OS CD 'Si -0 Cul In NO NO ,-- T- CA D1

CD CA M CA PO rl CA CA M ,- *- *- *- M M SI M M 'Si M NI ,- *- CA ,- c...1 ,-. --- ,- PO CA .- M

. • • . • • • • • • • . • • • • . • • • • . • • • • • . . • • . M M M M M M M M M M CA M +- CA M M M M M SI M SI M CA CD CD CZ) 'Si CA M la M

1 I I I I i I i i I I i i I I

N) — LO 0- M T... CO In In a- uo r, m CO -0 NI CO r, 43 (N tom) a- -- so CO In -a Do 'Si cs. Do O- ✓- a- cs. ro CA IN CA P.1 '4- CO In In siD 'Si SI NI *- Ni IN CO OD 'AD sO M 'Si NI a- -0 a- PO 1-0 In

CO +- M a CA +- +- •q- +- +- +- +-- ON ..-. M in In T... M a- CA ,- M 4 *- CA ,- M In M ,- ND a ca • • . • . . • • • • • . • • • • • • . • . ••••••••• :1

'Si CD CD CD CD CD CA CA M MI M M M M MI M CA MI M M SI CA fA M CD M M M M *4 fA ST CI

1 11111 1 I 1 I 1! I I 1 1 I I I 1 r_ T,

-0 ..01, -40 0, NI -40 +- -40 ro a Cr- Do -a CD ,- sO ..-- NO Cr- M CO ' CA sO NI In 'Si NI CD CD In CN

-010 +- Ps PO .:4- 0, +- NO bl CO 10 IN -q- MI ,0 CO CO M 0- CO ‹r 0 0 M 1'0 r-, NO 0, 0, -I- sl al

Z Si CD +- CA l'n MI CA M +- +- IN +- +- +- MT MI +- M +- +- CA cc, CA CA Sc, .-- +- +- M Si fA M LI • • • • ••• • • ••••• •• •••• ••••• •• • • • • • r - n IS MI M 151 'Si M M 'Si CA *a la 'Si 'SI 'Si 'Si 'Si 151 M M MI M CA CA CA fA CA CA SSA SI 'S M 21

I I I 1 I I I I 1 I I I 1 1 I I •H Li

a- a- a as N. CA .- 4* ND N. a- a- ,-. n a- a- os IN so co ro cs (N rs CO ,- NO sO PO M CO sO In

rs r, m CA -.ID CA 10 '4- CO -.0 In CO la ro CO CA N, CA CA PO PO In CA N., s3 -0 In 0:1 1-0) -40 1.0 CO

1: CD CD .- =- CD ..-- ,- +- CD 13'.. ,-- ....- ..-- ra MI +- CA CA +- ,- CA M ,- 'Si ,- t*0 M n CA CA MT S4 al • • . • • • • • • . • • . . . • • • . . • • • • • . . • . . . • .-1 CA m n m m n m n 'Si n m n n n m m m n n 'Si 'Si 'Si m n m n n m 151 'Si 15) M M

I I I I I I I I I I I E

OD n 0, CS V' a- a c...1 ro N) ro Do ,-. so r, CA ,- IN sD M PI CI n CA ,- s0 .cr a, 4 -0 0, CO f••••

CO CA +- ro so os a Do Do co ro a- es so .nr so ro r, r, c...1 -- CA n M In n sO CA OD n 0- In 0

--I 'Si +- CA In N M +- M Cl- Si .4-- +- +- CA +- CA +- SA +- CA CA +.- CA +- ro cc, rq 151 ,- m m cl 4- • • • . . . • • . • . • . • • • • • e • • • • • e • • • • • a a

M CA CA CA M 'Si ra M 'Si CA n M cc, CA M M M M 'Si CA MI 'Si M 'Si M M M MI CA M MI M I I I i 1 I i i I i 1 1 I ! 1 •'-i

!..- <Y: MI -40 P.0 Os IN IN +- +- CA a- so NI CA V') 00 in SO OD MA NO N. .4D V.5.1 Ll PO -4.- CA CD sO Os IN CD 4-,

CA N -0 sO sO CA M 0- In a -- NO CO ..- ,-. 0-. NO N ,- In N) a- 0- 151 -0 CO if) a- PI In M -.0 M ✓ 1.4 M T.. T.. 'Si M T- T- NO MI T- M T- M M T.. M M M T- M M M MI 'q- r- r4 'Si c..4 n cc -- ,.... z ,.--. • . • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • O CA M M CA CA CA CA CA M MI MI 'Si MT M SI M 'Si 'Si 'Si n 'S. CA n MI CA 'Si MI CA M 151 M M a

1.1. I I 1 I 1 I I 1 1 1 1 I 1 1 1 I i u r-

a- co In ND IN N. CA ,- 4 ,- ,- CO --0 OD M sO M

a- a- SO Mi a cs cs m a- 10 0- CA CA *A sO +- T, Pa r411 r K r3 ri.';Pi q! ';,7 ;17. N. '43 8:", gr'. E-,!. •ri

=C In ...... ro 10 In M ON ..... T'. T- Cl a rq ro ,- ro ro -- CA a If) 'Si r-, ,- 'Si •-• cl ,r ,- N) ,- ,.... r- • • • — • • •••• •II• II • ••• •••• • ••••• • a 177.

M M 'Si M M 'Si M 'Si n M MA 'Si MI M MI SI 'Si M n M 'Si M n n 'Si M MA 151 M MI 'Si Sc, > I 1 1 I I 1 1 I 1 1 I I 0

ILI

CO +- -0 -0 O. 0, IN rs a N -0 -- so a -- 10 In -- a a- CA CO Si ...- NI ..-- If) rl sO ,- -0 sO

SO M 0, NO C.4 '4"' Os CA Cs. c...1 a- 1--, Do -0 co -o -- ro a- r, ,- Do N m r, co Do a- ro cs co ro • .

CO +- CD 'Si -.-- ,- 0, M +- 'Si +- M +- Cq +- '71- +- M M +- M CA in 'Si 'Si M ...... *- ,- ,- SI a CA al

. . • • • • • • • . • . . . • . • • • • • • •• • •• 11• • • • r-

M M :Si M lala 'Si 'Si 'Si M MI MA 'Si 'Si 'Si CA 'Si la M 'SI m m m m m M 151 ,SA 'Si 'Si M.! m F1 1 1 1 1 1 1 1 1 1 I I I I 1 i I i i I

CO 'Si ON T... CN ON N. N. a- N r.1 Mi r., N 1•10 CS SI r..., Ct.. ....., 0... 10.1 'Si NO r40 in ,r 00 0, Is, In ,:-.4 f-

0-. so ro a- ro N '4' -0 sO NO MCA a- (-1 cs ,- 0- ro 'Si co No r.., CA Si CA -0 Cs U0 CA In ,- 0- N

I.L. +- =- +- *4- 0- +- In n CA MA V, ....- CA PO M M CA *- M CA -0 ,- 4 CA ,- M ,-. ,- ,- ro ,- 'Si a • . • • • • • • • • • • • 4 • • • • • • • • • • • • • • • • • •

'S. 'Si M 'Si M 'Si CA CA CA 'Si M M M M M 'Si M 'Si M M 'Si 'Si M 'Si M M 'Si SI la la 5:, MI CA I 1 I 1 i i i i I I I 1 1 1 ! <1....

cs rl a- n ,- so ro os Cr- cs cs os Do a- OT Do In 10 IN *- CO NI In NI N 'T T.... rl .q" ..1' Cr. so al

r, ro c-1 co a- ro n so ro CA IN NI sO CA ON CN In M LO M In MA +- a- co r/ -- n so m n ro r-I

1.1.1 CD ...- ,- co a- ,- Do Salts *- CA CA PO ,- n MI N,- M 10 4* CA a- ,- .-- (-1 ro c-1 rl -- Si n 2D

. • . • . • • • • • • • • • . • . • • • • • . . . . • • • . • • ,-

'Si CD CD 'Si CD 'Si 'Si 'Si CD 'Si CA 'Si 'Si 'Si CA M 'Si 454 SSA 'Si M CA CA MI M M 'Si 'Si M SA SI SSA I-- I I I I I I I I I I I I I I 1

CA fA NO .V. Os NO In In cs a- so Do N 0.,.. 0" ..C., N. MI 454 PO ,... a- ro Do os so r, r, -- ,- so a

ro a -CI CA NI 0- -0 sO ,- 'Si *- Cs sO M c...1 ro a a r, cs ,-. co N) rl n '5, N) a ,- In 'Si +-

C.) +- CD 0.- +- +- cD In +- eq ,- .- a- m m CA In ,- m ,1- ,- cl n ,r ,- cA m ,- 0 'Si ,- rn ,r • . • • • • . . • • • • • • • • ••••••••••••• • ••

M 'Si M 'Si M 'Si 'Si 'Si CD CD 'Si CD 'Si 151 CD CD CD '51 'Si CD 'Si 'Si 'Si 'Si CD CD 'Si 'Si CD 'Si 'Si 151 I I I I I I I I I I I I I I

SSA a- 'Si rq m ,-. co so n a ,0 ,- CA N. MI In in CO rl CO ....... sO n MI CA a co -0 so 0- sO 'Si

r, ,- a- PO -40 ro .gr CA CA IN in a- ro -a M ....- Cs In .zr CO a- CA M M sO M CA M a- Do a- c-.4

0.4 CD CD CD ..-- +- CD =- ,-- ,..- M M M M M M M n CA M M ,- MI *- ,- M *- M ,-. M ..-- M M

11•••••••••••11•1111 Il• ••• •• •• •• It ••••

M T.... M M Si CA M CA CA M MI CA M M M M M M M CD CD CD CD CD CD CD CD 'Si CD CD CD CD

I I I I I I I I i i I 1 i I I I

— m CI ON 03 co a N) CO a- so ro a- a- co c..1 ro cs r, co ON ON 1.0 T.. CA -0 CA M CA sO -.0 MI

CO N hl IN 0- 43 a- rq OD rs -0 ,- m co 43 co a- m Si In ,ID r.,, Do ,-. -0 co tO 0 'Si ,C1 -.0 CO ¢ op 'Si -- m .— ,— ro m m 'Si 'Si -- m rl m m a- ,-. .- m c...1 ,- cq 'Si M 'Si '51 +- 'Si Cl 'Si 'Si

. • . • • • • •••••••q• 11•••••••••••••••

CR CD CD CD MD CD CD CN CA CD CD CD CD CD CD CD CD CD Si CD CD CD CD CD C4 CD CD CD CD CD Si CA I I I I I I I 1 1 1 1 1 1 i I

04

C4 NO .gt C CA PO r, 4C 04 C-1 W La- CO CC '-4 J AC 2C CD CD CD C11 CD C. .cr 04 Li W 1.11. CD CC 0.-4 —I AC 2C CD Cb CD CD CZ

W u.

Page 172: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

.A. PQ .,.... (Si if) CO -O ,- (Si LO N N. NO 'C' ro m ro c...1 Cr- (Si N -- CA -O .,--. CO CA cr un co cr N. u.-.1 -t-,

,-- m CA cr KA m cr (Si (Si r, un (Si No r. No (Si N 1 ,- (Si <V .,..-. ,- CA CO (A to CA NO CA NO 117. E I-I M 4.- .1.... ,.... N 'Si .1.- cr ,-. (Si 'Si ..... ,... M 'Si (Si M (Si M .,.... r.- 4.... Lei CO CA Cl (Si rl 'Si (Si (Si 'Si E S- • • • • • • • • • • • • • • • • • • • • • • • • • • •

O (Si M (Si (Si M (Si M M 'Si (Si (Si (Si (Si (Si M (A (Si (Si (Si (Si (Si (A to to rA (51 (Si (Si iii (Si 'Si (A W 1.4. I I I I I I I I I I I I I I I I 1J

C:

V) M ro In (Si (Si (Si (A (Si CA N CO cr N Cr- -0 .0 0.. TT .40 CA -0 Si( ,- In (A CA CO CA CA 65

10 'Si ro -- CA r. un Cr- m N un n m co co or co un (A cr cr CSI cr. -7 r, ,-- cr u) if) rA ;JO •,-1

2 CA ..-.. cr cr cr m r, KA m .--- IN 4 rl CA ,-- N) Ci 'Si ro cr if) m t5i m KA -- ,-- un .--- N) (N CA r- • • • • • . . • • • • • . • II • • • • • • • • • • • • • • fi7I

M 'Si (Si (A 'Si M 'Si (A Si 'Si (A to (A (Si (A to (Si 'Si (Si (Si (A (Si 'Si (Si (Si (Si (Si (Si (Si (A to > I I I I I I I I I I I I I I Cl

13

ON -O cr ro Cr- Co NO -0 PO ON <I- ON CO (A ,- <V ON ,-'T cr co CA -0 N) 1r (Si In (A CA ON CA

r. c...1 co m CA 10 NO cr rl un cr CO cr m CA (A CO Vs. NO NO (A *It ...- MI ..---. sO LI? N. ,- N.) r, • .

CD *- CA M rl ,-- un KA KA ,-- n ItT4 *- 1- cr .-- *- m M ,- CA M (A •-• ..-- MA v- M (A ,-- NO ,.. CI

. . • • . • • • • • . • • • • • V • • • 1 • • • • • • • • • • M VA IA M M M In M M M fA M M M M M MA M (A (A .- M M m m m M (A M (A (A ..I..'

I i I I I i I I I I I 11 I I

ON •-• CO Os ON CO CO co ro ro No co ,-- r, (Si m r..1 KA cr ND CA CO ...- ,- ....- In Os (A ,- 5-

sO ..-- LO -0 r, ,- NO CO Co Cr- Li) Ci ..- (N N. co cl ma -0 it' cr (-1 Cr- Cr- r, 0.„ (Si Mi -0 113

LI- N rq cr NO N If) N (A N ..--- .,..- CA ..- ,- P1 'Si (Si 1", -,..- CA LO .,---. ..--- (Si CA CA (A Ni -- 'Si Ci • . • . • • 1 • • 4 1 • • • • V • • II • • • • • • • • • •

(Si L5q (Si 'Si (Si 6i M M (Si (Si M (Si (Si (Si (Si (Si M M (Si ,- (Si 'Si M (Si (Si Si M M M n (Si (N 1 1 i 1 t I I I I I I I ,x

co C0 ro '- -- bl M UZi CO SI CA ND Cl-. CO -0 rA In ND CO CA Cr- ,-- ,- ..-- Cl-. CO CA CO In In N. al

LC) CP (Si CO N. CA Ki 0, CO ID rl CA r, -o If) os m N No (Si si (Si Mi Mi ND LO cr co c, 8-1

LU (Si to ....- In CA rya <V to CA .,-- -- CA PO (Si (Si ,- (A (A Cr- CO ...- cr -- CA CA CA CA CA (A 'Si (A .i0 • • • . • • • . • 4 • • • • 4..484 ... ••••ft• • •

it

M CA M M M to M tSi M M M M M M M M M M M (A to CA (A M M (A to (A (A M to to II- I I I I I I I I I I I

N. (A N. Cr- P1 -0 -0 Cr- NO sO N (Si In TT If) •-• ND (Si -- ON CA CO CA (Si .7 ,- ,- 4-4- m1

(Si N V71 (Si 44- r, CA (A N. r'0 cr KA No M4 (Si ,- Cr- CO r, Lr.! ...- ,.... 1.0 MI Cr- A0 CA ,- 10

C.) ..-- *It CA M N ...... ...- *- mei' m (Si N cr -- KA Cr- 'Si (Si m PO 'Si CA (1 ..-- '1' 'Si ••-- Mi <1-

172

(A -0 CA NO 00 OD NO ON ON PO PO -0 CI C.4 N. ,- CA CA LO 4,-- cr. cr O. cr (Si Cl-. (Si

cr Co CA PO ON PO ND In 00 'Si ro un M r4 cr to CA ON NO P, 10 ND .- ,- (A ND Os If) ON Ill

CN CA (A (A (A CA ,- CA 'Si 'Si <' 'Si a NI If) Si to to (A ..-- CA (A CA (Si CA </- (Si (Si PO CO

. . • . . • . . • • • II • • 0 • • II a • • • II • • V •

M to M M (Si a Si (Si M (Si (A 'Si to (A (A (Si (Si (Si (Si Si (Si (A 'Si 'Si (A (A (A to (A (A

I I I 1 I I I 1 I I I I I

sO 0. 0 No r, or co un --. IN No I", In (A ON CA r, OD h, NO CA NI cr o-. c, PO </- to Si ,.-- CO (A Cl-. If) cr N. <1' CA ,-- 00 CI NI to CA NO (A ND i- un rn (Si O. CN CAM (A ...- cr .--. KA (A (A cr ro m M (A .,-- NI CA (A (A (A ,- CO (A (A Os ro

. • • • II • • • • • • II • • • • • • • • • II • • II

M M (Si (A to Si( to (A 'Si to to CA (A (A (A 'Si Cii (Si (Si (Si (Si 'Si 'Si to (Si (A (Si (Si 'Si (Si

I I I I I I I I I I I I I I I

..0 ON .-- .cr M ,- OD Cr- `40 CO .1-■ 0' ''- CO O 6'7 ,.... 1.17 Os CA CI cr CO ro r, Cr- cr oN cr m..! N -O 10 If) Si( 10 ON NI In to NO Cr- OD If) N NO CA <1- (Si ..-- 10 NO CO CI cr 17.4 N. -- ro un

CN CA -..- ,-- ..-- PO (A ro m m m to PO (A (A to -,-- (A NI -- NO to (A ...-- to ..-- (A OD rA M • • • . • • • . • • . • • • . . . • • • • • . . . . • • .

(A M to (A to 'Si (A (Si (A 'Si (Si to to 'Si 'Si to Crai (A M (A to 'Si (A Si 'Si to 'Si (Si M 'Si (A to

I I I I I I I I I IIIII I

CA -O cr Cr- -Ci No No cr ro co un Cr- so rs7 'Si co un (Si IN co r, No (Si N r, cr ro cr

to cr VO ON I'D co ro IN Nc CSi No to Cr' Ps. LO N CO (A PO r, R.. N. 117 Cr,

C5 to (Si (A CA •-• (Si -- CA (Si (Si (A (Si to CA (Si (Si -- 'Si CO to CO (A (A 'Si

. • • • • • • . . • • • •

• • • • • • • • • • • • •

M (Si to to MfAMMfAMM fAMMfAMMfAnfAMM (Si

I I I I I I I I

to -O h, N CO CA NO CO ON NO N) (A <r Si .,...- ON ..414 1 Cl T... 10 to In ro un c, un CA ON CA ON

If)Col4 (Si LO cr -- cr rq co PO ro M CA .•-• NO CA NO r, (A <V (Si 'Si PO NO ,-- h, CA <V -0

CD -- ...- L17 r..1 -- .- cr CA (A ( -- ,- (Si CO LO ...- 5i CA CA (Si LO CA ..-- (Si 'Si ON (Si ..-- N) ,r G. • • . • . • • • . • • • . . • • • • . . • • . • . • . • ..-5

(5, m KA n m m m m m m m (Si m m SiK m KA m m KA KA KA m M (A (A (A 'Si (A 0

I 1 I I I I I I I I I I I I I I I I I 5- Cr.

ri 00 .r.-. cr In r. CA cr ro KA cr r. cr. ro cr -0,- tS co -- KA un cr CA NO (A ON (A h, NO <V

CI CA ,- Cr- In -,.- if) -0 10 N cr vo cr cr KA M un Cr- -0 -- (51 cr r, -- -0 N) cr so n al

z m m ro -- ,- rq m CA M -,- to ,- ...- c..1 ,- m m rl rq -- -- to -- -- co (A ,-- M ...- CA U

. . . • • • • . • • . • • • II • • • • • • II • • • • • • • • r-

M Si to M n M (Si M (Si M M M MI to M (Si 'Si 'Si to M M (Si 'Si (Si (5 MI (Si (Si (Si (Si M 'Si al

! 1 1 1 1 I I t i I 1 1 1 1 1 I 1 1 1 I } •r--1

cr No cr ro -- -- cr -0 -- un 1.1 -- mot' N ,- N. Cr- CA ON -- cr -- N -- co No 10 CO N. 0'

(A to CA NO OD <V CO (A sO ro ro un m m IN •-- z

ter' MI'Si (A (A (A CA (A is -- CA ,-- ON -- (A PO ..-- uA (A W

• • • • •••••••••• • • • • • • • • • • • • • • II • r-1

M M (Si 'Si (Si (Si M (Si n (Si (Si (Si M n (Si M M (Si ':Si M (Si M (Si to 'Si 'Si to (A (Si M M 177,

i i I I I I I I I E

CA ON ro No PO CO PO CO CA PO <V <V 'Si 1- CO+-MME CO co r, CA ro r, co co

-0 tSI co CA N. P, NO LO '0 (S 'Si CA m Mt

Ao (Si s.4 (5, (-1 ,r ,r(Si CO o, s,

J Cal m 04 ,- (A (Si CO -.- to N CA 'Si 'Si CA 'Si to N CA CA ON •-• •-- (A (A IN 4- • • • • • • • • • • • • • • • • • • • • • • • • • • • • •

(SI 'Si (Si (A to (A (A (Si (Si (A 'Si (Si to (Si (A (Si (A to (Si to (A eSi (A (A (Si (A (Si (Si (Si (Si Si

I I I I I I I I I I I I I

0 -0

. • • • • .. • • . • . • . • .

(Si to 'Si 'Si m CA (Si t5i (Si (Si (Si (Si (Si (Si (Si to (Si (A 'Si 'Si (Si 'SI M m 'Si 'Si (Si m m KA (Si KA I I I I I I I I I I I I

ON Co ro PO P,.. In CO P, CA CI r, 00 cr m r, ro un un m cN No m cr os -- ND M Lil IA, CA

to If) "I" (A CI M CA CA CO In .1- N CA CO In 'Si in r, ro CO -r N) ro 'Si CA -0 ro -- r..!

04 ..-- N (Si ....-- ,-..- en .-. 'Si M 'Si (Si 'Si M M) (Si OD (Si (Si M (Si M Si 'Si 'Si IS) Si 151 (Si M (A • • • • • • . • . • • • . . • . • • • ••••• • • ••••

(Si M M 'Si M (Si (Si M M M M 'Si (Si to (Si (Si (Si (A (A (Si M 'Si (Si M (Si (A (Si (Si (Si 'Si

I I I I 1 I I I I I I I I I I I I

NI If) N. In to M ,- ND ,-- Os •-• CO ',1" (Si r..! OD N. -,- (Si N. cr No 0, Si r, so Os (Si CO NO N)

.1- Cr- cr ro CN r, ro ro r, co -- r. In CO CA CO 'Si ND CA (Si Cr- CA .- CA cr so m No CA PO

<C•1-151 *--NN C5i POM ,-- CA ,- ..- Cii VI (Si (Si CO (Si ,- ,- PO .- CA (Si to 'Si M ...- ,- CA 151 (A • • • • • • . . • . • • • • • . • . • • • . • • • • .111.11

MI (Si M Mi MI M Si 'Si M 'Si M 6a 'Si 'Si In to 'Si In In (Si (Si (Si In 'Si fil M M (Si M (Si l' I I I I I I I I I I

• • • • • • • • • • •

po <r S-

<r 04 (-3 WU WI- CD MC 1-4 J AC Z CD Ci Cb Ci CD C- <E OA GJ WW LL CD 2 O 0

-J AC NC CD CN CZ CN CD

Page 173: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

173

un c-.4 N O. O. In M CO NI r, co -01 .,--. ro -- uo ,..-.. ro N) r, c...1 CO -- ON ON Cl. C.4 C,, r, rq -4- *-

*I" CA ON r4 CO T.- .4- -- c...1 ON N0 M M T- M CCI 1.17 147 PO V, cl ro b7 r, to 41 CA CO N PO -4- ,0 ....-

a ..-- m No m m m c...1 m PI ,- M b7 M .... VI 00 .4-- MA 1- tn M4 M CA MI rl M .4.- LO CD =,- ro 40

•••••••••••••••• 18••• • ••••••••••11 tn in M M M M n en IS4 In Ma M tn IS M tn M1 tn In In MI M In MI tS4 MI to IS4 tn MI, M tS4

I 1 1 1 1 1 I 1 1 1 1 1 1 1111 1 1

os r, r, n sO 41 VI 1.11 174 M CO PO •.4-- .q. c...1 ,-.. Cl' r, ,c1 IN ,o -- ON N rl -o .v.- C4 Cr. .t -1- N)

ro N. c1.-. r. so Lin so ,- t (Si .44.- sO ...CI CO b7 bl 00 ...4. N. in CT. PO .nr ro N) PI PI ro N. -0 to -1- to

C t! m N V4 rl .zt- m M4 V7.-- C4 CA Cl tn 0, M MA M ro -- cl PO M SI .4-- r4 V7 CA IN tn uo 1- • . . . • • • • . • • •1111•• •••• ■ •• •••• • ••••

M M M M M M M M M MI M M M M MI M MI M M M M MI tn M M M MI M Mi MA n n t 1 1 1 1 1 1 1 1 1 1 I 1 i 1 i

m ro .4- b7 M .cr N a- r, uo co -- c..1 Cr. ..4- ro on to 117 W1 Cr- '''' (171 bO b7 tn b7 .4-- VI -0 s0 -,4.-

N M ,..- .4.- CO N. VI sO PO V, MA N. n -4- ro uo n N. ro --- .a.. ro ,- Cr- so PO .71' M rq ro -o -- ro

a ro m ...- n C4 Mi r4 M Si M M .4.- tn Cr- MA .4.- r4 M .4...... M CA M .-- M ,- tn n -- tn -tt m n • • . • • • . • • • • • • . . . • . • • • • • • • . . • • . • •

m m m m MI tn M M M M M M M M M M M M M M MA MI M In tn MI n MA MI M tn M

11111 1 I 1 I 1 1 1 1 1 1

PO VI N .4.- .4- co -4.. n -- 1.41 Cr, 110 'T CJ .,-- ,- .4- m -- to si m No CO ON ON T- .q- r, .q- uo m

.q. VO VD 1.17 PO r, m CO uo -- uo .q. ..-- -4. o0 -- r, PI VI .4-- CA r, ro m PD tn b7 sO .q. c...1 cr CA

CI M M CD PO r..1 r4 r4 tn CA N 104 tn M MI Cl M M MA M ..t r4 N 44.- CA -4.-- CA .4-- MI In v- • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •••

M M M M M M M M M M M M .4.- MI M M M MA MI M M tn MA Si MI t51 MI M M M4 M M

1 1 1 1 I 1 1 1 1 1 1 1 1 1

OD tn MI sO 0- MA ("4 VI CO 0, 1.47 b7 Ll ,- VI sO r4 sO 04 b7 .4.-. r, cl .40 CO V, sO -0 IN 0, b7 CA

mm 1' 141 CO -.4. cs tn uo sO . t. rl -4- m -.4o n -40 (\I m r, OD .4-- PO M4 VO 1- sO M n VO rl VI

CO ....- .,-- lil SiC CD CD Cl M CA .4-- M CO M -- CJ 1.37 .4-. M .q- Si n n ro n c., m tn uo n -- vo .tr a • • • • . • . • • • • . • . • • • • • • • . • • ••• ••• •• 7.; M M n M n tn n M M MI M M M M MI MA M M MA MI MI MA tn In Mt M M M MMA n MI 0

11111111 I I 1 11111 1 I 1 1 I r- IT,

m m C, r4 sO -.0 -0 .4.- uo N. m vo Cr- on co CO CA ..t 1.0 1- r..1 ON ,0 tn PO .4- -s ...o ro co c...! uo

a- .4- uo .nr ro 127 -- N ifl VD -0 1" b7 V, sO M r4 r4 MI CO r4 M V, Cr- sO C4 CO 0, 1.1.4 -4--. ,0 CCI VI

7st ..-.. CD CD VI CA r4 04 ...- ,-- .4-- OD MA C4 M Cl M M M MA CA r4 r4 .4-- MI In PO C4 M rl M -- tn .A.=

.••••••••••••••. .••••••••••••••• s-

Si Si n tn In tn Si tn Sit M Mi SiI MI M SD M CD CD CD M CD CD CD CD CD CD CD In In M M M 113

1111 1 1 1 1 1 1 1 1 1 1 1 1 1 a,

0, b7 M V, VI Ul sO r..1 .- r..1 V, 10, 127 bl M N. CO r4 V, C4 V7 tn Cr- C..1 -- 0, r..I ON -st" In ....- NO ..---1

PO rA 0, 0-- '4- r, 0, CO r, uo ro ..0 -t- MA .4.- PI .Kr. -- r, -4- uo -- r, co r, ,0 ,0 Cr- m m m ,... !1:1

17. M M ..- M M .4.- MA M M C, .4-- .4-- Cl tn ..- ..- M MA GI .4-- MI -4-.. MI MI MI CA tn MI VI In tn .4.- E • . • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • al

M MI MA CD M CD SI CD CD M M CD CD CD MA t M M MI In MI m m m MA Mt MA 'Si tn Si 'Si M Cr-.

1 I I 1 I I 1 I I 1 I I -

In =-- CO r, CO CA 'Si NI CO -,-- Ill CO .4-- V, IN VI CA .4-- .q.. ro ,o -- co CJ No ro r, tn ...o n in, ro in

-- ro PO Cr- in -.0 0./... -10 'Si V, b7 147 1.0 V, t 0, r..1 MA 0, VD Si r4 Si uo ro bl MA tn 0, 1- CA sO 4-

J .,-- CD PO NO =- ...-- CD CD mm ....- CJ CA CD NI NO M CD rl NO CA ....- CD ,- NI ..-- CA CA .4--.4 .4-- r4 .g.. • • • • • . . • • • • . . . • . . • . . • • • • • VI • • • • •

M tn M M M M M M T... M M M M tn M M 'Si M M M In M M MA M MI MA M n Si M MI

1 1 1 1 I 1 1 I 1 1 I 1 I I 1 5-

SE ....- In CO CJ r, 0, In CO NI r4 ,- In n 1- b7 CO -4-. 0, V, CO .4-.. 0, V, s0 N rl N. ,.... ... OD V, sO ill

P., IN c...1 .1. .4- -- IN bl ,CI CO C...1 n CO VI M Cl sO PO b7 C4 .... V, Sr VI ro r, uo r, -- m m ..s. r E 1-4 M M M ,- CD CD CD r. n tn .,-- n n CD CD SO 'S CD CD =-.. -,-- 'Si CD CA CD CD CD CD CD CD CD 'Si

• • 11 • • • • • 11 II • • • • • • • • •••••••••••• • • ai

O 'Si *4 t54 *4 M M M M MI tn MA MA M In M) Si (Si tn tn on In IS1 MI M en MI In m m m tn m U 14. 1 1 1 I I 1 1 I liti I 1 I .-

co

ro CA ........ r, In -0 -0 In CD ,0 .0 c.-.1 .a.. cl ro n N. c...1 .vi- r, os 00 oo r, .1- CO 111 CD CO .1- ,0 CO .4-1

In NO CA CD 0, N. r, r, Cr Cl'- .- Cr- Si 40 T.. T- Cr- 'q- If, N. T- ON -40 -ci r, n -- Cr- .q. uo Cr. ..,-- !:-

mc rq m ro uo .4- m 0, CD CD M CA CA CA r..1 SO rl CA (Si CA NO In SO r, ton m -- MA VI .4-- VD 'S .4.- 115 • • • • • • • • . . • • • . • • . •• •••• • • •• • •• • • >

'Si M 'Si 'Si Si CA CD CD CD (Si 'Si SiC 'Si 'Si 'Si CD 'Si (Si 'Si 'Si 'Si 'Si (Si 'Si 'Si 'Si CD (Si (Si 'Si CD 'Si ) I I I I I ! I I I I I I I I (3

N .4. so -- r, Cr, so Cr, rq uo so m CO !cr uo bl 04 V, c...1 r, ..- N) -4- ...- -,tr -- 'Si CJ os N CO 0.` ..

cq oN -0 so in ,-- r, ,- so N. In -4- r, ro ,0a 'Si os No ro r, r, c..1 ro N) 'Si No CA MI .4-- tn 0- CI

CO ,... 'Si (Si r4 CA 0, 'Si St ,-- — CA CA CACD .4- (Si 'Si 'Si ,-- rq cs.1 .4. -,- 'Si (Si VD C4 'Si VD 'Si -4- .,- r-

ME• ••• ••• 411•11% • N • • • • • • 4 • • • • • • • • • 0

'Si MI n M CA CA CD 'Si CD CD M M MA ISA M MA M CD eD CD CD CD M M CD CD CD SI CD SO 'Si CD I I I I I I I I I I I I I I 1 I I 4-5

r

ro ro -- Cr. a. r, uo r, CO ro NO Cl'- .4- Si -0 Cr- Cr- L47 r. uo .i. vo Cr, CA VI -,,r .- CO ro ul N. ,0 T

,-- m 4:1 m m LI...) Cr a to -4- N') co ro r, uo -- rq CJ 'Si r, N uo ,-- oo .4- m uo .4- ro 'Si C'- NO La

14. N -- --- .4. 00 CA v m ,-- m rui CA CA CA CA M (N m m r..1 In (....! .,s m n -- -- -- -- ro -- -- • • • • . . • . • • • ••••• V • • • • * •Illt• 0 • • ••• 1.41

MA M CD CA CD CD M M CA MA Mt Mt M (Si CD CD (Si CD CDCD CD CD CD M CD CD CD CD Si I'D CD CP SE I I I I I I I I I I I I I I I I I

a)

-- so ro.,- oN -- r, CA IN r, r-.1 so --- L7 MI ON rl OD L7 MI MA ISA 1.0 VI CO .4.- V, 1.17 -CI r, m Cr. 4.-4 N sr ,- PO 'Si -Cl M St Ok Cr- .4- uo uo CO -0 CO V, 'Si U7 1- sO 0 r4 v CO M 'Si ro ,- sO VD Si J...5

uu 'Si m m oN .tr CA In ...- NO CD NO CD MI CD CA CD CD CD CD In .q. r.4 v .,-- -- ro ro ..-Mm -- ol M I--

SE

c-1 PO E r.1 sr 5_ .gc cw CD mc -J ra Mt CD CV CN CV CD G.. SE 04 LJ LW Li_ CD MC F4 -J NC Mt CD CN CN CV ,I 0

Li-

• ••• 4 %• •••••••• ••• 4 IRNMO 4 UN • •••••

Si Si Si Si SA S! ta Si n n n Si VA SI Szi Si n M tS4 M S4 'Si (Si 'Si (Si 'Si (Si m 'Sim m (5 I I I I I I I I I I I I I I I

PO 0.. SO NI ...- -.0 ,- 00 CO CD 0, CD r4 1. V, N. ON. .,... S! cq o, n ....r N3 00 In '4' ..-.. PO ln In CO

0, .,-- 03 =.- ..0 ,0 CA CA NI Cr- L7 .4- to -- r, CJ uo cl 'Si uo -4-. ,o N. In VI In 0, .1- (Si CO co co

CJ CD CD CD M =-. CD PO CD NI -,.... CD In CD ..--. CA ,I3 =-. CD UO CD ...- M NO CD NO eD CD In CD =-. CA ,r . • • • • • ...... • ••• • • •• • it ■ • • • • • • • • ••

'Si 'Si *-- 'Si m m Sit St t'M In Mi (5) Mt *4 St M (S4 S4 tsi 'Si 'Si (S) (Si (Si t!A St M MI 154 P.54 151 S I I I 1 1 1 1 1 1 I t I I

CO ,..-- Cr- -.0 ro .4.. c...4 un -- uo VI MA 1.1 VD r. CJ b7 VI MA M .4-- Cr- 1- bl 1- .4-- ....- CJ MI r, Cr- --

No Cr- ,- .4- m Cl- ro rq PO C4 1- (Si PO .4.- Cr. 0" -0 M '0 1.0 CO Cl .4-- b7 -4-- 04 (Si b7 bl -0 'Si Cr.

P4 CD 0, CD M ,- M 15, CD CD M MI ,..... M (Si M M Mi VD 1$) n l) -4-- (5) to M M -- M •-'5; .4.-- tn ••• • • 1111••••• •••• ••• Ili •••• 0 •• • ••• •

MI In *4 (Si m 1s, m 1St Sa ,S, 151 S4 Si VA Si (Si (Si 1:54 I:54 St (Si m '5) (Si 'Si M M (Si Si Si m 'Si I I I I I I I I I I I I I I

PO CO NI ..- PO CA N) ...-- Ul 0, N) CO NO rn Cr, 121 CO r4 b7 1" CO 111 CO V7 Lil Cl Cr. .4-- CO Cr. CO V1

r, ro 0- CJ --- rl In r, -- ro .4- m .1- m r, r4 MA V, 03 MI C4 r,...4.- -- ,- .4- m .0 co ,_ r.., ,D

Ct CO CD CD M CA ..-- CA OD =.- CD =,.. =-. CA NO S4 =-. In CD ...- ...- CA CD CA CD =-- CA ...-- ,-- CD ..r. m n • . • • • • • • • • 1111•••• ••• ••••• • •• • •• 4 •

MI tn MI M MI MI MI MI MI MI trA Si S4 VA S 1 VA SI SI 1:54 VA tA ISI Si VA St VA VA MI VA SI VA St

I I I I I I I I I I I I I I I

Page 174: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

174

ro co cr• so cs co ..aci ro PO In CA CA Ar Al A- At AO 0, CO Co PI 0, A- At ,0 A-- OD Al CA ro PO A- eN Cl Co -4- Cl A- A- A- Co 4- Co -4- Co No

L11 .4 4 r, 1.0 NO L7 un Co N) cr. 4- in CO 4- uo CSA ,- to Co oo Co Cl r, co Cs 41- c.4 C4 Al Co CA In A- A- CA A- Co r0 Co CA 4-

••••••• 2 .2••• • /I X • • • • •••••11211••• • •

MI Co Co rA Co Co CA CA CA CA Co CA Co Co Co Co CA Co Co Co CA CA Cl CA Co Co Co Co CA Co Co Co 1 1 1 1 1 1 1 I I 1 1 I 1 1 1

CO Cs 1.11 CA N. CO AD r-, m- c..4 wo in -0 A" -zr to n ro or. ,r .4- 4- Co Co .4- CO 5-, At r Co co

N) N GI CO N) Cs Co 0, CA CA Co ,0 CA At -- 4- AD Co CA A, CO 0, In Co 0 -0 N) In NO AD A- CO CA

Cl Co ,- (-A .-- A- At CA Co CA Co A- N) A- CA AA N1 rA A- PO Co A- N) ,- CA CA CA CA Al A- CA 0, ,r • •••••••••••••• • • •• so••• • . . . • . . •

Co CA Co CA M CA CA Co CA Co CA Co Co CA M Co CA Co Co CA Co Co (Si Co Co Co Co Co Co Co M Co I I I I I I I I I I I I I I I

Cl' r*, In N. In N. 4- co n un co cs .4- ...0 ,r C4 '-C) If) Co -- ro un el cr. or to ..r Op ,- ..4-. U')

Cl ..-- ,o CO -0 (Si A- AO CA Ar Co A- AO CA ,0 NO At NA ND ND Co N. Ar Co At Al N) "0 A- (N r, ,- CA

Cl 4- Co -- Co ro Co ro Co -- Co Co .-- -- 4- Co A- ND Co CA CA CA Co Al -- CA A- Co CA -- 0- m -- 1. 111R ••• 1121I• ...In • • • • 1• • ••••• •••••• •

M Co Co Co Co CA Co Co Co (Si Co Co CA Co Co Co Co (Si CA rA Co Co Co CA Co Co Co CA Co (Si Co m 1 I 1 11111 1 I 1 1 1 1 I 1 1

CO CA N) NO ND 0 0) A-. -.0 At N) N. A, VI Cs N. 4- A3 N r, CO 0- -T Cl co Cl. Cr. co '0 co ...r .4.

A-- Co In Co A- N) CA 4- ,- Cr- Co c, m ,r N) -0 ND 4- r, ro Co (N N. ro r, 4- Co -0 CA Cr. Cl UD Ar

eN CAA- Co PO ..-- ro -- n ,-. ro CA CA In CA CA CA CA Co CA .4. ...- (A ,- is, A- ..r. -- Co AA ,- ,- --

..• • • • •••••••••• 11••••••••••••• X M

'Si Co Co Co Co Co Co Co Co Co Co CA CA Co Co Co Co Co Co Co Co Co Co tS, Co Co Co CA A- CA Co CA 1 I 1 1 I 1 I 1 1 I

A- CA A- If) CO Cl CA A- Co 0, AD -0 4- -- Cl. cs A3 (-4 In Cl CS AD Co At In CA CO -0 CO At N. Cs

AD 10 At N) At CA 0, A, CI Cs Cs CA AD CA N. CA A- Al OD .4. -- CA OD In ro uo r.„ un el -- to 0,

co ...- Co un -- -- M PO Co CA Co 'Si In Co A- CA In Cl Co 4- .- el Co 4- Co Cl CA Co CO CA Cl VI 4- CI

••••••••••••• 4.1 • • • • •••••••••••• • 21

Co (Si Co Co MI M Co Co Si Cl M Co Co Co Co M Co Co MI SI M Co (Si Co Co Co CA CA CA (Si CA CA ID 1 1 1 1 1 1 1 I 1 I 1 1 I 1 I 1 1 1 1 1 1 r-

cr,

0, -- AT r, A- Co AA A, A, Cl 4- -o -- Al ,r (N co co r, 4- .- to -- Ni -- N. Cl co Cr. un co tO

m Co CT- M In AO A- 1271 Co AD CO '0 AO Co N) co ro so Cl un --- ,- to 4- ro -o ro N. AD -0 in CO 1.11

At A- A- Co NI A- Cl CA CA Cl CA CA Co A- Co Al A- Co CA CA CA Cl CA Co Co A- CA Co Co A- Co Cl CA -s) • . • • • • • . • • . • . • • • • • • . • • . • • . • • • • • •

Co Co Co Co Co Co Co Co CA Cl Co CA CA CA (Si (Si Co Co Co CO CA Co Co Co Co Co Co CA Co Co CA Co 115 1 1 1 1 1 1 1 1 1 1 1 I 1 I I 1 t

al N -- un ,- 4- A- Co CA tO CS Ar A, 0, CA -0 Ch .47 N. CO or. ,r 4- Co At AD If) N. r4 rq cr. 4- Co ,...

At CA In CA CA rA CA A, In AD CA At CA At AO CA NO At =- A, CO OD CA CA Al CA NO AD CA VO tO A, Al • N Co m PO A- NO A- Co .- CA ND (Si CA Si CA CA A- Co •A- PO Co A- A- -- Co Co (N Co .4- -- Cl Co 1-. -

•••••••••••••120 4 •••••• •••••••• • • al

Co Co M CA Co Co (Si Co Co Co Co Co Co M 'Si Co CA Co Co CA CA (Si Co Co CA A- Co CA Co (Si 'Si Co 4- 1 1 1 1 I 1 1 I I I I 1 I

,:...

In At CO CO NI .4. .,....r r,... -0 -- N) CO 0, VI Cl a -0 -- Ll -0 0) ro Cl 4- CA 0 .- In CO Cs CA At CI

A- A- ro co 4. to r, to ro N AD to ro to PO un 1- un co IN rA A- CA r4 to, ro AD Al .4- un -0 (N 4-

J A- Co N) A- Co CA (Si Co N) Cl Co CA A- A- A- CA A- Co Cl (N CA Co A- A- CO Co A- CA A- CA Co N)

. • • . • . • • • . • . • . • • • • . • . • • . . . • . • . • .

CA CA CA CA CA CA CA M CA CA m m M m m CA CA CA CA CA CA CA CA CA CA CA CA CA CA CA CA CA 44 i I I i I I I i I 1 i IIII I 1 I 5-

.1.) PG ro un ro ro Cl A- N -0 CA Cl CA -O CO In CA Cs A- Co 4. co Co co r, Cr- 4- 1- to 4- Cl CA CA AO !Z •-U') In At CO NO *0 ND In CO Cs CA CA AO AO Co ND A- CA CO AO AD A, Al CA CA At In A, At ,0 CA E

E hi la CP Co A- (Si Co CA N -' (Si (Si (Si Cl Co CA CA CA Co Co A- ,- (Si Co r.„. ,- ..-. tsi 'Si 'Si ,- Co m 5- • . • • • • ••••••• •• • • • • • ••••••• 2 n • • • al 0 Co Co (Si Co Co m Co Co Co Co CA CA Co Co Co CA CA (Si Co Co Co (Si CA '5) Co CA AA CA CA CA (5) CA AI U.. I I I I I I I I I I I I I I I I I I I I r-

tr,

co 1r 41- un cs 4- co N co CN AD CA -0 CA 0, .- CA AO Os In ,- In (Si N. N f;.! ,- Co .4- 4,-, .4.- IJA •!..., ,r --- 1-.., el ,-. ert -0 ,r Cl N. N ro ro lo- N) A, A- NO cs .4- ND PI AD A, CA CA ND CO NO Co AA CO '...-

:C CA CA ro -4- .4- -- A, CI CA CA A- tO A- ,- AA CA ro m eq ro 44- ..- o, m .- ,- m -r -- ro -- ,-. III

. • . • • • •••••••••• • • ••••• ••••••••• >

Co (Si Co CA CA CA CA rA CA CA (Si CA CA CA CA CA CA Co Co M (Si rA Co MI 'SA Co Mi t'S! m m (5) m a i I I I I I I I I I 1 I 1 5_3

1.11 0, CA CA AO AI CO Os A- CA Cs. N. CA A- A- CO Cr- CO AA r, r, .4- In co ro ,r. ro -0 0, -0 4. -0 ..

N. CA A3 In LO N Cl'. N. CA ....' M .I...' r‘.. ...- 4- Al A- A, A, AO N. N1 ND AO A- CO A- CA A, AV In CA G

0 mi A- GI cq N ..T. m m -- -- rq m N m to m Co CA 'Si CA CA Cs A- CA CA A- CA CA CA CA ro el .

. . • . • . • • . . • • • . . • . • • • • • • . • • • . • • • • -r.,

m in m m Co m m m m m Co m CP CA CA CA CA CA CA CA CA CA CI Co (Si CA (Si 'Si CA CA (5) Co I 1 1 1 I 1 1 1 1 1 I I I I 1 1 I I 10

r.-

co .2... CP-. Co .4" ,- CN T- -0 1'0 rq T.... M Cr- ND CA Cr- CO Cs At At N -- co CO V ,- 0- CO ,-- 4- un !.1

Cl CO At sO N rr.. ,-. ..- Co vo Cl co CA tO Ni N) AO Ar Cl (Si N) N. ,0 Li) CA CO -- A- Cl A, Cr- N) CI

U. Cl Co ....... .4- un Cl in .,-- CA Co CA Co Cl CA Cl Co CA Co CA N) Cr- CA AP A- CA CA CA CA A- CA -- A- ▪ • • •••• • • • • •..• • •• •••••• • • •• MI

Ma MI Co MI CP Co Co Co Co Co (Si CA Co Co (Si 'Si Co Co 'Si CA CA Co CP rA CA 'Si (Si CA CA Co CA (5) ANC_) I I I I 1 II! I I I I I I I I

at

,r 'Si Cl 'Si Al A, N. CO N) Cl Ar In AD In Cl A, N M IN 4' 'T N n CO '0 0-' .q.. Ol N n 0... N ,..1

Co 0 n 4" N N) N. CA ND At CO N ..- 4- CN Cl In A- 0, r"., CA Al At CO (N r, un 4- (Sr Co OD A- i

LU A- 'Si Co In Cl Cl Al A- N) A- Cl Co ,r Co -- 'Si Co Co Co to... N) C..1 N) ,- Cl. N) CA ,- ....4- Cl 'Si ,- • . • • • • • • • . . • • . • • • • . • • ••••••••• • • j•••••

M Co Co 'Si Co Co 'Si M M (Si (Si CA Co 'Si (Si (Si Co Co Co CA Co Co Co Co (Si (Si 'Si Co CA AA CA CA 1 I 1 I I I I i I I 1 I I

In to Co AO N. CA At r, 4- r, uo r4 ...- un -CI N) vo N) N) N. 0, I..0 0-. 4- un co r, un r, un -0 4-

CO -0 Co AO CD AD AO AO Cr- A, (Si CA N) A- AO IN r, ro ro or. N N. cr• 'Si CO ,- cr./ co -o No r, ,-

L) ...- Co LC) Co Co .- Cl Co N Co Co 4- Co ..- N) 4- ,- Co OD CA CA Co Cl CA CA A- (N At CA Cl in 1.1") • • • • • • • . • • • . . • . • . . • • • • . • • . . • ••• •

Co Co Co Mt Co Co Co CA Co Co Co M (Si Co Co M Co Co CA CA CA CA CA Co (Si Cl Co 'Si Co Co Co Co I I I I I I I I I I I I I 1 1 1

CA N) A- CO AO r, N cr. ,-. c...1 4- -0 m N) r, ro ...o r.... N) CA CO CCI AA Co A- r, CO CA "0 Co Cr At

N Cl CA CA Cl 0, At N) Co A- Cl CA Ni AO r", ND 0 CO ND A- 4- r, -0 -- 'Jo 4. -o N) r, N) r4 n

04 Co PO (Si (Si CA Co Co Co Co Co Co M M Co 'Si M Co r, Co Si 'Si Co n (Si Cr m m CA m m -- Co • • . . • . . . • • . . • • • • . . . . . — • • • ••• • •

Co (Si Co Co M Co 'Si M M Co Co 'Si Si Co Co 'Si CA Co CA M CA Co 'Si Co Co CA Co Co Co Co CA Co I I I I I I I I I I I I

CO in Ch C.4 ON N N .h.. Cl CO Cl. Cl. At CO CN ..... 03 ,0 VD CA 0, 0, CA A- AD A3 CO NO 4- ..- N) lin

Co AD 1.11 r., c-4 Co 0, AD Cl 4- CA A3 r.... r, Ar AA N. ,0 N. UO N) A- A- N1 At N) A3 A- ,r ro Co In

<X In CA A- Co Cl Co Cl 'Si M Co Co ....... M Cl Co ,- N. Co ..-- (Si Cl Co P7 Co A- A- Co CA (Si N) Co M • a • • • 1. • • • • • •0••• • • • • ••••••••• • • •

Co Co Co CA CA Co Co CSI CA Co Co Co CA Co Co Co Co Co Co Co Co Co Co Co Co Co Co Co CA Co Co rA I 1 1 I 1 1 1 I I

AE i.

E ...- c.4 Ni .7- E — N I,r) V- A. .4C PA 5-.) LW U. Co MC 1--1 ...I AC NC CD CM Cl Cl Cl C. *X Oq 6) UJ U. Co = 1-1 -I AC At Cl CA Cl Cl CM 0 0 U. U..

Page 175: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

175

r, No (-1 r, No ro V1 n co No -- No r, r- sr un 0, CO CO 4- sO Cs 44- r, r" ro

.tr ,c1 t tSi N) EN CO r ,0 4- Si N N r -CI CA N7 CCI r.r I IS; 1.0 - CN r- (5) N t51 1'0 m EN co ta m 4 IS) r m rl m N) a m -4- m CA Ul

masa.. • • •••• • • • • 11•••••• ■■ ••1111••••

M CD CD CD CD CD CD CD CD CD CD CD CD CD CD CD CD CD CD CD CD Si Si fAntantamtamm I 1 I 1 I I 1 I 1 1 1 I I I 1 1 1

• r, 4'0 Cs 00 r, M In M r, ....-- PI CA 4- CA PI NO ..... Cl-. -Cl r, 0- sO CO N) CA r- r0 ,-- -o -o ,---

PI SI 4- M N. CO 'gi- NI PO ND CA In -o PO sO NI CA Si ro .q- 4- co c..1 rg .1- clop N) ..4- Cl r...1 If) ,1.- 0.

C5 CD CD CA r- ,-- PI M ,--- CA r- CM r- M to N. Ni ,-. M .- ,- ...- po m m m ,- r..1 ,- ,- r 44- Cl. • • . • • . . • • . • . • . • • • . . • ... • . • • •••• II

M Cl M M M Si. M M M n Co M M Si M M Si S4 fili M M (5, M M M n M SI M fA M M

1 1 1 I 1111 1111 I 1 1 1 I I I 1 I I

CI M CO CA sO PO r- r- N - m If) co m Tr N. CO In rl N) Cs N. sO CA PO PO -0 N. 0, 0, If) ,--.

CA M NO -0 CS If) ,..-- CO N) a- -- r, r, m - -Cl No .cr Co uo ,-- 4- c..1 - If) -- - c..1 If) r, n .4- CO

CN rl I:A ,...- SI N) tA N) .r.- Si Si Co ,--. IS) CP M ..---- CA (Si 51 Si CA V.:A PO ,--- CA M M ,--.. M .tr m m

. • • • • • . • • . . . • • . • . • • • • • • . • . . • • . • •

CSI Co Co M Co Co Co Co Co IS M tA (Si M IS) Co Co Co Si Co M Co (Si Co M Co M M SI (IA n n

I 1 1 1 1 1 I I I I I I I 1 1 1

N .q- CO Os -'Cl N VI If) CA Co If) SI r- CO CA sO .q- N) m In SO 4- N. 4-0 N) r, CO PO CO CO On ,-

.- N SI ,ON Li) ,..- N Li) PI `AD 'V Co '0 M N) ..--- C.4 SA ,-- N 'V CO CA ,-- ,0 Li) ,-- 'f- 15) 0, If) .-0

0 Co Co M CA .r.- .1-. ,-- Co I- Co r" ,-- N Co Co Co SI SI SA N I- .1- T. •■•• SA ,- ,- M tr n M M • 4 •• • • II•1•• •••• ••••• 4 ••• ■ • • IR

IS)•

Co CD CD CD CD Co Co Co (Si M M M Co Co M M CA M M Co 15) tA M Si Co M M Si M SI M

I I I I 1 I I 1 I I I I I I

(N M In r- ,--- In r-. r-- M -Cl In Os Co If) r.1 ,- N) uo N) CO CO '4" Os P., CA CA ,0 -T. CO rd ND ,-

-0 M M ill 44- ..-- Cl Co 4- 4.- Cs PO Si r, -Cl N. sO CO r, 00 uo 4- .a- ..-- -q- ro ro CA CO 0- CO N) 0

CO ...- Co .q. Co ," M CA r- CA Si M N. T... .,.. .,-- .q. Co-- N) Co ...- M CA r- CA Co M TV tA ,...- CA 4.0 • • ••••• • •• • • •• •• ••••• • •• ••• •• • • • 0

M n M M M n M M M M IS) M 15) M M CM Si M Si (IA M M M Co M St M I5 SI Co Si (Si r-.

1 1 1 1 1 I I 1 1 I i 1 I 1 I I I I 1 I Co

MI Os N. r- ,..- Cs PO 00 fA r- Si If) In M .- NO M T." N. -0 N) ,- n PO r- 4'0 N. M CO OD Cs 4.0 Cl

,C1 In VI 0, CP ro .,-- m co r, sD 0, '4' r, n 4,, In M ,0 OD r... M 4.0 0 PO ro CO M CO CA CA SA U

Z 'Si Cl ..-- CA ,- ,-. CA M M M CO M r- M M Si M r- Co ,-- CA CA CA CA Si CA ,--- r- r- r- M M r- • • ••••••• ••• • • WV ••••• • ••••• • ••• • al

Si fA SI M CA M Co M M n M Co Co M (Si Si (5) Co Co M Si M Si M (S Si M M Si M 154 CD •.

III)! I 1 1 1 1 1 1 1 1 1 1 1 I Li u'i

*A 0, Cs ta 4- r, ro CO NI M ,...- '40 M r--. h, 07 In r- CA If) sD .q" tA N. N CO SA .q- -- co N)

-40 In In M MO Os ,-. Si CA Os r, 4- -c -- CA M CA CD CO CA Si M ---0 0, Cs CA PO 0, Cs r, 0, ..4. el

Co M Co ,-- ,-- Co M CA Co CO M M M M r- r- r- M M T- ,^ M M M M CA -- SI ,--- M SI SI 1-4 • • • ••• • • ••••• • •• •••••••••• • •• •••

Co M Co Co M M Co M M M Si M M M M M KA (Si Co M (Si M M M M (5 15) M SI n CA (5) :-- I I I I I I I I I I 1 I I Cl

5-

..,:r In CO CA I5 CP 'cl- If) -0 CA fA M CA CA M 4- N. uo ,0 4- - 4- - n .4. Cl s0 CO T- N) r, CA

N) CA ,-- CA Co r- N) If) SI CA CO 4- N) a- ro a- -A- uo OD r, rl '5) ta 4' CA CO CO r- CO N) if) N) 5..

--I CD Mt N) ro cl ..-- - 15. CO M M CA ,-- M CA 4.1 M M CA r- CA (N Si M CA M r" CA ,--- M CA sr CI

. • • • • • • . . . . . • • . . • . a * • * • • • a • • • le • • r.!..- MI M M M M M M M M M M M M M M M M M M MI Co M St S4 M M ta m M SI M Si

1 I I I 1 I I 1 1 1 I 1 I >: •Ti

Z 1'0 ..-- 4. No cl., NI No CO If) CO CO r- n ,--- if) M CA r, N. sO 0, CA CO fA --0 CO CA 4- 4' 4- -0 CA 1.

'0 sD M ...-- sO n CA n In M M M 1.0 PO ro -o rl to r, .,-/- n M ,-- sD Tr ,--- •4.- 00 s7 In .CI Si .1-, E N4 Co M M M M M M ,0 M Cl M T- MI ,.... T-. M T... T." M 15) M M M N) M CA Co M M (S ,-- ,--- i-C, S.- • • • • •••• • • • ••• •Il •••••••• • • ••• ••• P.

O Co M M Si 'Si M M SI SI M M Si fA M M Si 'Si Si SI KA M M M M M M M St n M KA M Li.. I 1 1 1 I I I I I 1 I I Cl

u

CO If) CO M M ...- sO -0 'cl" VO N) N PI ,-- Si n TI" M Cs N. rl CO CO r, Co -0 Cl- Os ...." CO M (5 1.---- ▪ - CA CO If) KA SA (Si NI 4'0 r- ,..- CA N. CO PO r- .q- No O. OD 4' ,-- N) 4- cl,.• r, CO N. (-I No n -0 6.1

S PO r- r- If) -AO M M 'S ...-- M CA CA r- 4.0 KA CA 4. m (S (N uo - r, n n - ....- Cl rq .,..1- - ta •,1

. • • • • • • • • • • • • • a • • • • • • • • • • • • • • • • • . r.- M M M CA M CA .-- M M M M M M M M M SI M Co M M M M M M M M M M Co SI 'Si n

I I I 1 I I i I I I 1 I 1 1 I ..-> 0

If) N sO CO CO ..-- r- CA ON N 0, if) N. PI N. MI PO FO M 4" '47 Cs -Cl r, Os N. ,--- '0 '4" 0, IN PO L)

M r, PO M n sO M In -,-. CS PO ...- ..-- r..., .4' CO sO r- Cr. PO '4" M CA r, ...- a- .4- c.A Cl uo r, a-

CD ..-- CD Co CA ...- N. SI 1:5/ r..- M ,-- M r- M PD M M Co M ,..- ,-- In M 15) M ..-- CA 15) ,- M 4-9 r- •11

• • •••••• • • • ••••• ••••• • • • • • • • • I.• • a,

Co M M Cal M M Co Si M M M M M SI Si M M M M Si KA Co SI M M (S M fA St Co M m I 1 1 I i 1 i 1 1 1 I I i I 11 I I C'-.1

Co .4- CO ,-- N. CO M CP Si 4- ..- -- -0 ,13 CO -0 PO 4' CA If) Ul 4- -- if) ijo ,-. rl 0, ro - 4- (-1 .4-,

n -o co -- to Li) m 4D M ro O. -A- If) If) CO CA .,-.r -gr c- r, r... Tr CA CA CA ,- CA CO OD Cl '0 IN r-

11.. PO r..-. r- ,lt CP ,-- -0 Co CA ,-- ..-- r- ,^ N) ,-- ...-- PO r- M ....- -0 CA In r- ..--. ,-- CA ,--- -,--- 4- M SA ill • • • • • . • • • • • • • • • • • ••••• ••••• • ••• • Ci. C4 M to M M M to 'Si Co Co M Si 'Si m Co m CD CD CD Co CD ta n M SI M M Co M M SI M

I I I I 1 1 1 I 1 I 1 1 1 I 1 4-

C?<4:

r- r- .tr r... CO Co SD CA Co -..- r- Cl- CA 0, N. fA n N -q- Cr. OD r, PO CO ,-. r, oN . 1.- N) CA o,

..-- in - .,-A- - m uo -- CA M CP In Ns 0, N. PO ,0 In ,0 In M 1.0 0, PO CP NI PO n r, CS 'CI IN Cl

LU ...- Co CD CN ..4. CA 141 SI N) ..-- CA Co NI M ...... 15) CA (Si Co .a- 4- N) PO ,..- Co N) .q. m ro -- n ..-- ,--1 • . • • • • . CA

S • . • • • . • • • • • . • • • • • • • • . z

CA CD CD M M M M M M SA M M M M M M SA M Cil M M M St M M M St M M St 65 I 1 I 1 1 I I I 1 I 1 1 I I I I-

.4- a- r. m 00 sO CO 4" CO 0, N. In CO CO PO CA r- In PO CP CA uo P. t'... 41 CD C) 4- CO r, ,0 CO

00 M r, - OD HO OD M ,..- n 441 M SD ,0 M M CO fA Os SI CA n sO ,- r, .P r0 ,0 r- 1.0 4.- 4-

U Co ...-- 0.-. CD ,- Co ,- Co PO 'Si ...- 4' Co ,-. CA In SA Co N) M CA M N 'Si CA Co r- N) Co CA CA Tr • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • CA M M Co Co Co M M M Co Co M C11 M Co IS) Co M Co M tA M M fA M M Co Si 'Si SI Co M

I I I I 1 I I I I I I 1 1 I I

CA rte} rt" r- .q.• Cl n ,-- n Cs Cs Co .nr Co r... No uo 4- 1.0 SO In -0 ...r ro c-. N. No CO Cs sO rA .q.

N ,•-• Si n ,CI N C.4 ,0 CA If) V) Co -Cl -.0 4- Co ,--- NI ..-- ..- (f) CI, ,II •-• NI MI N CA CO ,- TI- ,-

P4 CD ON. ,-- CD ,..- CD ..... CD CD CD CD CD M M Mt M M MI M Co r.- Cl fill ,---. (Si Co Co M M ,-- Co Co • • • . • a • • • • 4 • • ••• ••• •IIII•• ••••• • • •

Co M Co Co Co ta to Co m Co m (Si m Co n m m to M M CD CD CD CD M M M fA M M M M

I 11111 I 1 I 1 I 1111 I I

C'..I C..1 ct ,- Co n 00 ro cr to N) N r, Co '- r, MI N -- n VI CO r- sO 00 Co ,-- N) 1.0 ,-- CA VO

CA N. CO r- Co (Si 'V' sO N) NO sO sO CA SI M sO .q. m ...- ..-. ..-- '40 -0 PO Cl--. r0 r0 'ql" N. Cs ,-. .q" <C CO CD Co ..-- 4.1 ..r. N) m to m Co ....- m ro m Co In r... r- M ro Co N) IS) to m m ....- Co ro - Co • . • • • • • • •• ••• ••• ••• •• • ••• •• ••• 1111

Co Co CD Co Co Co CD Co Co Co Co C4 Co CD CD Co Co CD Co Cal Co Co C.4 CA Cl CD to CD Co CD CD CD I I I 1 1 I I I 1 1 1 I I I

04 MI I".7 sr sr 04 GJ Ul U.. CD MC -.I AC .7.M CD CV CV CO CO c_ 04 C. W 1.4.. CD MC AC Wit CD CO CO CO CO

0

Page 176: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

176

PO a- co a Cl ro m Cl CA +- PI +- +- +- N) m uo n m CO Cr-. ,4 ON In in T-• in 54 .1- a r, r, No n ro n ro No Co a N co n CA KA In h, A- r, m a, a CA VA on m m m m n m N) KA KA CA In KA KA m m m m m Co

• • • • • • • • • • • • • • • • • • • • • • II ••••••••• MMMMMMMMMMMMIMM M M M trig MMMM M M M MMnM I I I I I I i 1 1 1 1 I 1

CA CA NO CA 'gr h, KA NO h, NI a ro Cl-- DO NO A- Ai +- +- CA NO CO In +- N. a- ts, r, -- n uo n

PO +- '1- a- No No r, m No In 0, CA OD In 'gr "gr 0, +- NO KA KA In a- No a n a- uo ro m a- -o .,71..

cr. — n cq m n N) -- -- CA KA KA CA KA KA 'gr CA KA KA N) KA +- NI +- KA A- +- CA CAI KA KA 00 N1

/••21••••1111•• ■ ••• • • V •12•••••••••• a

M M M M Mi KA KA KA KA KA ISA KA Si SA KA M KA KA KA KA KA KA SA KA Can KA KA SA KA lx) KA I 1 1 I I I 1 1 1 1 1 I I I I I I I I

+- .-4D Pa N) +- Cr-- co a NI CO CO CA CO Cr. '<I In PO CP NO — a- -- r..! No -O uo -- a N. No KA In

CA 0- -- uo ON CA In NO In ro N IN0' ON KA VI a- N MI NO 0-. SI M T". MI a CA ro No co uo a uo cv ro -- CA +- 'q- m a- m n n -- -- r a ts, .- ro n -- m a n a- -- CA m n -- -- a n n

. • . • . • . • . . • • • • • • • • • • • • • • • • • • II • • • M M M MII M M MI M M M MI M n M MI Si M n M M M M MI M M M MI M Si M MI 151 1 1 1 1 1 1 1 I I I 1 1 1 I 1 1 1

In CP co a ro a- -- a -- ,-. ro OD OD ON TT. +- CA a- No N1 AO a !Jo -- ro 0, No a in r, -- ....-

-- r, co ..-- r, co n CA NO CO ON CO CO M N. Cl N) — Cl CO ON -CI CA CA CO NO CO CA If) A-- CO KA CA

CI KA KA KA NI ,-- +- CA KA +- +- +- KA 'gr KA +- KA CA KA KA CA ,--- +- +- KA KA PO +- KA ..-- +- KA +- • • • • • . • • . • ••••• II • •a• •• ■ ••••••• a a

M M M M IS, 'Si KA KA KA KA M in M M m KA 151 CA Mt CA M IS) M MI CA CA MA 151 +- 151 CA M I I I I I I I I I I I I I I

NI CO 'cr CN CN -CI a a- co a- n ,- ro r, ro N. a- uo -- ro -..- IS -- a NI NO V1 CO 't a r, a,

a- CA No uo CO CA N. Co T.". 0. M CA a 10 CA N. a- 1 ro a CO Ar CA KA MI +- N. AO 10 --0 PO ON Q.

OD +- M N) M — M CA KA CAI KA A- a n -- -- a -- m a n ,- m 1,0 ,- CA m m N 15 ,_ CA m :-..!

. • . • • • • • • • • . • 11•• • • • ••• • Il••••• •• • ri

M M M M 151 15) 151 151 1151 M M CA CA CA CA MA MA 15) CA CA MI to 15, CA Cal MA MA 151 MA M CA f- IIIIIII I I I IIIII I I I Cr,

--O M N. C..I ,-.. 0.- C1 NO M N NO CO -0 ,-. a- -- r..! co No a co m -- a r, a- In -4D ,- KA In 0

NI CA NO NO CA a- co a- co ro co ro ,-.. CA a- ON CA 10 +- ND KA CA h, +- 1.0 CA r0 IN CA ro uo 0, Li

74 'Si MA +- 1 N N — 157 .- +- +- KA +- KA CA KA KA M ,- CA CA NI KA KA KA CA 0, KA A- KA CA ,- r- .- • • • • • • , • • • • 114 ••• • 1.•• • • • • ••••••••

Iii

CA 15) CA 151 CA CA 15, CA CA MA CA 15 KA KA KA KA IS, KA KA KA n M Si 151 KA KA KA KA 151 ts) m m •m—I

I 1 1111 I I 1 I I 1 1 1 1 I I Li U)

SI r'.. CA +- +- r, No co c...! co PI CA h, NO CA OP a r, AO +- CO In PO CA h, 00 1N. .'40 0.. uo a- uo

PO ro a- ro — a- r... ..--. co CA PO NO In ,-- NI In +- a- uo No uo -Cl No co uo -'1 rA -- co c..! a n al

CA CA CA NO +- +- +- N M N Cl MA +- CA +- SI Mt M M! N fA ,- ,- ,- Si Ct. I N n ro T.! — m r—I • • • • • in • • • st • • • • • • • • • • • m • • ••••••• • IT;

CA CA CA 15, CA CA MA CA 15) CA MA MA 15, MA MI KA 'Si KA KA 15) 15) M M '5 15) KA KA KA KA KA KA KA F• I 1 1 1 1 1 I 1 I I I I ID

CO ON 10 00 10 C► 154.0aCA ,- (A ro ro PO N In KA NO AO +- P 'gr CP h, 0 N N AO N 0,4-

CP NI N a CA +- 0, "4" CA ON In '71- NO ,-- CO 10 N. .q.. N) '10 "4- NO +- VI 10 DO NI NO a- n N. *--

-.I KA KA CA MI +- CA CA CAIN m m c..! m rq M PO M M Cl ,- — M ,- M OD 'Si 'Si CA KA Cl +- CA Cr

•••••••••••••••• • ••••••••••••• • • 4-

CA CA CA CA CA KA KA KA is KA CA CA CA CA CA CA KA IS! MA CA MA SI M CA CA Mt CA KA KA KA KA KA 1 1 I I 1 1 1 1 I I I I I I I

AA al NO N) 1."N 1'..0 uo r, N. KA KA N. PO N. N) CA CO In CA 1...... Cr-- M Ul 00 1.. n .I. 01 '". 0'. — '0 T... ..40

N) T... .I..... PO N ra a No a- Cl-- uo ,- ,- If) a m No m m No -- NI CA PO +- 00 +" KA CO KA 0, +A AA £ 1"..1 M "..— M ,... ....— M IN) M 15 157— ,..." —5715.1 15) MI 15) ....... ".... "-- M CO M ,— M ...... M "-- M 15) al C. • . . . • • . . • . • • . • • . . • • • • •••••••••bq £ O 15, KA 15) ISi 15) 15) KA KA KA KA IS) IS) KA ISi 15) KA M ni 15) M M 151 Si SI SA S4 S! Sf Si Si ts, ts, U. I I I I I I 1 i 1 1 I I 1 1 1 0.1

u

,- .q.. N. N. ,- NO CO CO ,-. M 1,1 0.- N. NO -0 a- CO CP N. ON CA +- CO a- a- ro ts) -- In CA In 0,

^.10 .0 -40 CI- CA Cl N) +- KA NO NI At CA +- CA N. 0, N) IN T'. CA 1.0 CO CA ,0 NO IN CA CA +- -0 h, !li

MC IsO n CA ND til KA P, KA KA KA CA CA A- PO KA CA N1 KA +- c....! a- m co m -- -- ts) ro — a -- -- •-, • • • • • •••41.1.111.••• 1I•• • • • •• ••••••••

5)-

ts) M 15) 15) IS) IS M m n n n 15:i n 137 I5 n n n IS n 1:34 n m KA IS) 'Si 15! KA KA IS) KA KA Ai I 1 1 I 1 I 1 I I I I 1 I 1 1 A.

0

Co NO In CO 'gr CP CO CA a- a -- a a- r, Cl- co ,- a. ,- m a- co -- CD CP In CO KA a -- co 07.1 C.:1

-0 0.- N) uo a KA +- KA KA 15 IS) 'gr CO CA CA In N. CA A71 KA CA N 10 PO a- -o cl a- CA In a uo

(Si CA MA IS) ro CA III +- KA CA KA CA IS) +- KA PO KA KA KA KA CA NO OP KA +- KA +- N) m -- ts) ro -- ... • • • . . • • • •••• ■•••• ••• • • • •••••••••• 0

M 'Si '5 MA CA KA 'Si KA KA KA 'Si KA KA KA KA KA KA KA KA 151 KAKA KA KA CA IS) 'Si M KA 15) 1) is 'I I I 1 I I I I I I I I I I I AA

NO PO CA CP LO NO VO ON T- ..0 N) CO ND 0, h, CO a- cA PI r..... TT a rq uo -- co a -- No a ro l5) .-P

-- uo CA m r, -1- a uo c.., IS) -- uo a a CO -- CA -O r.... .'T N) c..1 CA -- KA AO KA CO NO KA In KA 5.-

Lt.. NO +- CA a -O— in 'Si Cg +- CA +- +- CA +- +- PO KA KA +- CP N) 1 -- -- m c...! -- -- a— u

ts) m tsa Si St ,$) n ISA n 151 tom n n 15t 13.1 n 151 n m IS m M 15) 151 M M M M M n n I I I I I I I I I I I I I I I a

.(1:

CL its

z CA ro a E CA ro r- CC OA C.) W (.L. Co MC H ....I 3= Mt CD cn CN Co Cb 6.. -'Cl 04 C.) W L. Co DC t..•i MC Mt CD Co Co ci ci 0 0

LL W6.

In NO O.1- uo a- N.. "0 'gr LO NO CO In NO NO OP 0, NO N In N. KA CP KA In A- AO NA I'A.1 A- (-1 A-

,-- ,- KA. (r) IN tO CO "1- IN C.4 CO CO CA — ..zr ,-- Cr, "-ID V.1 C'4 'I' n ,--- -0 -O "4:1 "0'1' 0-- Cr, n C4 al

LU CA KA KA 'gr +- +- CA KA +- +- +- KA CA KA +- KA to KA KA CO +- c4 C..1 ,- ,- C..1 C.4 in cl M m ,- ,-.4 • • • • • • • • • • • • . • • . • • . . • . • • . • • • • • • •

CA CA CA CA CA M M MA MA M to KA KA KA KA 154 KA KA K KA KA 15) 'Si KA I5 KA 15, '5) KA IS) 'S to ..i17.

1 I I I 1 I I i 1 I I I 1 I h-

+- VI ro r, CA KA OP N ".10 0..1 r"... 1"0 M I'0 ON In .t. a ro r, ro -- r, a ,o uo CO +- NO ,C1 A- KA

+- 4--- OP NO CP OP OP r, co co No r, ...-- if) a- n a- a ro ro r, uo h, KA AAJ 10 +- PO CO NO KA KA

C.).— CA ro m m m m IS, rA m m to m n ,- a .-. n r, ts., m ts) -- m c...! m -- a n -- ro a- • • . • • • • . • • • • • . • • • • • • • • • . • • • • • • • It

M M M M M M 151 KA KA KA KA KA KA KA KA KA KA KA KA IS) KA KA KA KA KA KA KA KA KA KA KA KA I Il• IIII I I 1 1 1 I 1 I

r., a- uo uo a- ro KA r,. uo -- .-- If) ro If) +- a n co a- No 0.4 a Cr, r, I5 N CA UO Al- CA A- Lo

NI N) KA In a- .,..- No ro In CO KA CO KA KA N) a a- N) Cr- NO ,10 C'.4 1...) Cil a a- uo a- c...! m PI CO

aci -- NI m m -- m M — M S4 +- KA KA 15) 154 KA KA Cr, KA 'Si KA to KA KA KA KA KA KA KA 15, KA KA • • • • 1. • • • • • • • • • • • • • • • • • • • • • . • • . • •

CA Mt CD CA CA CA CA MA MA MA CA CA CA CA MA M CA CA CA CA 15) M MA MA CA MA MA MA KA KA KA KA

I 1 I 1 1 1 1 1 I 1 I 1 I III! I 1 1 1 I

N) In +- KA N) ro a- CA N. In 'Si ro a- co No — CN M a- Cr, a- ...- co CA uo a -.... a C.4 N) a m

C ..... co No a '0 a- rA a- CA In ,0 CA a- m co Ar a- a- Cl- c..! N. Cr. No r, -- CA a ...- N ,- IN

‹t VI CD CA N PI Mi a ,-. m ...-. M +- MA N +- CA CA 5i— MI NI CA ro es) m m m -- rq ro m m . • SI • • •••••• ••••••••

CA CA CA CA CA CA CA CA CA CA CA CD CA CA M MA +.- CA Mt CA CA CA MA CA CA CA CA CA CA CA M to I I I 1 1 I I I I

Page 177: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

177

r, D1 SO OS '40 N) +- +- In PO -o VI CA CA V, M In CO In VO M U7 CO PO Os r, -o co si a Cr- No

a Cr- -- Cr, r, co uo 1' 1' CO 1- M CO U7 CA r, PO CO CO so ro -- CO N Co up CO ,.r v, oq !ct- os --

cn n CO bl CA CO +- N) +- CA ,- +- In CO CO PO Cr- r CO in -.-- -,-- CO 1- Cc, Cl M ,-- U7 CA .- ro No

. • . • • • . . • • • • • • . • • . . . • . • . • . • • • • • .

CO n CA CA M CA CO CA CO CO CA CO Cl CA M CO CO CO CA CO CO CO CO CA MI CA MI fA CA CA M CA

I I 1 1 1 1 1 I I I 1 11111 1 ; I I

co .- CO ci, r, m r, — CO co Cr. to,-. CO co CO r, r4 N) b7 V.. CO CO 1- Os CA sO CA -,-- Cr. U7 PI CA

N) co N) CO r..1 cr CO N) co N N N) PO N CO ,0N to VON CO CO CO CA in Co ,0 CO N In Cr- -

C5 CO CO CA .- M U7 +- ,-- -!-- CA CA ro — CO Cr N) n CO N) CO +- 1- -- CA -- +- CA ro CO CO D7 N) • • . • . . • . • • . • • • .. • • • . • • • • • • • • I • • •

M M CA CO CO M CA M CO M CA CA M CA CA M M M M M CO CO CA M M M CA CA t CO MI M I 1 I 1 1 I I 1 I 1 I I I I I I I I

CO ro CO p, .q- If) -- ,- co co ,- uo -- -- co Cl C4 CA -q- a CO U7 Cc) D7 ,-- Os sO .- M CC) ro a

CA 00 •ct PO ,-- V, CA PO M ,0 +- PO 1- CO M M fA r, uo m — — -- No ro -- ro co ro n No n +- i to CA n — VI +- N) M M M - M CO M QA CA N) fA M M ro — c.4 n m m CO m m a M CO

• ••• • ••••••••••• •••••••• • •••••••

CO M n Si n M M CSI t M M M Cr ....... CO CO CAM SI M t n Si CO M CO CO M CO CO CO CO

i 11111 I I I I I I I I I

r, ro r4 Cl Cs sO CO CS CO M '40 ,... 0 T.. M CA Cr, sO VO M CO a- Cr- P7 N. ,-- V, 1- fA CA +- 1-

+- M ,- sO V, 1- CA CA VO M CO 1- sO M M CA 117 Cr, VO M U7 CA •-• Ul sO -,- +- sO ro co r4 CO r4

cn m m m ro c.4 (..4 CA CA CA +- .-- +- CA M ,- CO CA Cl MI ro r4 r4 -- — (-4 CA ,-- +- VO CA CA M

••••••.••••••••. •••••••••.•..••• CO CA CA CA M 15) M SI CA M M M .— M M M 15) M MI M M CA IS) M MI CO n CA M CO M M

1 1 1 I I I 1 1 1 I 1 I I 1 1 I

m r, Cl 0% Cr- VI sD 1" CA M -- M +- In Cr- V1 +- .4- -4- OD U7 OD 174 r, No OD V, OD PO CO Cr, OD

In CO U7 CA sO a ro 'q' N M OD OS SO a PI OD a CA CO CA V, 1"CO a- cl ro N. ....- No n oo cr

CD ,- M In r4 -- — a- -- -- CS -- Cs ,- M PO b7 ...- M In CA +- M '‘r- 4.- CA ,- CA sO M CA PO 'Tr • • • • • ••••••••••• • • • • • • • • • • • • • • • • u.

CO CO CO CO CO n CO M CO Cl CO M CO Si CO Car Si M CO Cl CO Cl M CO CO M Si CO CO CO M Cr 1-.

I 1 1 1 1 1 I I I 1111111 1 I i 1 to

r, (-4 CA CD VI CA V, In N. CA sO +- sO +- Os sO +- CO M M N. 'Cr M 00.. n SO rl Cc .,.. n '''''' T''' !il'i:

bo a a m CCD uo CA a — Cr- N. OD a N) PO IS) a a cl Cr- CA CO CO sO MI U7 '— sO sO N') N) a

z CO m — ro ro CO N) CO ..- CO CO +- -!-- ,-- Cl +- +- CO +- .- N) m 04 CO m .- 04 .- CO — CO ts, e-, • • • •• •

• • . • . • • • • • • • • • • • • • • • • • • • 11.;

Cl M M M KA CA CA M CA M M M M CA CA M MI M MI CA M M M 15) M Cl M CA MI M 15) 15) 4.,

I I I I I I I I I 1 1 1 I I I , 0 4-1

CO OD CO Cl CA N) V, sO U7 U7 CA CA CA OD CO VI CA CA MI CA +- V, Ul V7 N) N) in ,- OD M Os Cs

N. M M lr CO NI 00 .- CA +- Cr- 15) CO +- V, a uo ro CO a No — No (-4 — Cl-- uo IS) co r4 No r. al

A: CO 04 M M 151 Cl r- CO 15) r- r- M r- r- n M r- r- +- VD CO +- CA M M 15) ,--1

•••••••••••••••. • • • • • • • • • • • • • A • • al

M M CO CO CO CO CO M M +- M CA 15) 15) M CA 15) 15) M M M M M M M 15) Cl MI CA CA CA CO E I I 1 1 1 1 I 1 1 1 1 I

N) r,r, -qo -- M N a c.4 ro uo r, r4 CO co CO uo n a- r, uo r, In CO sO +- N. V, OS 0 N) ,.. ,^ 0

Ca? a cl ..-- r. r, ro in No c.. 4 — r, n No r, CO CA Cs V7 Cr, r... M qti ro a ro a or No — ro r4 4-

J .-- ..-- Cl N) CA CO .- M 0, M +- ,-- CA CO ...- Cl ...... CS? CA +- Cl T..... Si M In IS! CA CO '— m -- N) • . . • • . • . • • . • . • • . • • . • • . . • • • •••••• X

n M M M M M fA CA CA M M M M M M M CO M M CA CA CA M M tA CA M M CA M MM •■••4 I I I I 1 1 ; 1 1 1 1 1 I I

..Cr .q- co 04 Cl ro N) rq to 04 -0 uo Cr OS ... •-r CO Cr a Cr-. or r4 N SO I'l MR OD CO ro a In IT!

No a .,-.. No uo uo uo r4 In +- 1" a- rot„ OD 1' CO In fA a- n cl or n vo or uo ro oN ,0 r, ,- c E 1,-I CO r- 04 M n .- CO Cr, CO CA t5 +- M CO +- +- +- CA CA M Cc, CO M b7 15, CA Cci CA 15) CO +- CA I- ................................0 O M CA CA Cl M M MA m MA M rA in n m n CA M M CA CA M M M M M M CA M M CA fA Cl U

L.L. 1 i 1 1 1 i I ! 1 1 I I I 1 r- f,

.gi- Cr +- N +- 1" CA a r. r, No CO ,-- r. — uo CO CA CO OD sO sO CO CA Os sO ,-- CO D7 -0 Cl-- ..-1

+- 'Si N a ..-- Cr- CO U7 N) 00 CA N) CA N) V7 a CA +- CO V, 1.0 V7 VI CO U7 -/- N' ,- !gp ,.„1- Cr- v, r-

....:c a — ro In sO CO M CO +- ,.... V7 1' CA N) +- VO •,,r +- VO a '0 m CO +- CA +- CA 1' +- 1- ,-- ,- n • • • ••••••••••••• • • • • • • • •• •• • • • • • >

CO CO CO M M M ...... n M CO M CO M M CO M 15, CO M 'Si Cl CO KA fA CO M CO CA CO M CA M CI 1 I I 1 I I 1 I I I ! I 0

CA 1'40 h0 f'... Cs CO ,- VI r. ro CA NO sO In M NO a PO +- +- OD Cs M sO M VO MI Os (-4 r, a uo • .

r. a N Cl +- V7 Cr- uo r\ N) uo -a c.4 c.4 n uo (-4 No r. Cr, N. !..0 'JO CA 147 N) Cr- CA V, ,- CO +- al

CD +- M M +- +- Cs fA ,- M +- MI +- CA +- U7 +- M M +- fi? +- D7 M M M CA +- +- ,- .,- !q- cA r • • • •• • • • • • 4 • • • • • •••••••• le •• • • • • • ip

CA CO CO CO CA CO CO CO Cl M CO CO 15) CO CO CO CO MI CO 'Si CA 'Si Cl 'Si M CO CO CO CO 15) 'Si CO I I I I I I I I I I I I I I I I I I I I ▪ 5-

D7 Cr- a r„ a- or r. N) M fA VO Cs Cr, a r, No CO N) VO CA Os +- CA N. In 1- VO IS) r4 uo r4 N) ai

U7 CA r. No a -- — Uri r, m CO sO 1- V, Cr. CO VO Cr. CO M sO sO -- OD Cr sO CA OS ro uo cr r, Cr

IA. 04 -- !!.- 1- 'Si -,- sO CO CA ,-- ro — r4 ro 15 'Si N) I5 CO N) r, -- uo ,- ,- m (..4 -- -- .4- CO CO

•••••••••••••••. • • • • • . • • • • • • • • • • DO

CO CO M M +- CO CO CA CA M Cal fA M M CA 'Si CA CA CO M IS, MI CO CO 'Si 15) CA CO CO MI M Cl <C. I 1 I 1 1 I 1 1 1 I 1 I I I I

a!

CA cis up N v, v„ +- CA +- CA OD 0, CA V, 0, 0% +- %CI PO M D7 M ,0 Cs ro r4 cr ho uo

Cal r, CA N. SD CA a sD a m CA V, +- CA V, -- CO CA OD VI M 117 D7 0, CA CA M U7 .0

W +- Cl +- CO a uo CO ro ro N) CO CA M M 1- U7 -- CA CA ro r4 n t5) 17 h-

=I E ...... IN N) ...,r Z ...- (-4 ro Ar r- (C 04 La w U. CD 22 I-1 -J NC lt CD CV CV CV ci I. c 04 LI LW U. CD 2C 0-1 -.I DC DC CD CV CD CV CD 0 0 U. UL.

• • A • • • • • • • • • • • • • • • • • • 4 m • • • • • • • • •

M M 'Si CO M M M CO 'Si M M M M Cc M M CO CO CO M M MI CO CO CO M CO CO 'Si f'A M M I I I 1 I 1 I 1 1 I I I 1 1•

I

ct, r, ro uo a N) Cr- CA -0 CA CA CA CA M CO ,0 M CA M CA Cci N) U7 CO M a uo Cl a to N) a

(-4 ro -- cl N. r, r, — c.4 'St a uo No ro 03 Cr- 0% sD sD r, co uo h7 M r, ts) ro oN r4 — to ,--

C.) +- CO CO ,-- +- M VO CA CA ,-- .!-- U7 15) 15) CA U7 M CO Ul ,- -- CA 1- CA r4 n -- uo CO (-4 ro uo

. . • . • . • • • • • • . • . • . • • . • • • • • ••••4 am

Si Cl ,-. Cl Si SI CO CA n CO Si CO CO CA M CO CO CO MI MI M CO 154 15) 15) CO CO CO CO Is, Is, Is, I I I I I I I ! I I I I I I

un a r, 0- Cs N, a CO V, CO CA V, N) ro -- uo r, r, Cl-- r, Cr, 1., IS! CR ill CO M1 15) 151 ,0 in 1.0

SO 0.1 NO r, r4 a MI 1- •1- M a n — a ro -- uo m uo 1- VI ,-- Ul M ,0 M -- V, a to — (-4

gi m CO CO Cl "-- 'Si -- -,-- — — CO m CO 15. CO Ls, m ro CO CO ...-. 'Si 15) r.. M •-.. CO M IS) ',.. M CA • • a • • ••••••••••• ••••••••• EN • •11 NT•

CO .-MMIMIMIMCAMISIMMMMMM CO COCO CO CO CO M tS1 tSri Mt Si CA CO ts, ts, CO CO 1 1 1 1 1 1 I I I I I I I I I 1

+- U7 Cr-. CA bl CA a a N) CO N Cl N. M CO N. a N. SO N 0" Cr- 'T 00 CA N '0 M c.4 r, M %0

CS sD CS CO U7 V, '— -0 M V, U7 U7 CA CO CO 0, 0, N. a r, Cl -- No ro ro CA CA V, CA .!-- V7 L7

CO M M +- +- CA •-• .4' M +- M CO +- CA VP CA CA U7 CO +- M VI CA VI M CA •-• CA ,- CA V1 ,- CA ••••••• •••••••• ••••••• • ••0 • 111.9 • .--MAMMAMISIMMMMSIMMISIMM Si154 M M MASIMMMIMMMMIMMM

I 1 1 1 1 I I 1 1 1 1 1 1

Page 178: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

178

sO N) 4" CS NI In Cr. In T... Ch ....... co 4- 4- c..4 No r, 'Si CO CA C4 .1- ro =4- co r, No -ci No ,- r, CA •q" DO rA — VD N. ,- h, ,- CA N. 4- Cr- CA ,- Ch. ..-- 'Si CA 4- r, n No cr. r, ,:- PO CA -4- uo 'Si ro ro

Cl CO CO uo CO CO CA -- CA ro CO 'Si =4- CO CO ro No m CO In PA +- +- Cl +- CA MD CA In MA +.- 4- CN • • a • ■ • • • • • a • • • • • • • • • • a a • • 11••••••

CO CO CO CD CD CO CO CO CO CO CO CO CO CO CO CD CD 'Si CD CD CO CO Cl 'Si CO CD 'Si CD t 'Si CD CO

I I I I I I I I I I I I I

CO blr.1121 CA 4" sCi 4" +- Cr. +- Os +- Mr./ Cs +- C..I CD -.-..!" If) N. MI ND N. DO N. -0 CO •q• un N

PO NO +- 1,1 n Cl, CO ON I'N ro ,o PO CO CO CD DO ON CD CD CO DO ro -o PO h, NO In P, NO IN OD ro

CN ,- CD MO CA CD =r ,- ,- ,- m m ro m CD V1 PO CD CD PO CD ,- ro CA n mi n CA ro n Si O. -4-

•••••••••••••••• • • Olin% •11•1111•••••

M CO CO CO CO CO CO CO CO CO CO CO MA CO CO CO CO CO CO CO MA 'Si CO MD CO CO CO CO CO CO 'Si PA I I I I I 1 I I 1 1 I I I 1 I I I 1 I

P. sO PO ...- In N. WI In PO CO 'Si CO CO CO vo =4- 4- ,- 0, -0 Ch. Co -o =4- lo, CD DO N. cr. r... =r =-

C4 ..-- CD ,-- CA In +- +t SO +- CA NO 'Si CA '0 I', 4- -- CA ,-. NO h, CO ro -0 NI CO ,r N. =4- =4- r, CO G M +- CA +- =r ,- .4- m m m ,- (-.4 n =1.- m ,- ro mi r4 n PO +- 'T PA +- +- m ,- m o- m ,-

••••••• • . . • . • • • • • • • • • • • • . • • • • • . .

CO CO CO M CO MA MA CO CO 'Si CO CO M M MI CO CO CO CO CO CO CO PA MA CO CO 'Si CO CO M CO CO I I I 1 1 1 I 1 1 11111 1

C4 t!D =4- un CA CA CO CO VI CO •-- VI CO CO Cr. 'Si N. ,r CO ,- No CA Ch. 4- (Si QD ...- Cl- Ch. Ch. CO NO

+- CA 4" CA Cl I'l r". 4" 0, -40 CO ,0 ,C1 CO (Si CO IN 0, 4' ,C1 ,0 ..- CO CO Si =4. -0 1> uo lo- =r -0 un

Cl CO CO CO CA +- ,- .- CD ,... r..1 CO CO ro 'Si 'Si 'Si Si 'Si (Si (N ,- ,- CO 'Si -- PO +- MI CO CO PA PA

. • • • • • • • • • • • • • • • • • • • • • • • • • • • a . • •

CO CO CO PA CO CO CO CO CO (Si MD CO CO (Si CO CO CO M CO CO (Si Si M CO 'Si CO CO 'Si Cl CO CO 'Si

I I I I I I 1 I I I I I

M CO CA Cr- CO CN. +- CO Os +- CO co 4- -- ,- co ,- 4- CA CA ..--- ..-- Cl- CO ON 'rr (Si N. 0,- N. NO ND N N 0N ON ON (N — NI Cr- CO NO — ro ro No N. =4- Li) CO — 4- PO 4- ri -o ro IT N) un r, N. 4-

CO =- MA In +- +- +- ,:- (-1 m ,- ,-. No -- m PO In .=-. r.ID NO CA rd CD bl CA ,- CD CD CP. CD +.- ro 10 • • • • • • • • • ••••••• •••••••••••••••• ca

Ts:Isis-Inns, 'Si CO tSi nmin CO CO MtS1 ("Si 'Si CO sins: nmsins, (Si sins, CO .--..-.

I 1 i 1 1 1 1 I I I I 1 1 I 1 I I I I 0 5-

-40 n n ,- MI CO NO IN 1'... n CA IN rs... NO CA NO ,- CA 140 CA In n Mn IN =..... n =- n .-.. 0 IN -0 Cr,

CA ,- No un CA ON CD DO 4. mo ,- N sO OD CO *nl if) =1- N. N. =4- co CO n N. ro in Cr- Cr- ,r 'Si (N

z CD CD ,- MI CA +- CA MA CA MD CA M +- M CA +- 'Si 'Si +- CA ro ,- -- n ,- r.4 cs n -- m rq c..4 r-I • • . • • •• 1.• •••••• • ai• ••• • 1.• • a •••••• Ir.

M CO CO (Si 'Si CO tSi 'SiCD CO CO CD CO M CO CO CO M Si Si Si 'Si CO M 'Si CO CO CO CO CO CO (Si Si 4= I I I I I I 1 I I 1 I I I 11 I 1 I I I 0

4., N CO ."41" (N ,1- PO Ch- PO h, PO NO OD ,-- ON NO N. NO CA ON OD PO NO =r ND I.40 SiL uo ,r 'Si 'Si in r,

CA CO CO In sO ro 1- cr. PO ON DO PO +- PO LO CO +- Ni 'Si NO CO NO 'q" PI NO CO NO MI NO 'Si 1.41 N) I)) • ,- ,- CO (-.4 (Si 'N ,- CA CO PO — +..- CA CO ,- CO CD 'Si CD CA — 'Si ...... CA CD ON 'N CD PO ,- 'Si 'Si r.4

■ • • • • • •• ■ ••••••• • •• ■ ••••• ••••••• ill

M CO CO CO CO CO CO CO CO MA 'Si (Si CO CO CO CO 'Si CO (Si CO 'Si CO CO 'Si CO CO 'Si 'Si 'Si 'Si M CO E 1 1 1 I I 1 1 I I I I

!:-

CA VD CO PI VD 'Si CA PO ,-. NO Ci -0 N. 4,-. CA 0, CA h, PO CO OD h, N) Cr. CO N) ,-- O. (Si 0-.. N. 0) 0

PO NO N ....". M Ul In In =4- ,- n CA ,-. ..-- CA VD PO 4- N PO PO h, PO CD -0 -0 h, NO 4- PO ND ,r re_

.....1 'Si CO CA CA •-• CO CO 'Si r•0 =.- CO CA CA 'Si =-. CA MI M CA CA ,-- qD CD ,-. ON 'Si ,- ,-. ,-- ,- CD CA • . . • • . • • • . . • • . • • • • • • • • •ot ■ a I. ••• ■ 11

--

CO CO CO CO CO M CO 'Si M M M 'Si 'Si 'Si M 'Si SID 'Si 'Si 'Si 'Si CO CO 'Si CO 'Si CD (Si CO 'Si CD CD •,-f I I 1 I I i 1 I 1 1 1 I 1 I 1 1 5-

4.-' PP CO Cs CO Cs r, -0 CO ,C1 -0 N) CI-. N. If) DO Cr. N) N. N (Si ND 4- 4- 'Si -0 Ch. -0 N. CO '1' -4- N) .1- 113

M m n n CO rl m n ro IN No =I- No ro CO 00 =.- n CA Cr. CA .r.-. CA CO n n n CA n ,0 N. IN E E ,-4 M +- CA .h.- ,-. CD — DO 'Si ,-. CD ,- ,- CO CD (Si ,- CD — Si ,-. ,- ,..- CO — CA CD CA 'Si 'Si CD ..-- r- • • • • ••• • • ••••••• •s• • ••••• ■ •••••• a., O CO MD 'Si CO CO 'Si 'Si 'Si 'Si CO MI CO 'Si CO CO 'Si (Si CD Si CO CO CO 'Si CO CO 'Si Si (Si CD (Si 'Si CO lj IL 11 1 I 1 I I 1 1 1 i 1 I 1 I ■-•

'15

=r CO uo NO CA 'Si NO N. CO DO CD CA 0- CD 41' CO Cr- PO ,- DO NO CO NO (Si PO 4* Do 0, 0, ,0 ho N) .r.1

,0 DO 111 PI ,- VD N) Cr- CD NO CO CO 1.0 NO CO CA 4- un C../ ON Cl. Li) CO CA PO '1' 'Si =r Co ro PO 0-. 5- S MI CO =r- 4- un CO co CO CO — 'N 1- ,- CA ,- 4- ro n 4- N) DO 'Si 'Si ,- CO ,-- ,-.. VD 'Si -1- c-.4 rq (F.-. • • • • • • • • • • • . . • • . . • . • . . . • • • . . • • . • .7:-

CO CO CO CO CO CO 'Si CO CO (Si CO CO CO CO CO 'Si CO 'Si (Si 'Si CO 'Si ,- CO Cl 'Si CO n CO (Si (Si n C i i i i I 1 I 1 i i I I I 1 CI

Os r, In MD ..-- Cs sO CA in IN 4- co 4- DO CO DO -0 0., OD N. Cl, un co 4- r, -0 DO ,- CA CO 1-, ,r ..

,-. ,- un co No ro ro rq (Si ,- co =4- -- ,- CO CO CA Si N. DO PO CA DO ,-- N. -0 Co P.".1 tSi CD NO ND ci

co cs4 m m ,- ,- uo n m ,- mn 'Si CA +- =I- m —'S; CO ,- c.4 n n ,- n n ,- m ,- ,- PO +- 3 • • • • • • • • • • • • • ••• 11•••••• • ••••••• • 4J

CO CO CD CO CO CO CO 'Si CO CO CO CO CO 'Si CD CO CO CD CO 'Si CO ...- CO M CO 'Si 'Si 'Si tai (Si CO M I I 1 I I I I I I I I I I I I I 4-.'

..:-.

Os 0" CO un o co co cr. r,, ,- r, un CO CO COO CO CO ...- ro ,- cl- ••O 4- CO ro un ,- -0 0, DO CA !T1

CO ro co CA NO IN In 'Si IN NO CA IN Cr — N. =.-. N. -0 CO r, No ro 0, r„, 1.,0 (Si ,r cr ,- N. 1-4,) 'Si C).

U- NI +- +- In N. -- -0 CO CA (Si In +- CA In +- +- NI CO +- ro ,- rq uo ,- ,- ,- N) (-.4 -- r6 -- -- • • • . . • • • . . • • • • . • . . • . • • . • . • • • • . • • In

PA CO CO CO CO PA CO CO CO CO CO CO (Si CO CO M CO CO CO tSi +... CO CO 'Si 'Si CO 'Si 'Si 'Si CO 'Si (Si tZ I 1 I I 1 I 1 1 I I I I 1 I 1

al

N. N Cl 'Si CA +.- CO Ch. DO CA CD CO (Si =1- N N) ,- -43 CA CD NO N. un ro CO co CA CA +- '0 4" CA +-I

N. ...1- N CO CO 0... r..., 4" Cl, -4- Cr- CA DO ,- PO PO N. CD CO '0 N In Cs CN tO s0 N — "0 '0 If) N .0

UJ CO 'Si ,-- "4- PO CD 4" (Si ..--- ..-- ,- CA PO CO (Si ,- - 'Si (Si co ro ,- ro 'Si CA CA CA CA CA 'Si CD CO ill • . . • • • • • • . • • • • • • •••••••••••••••• I..

M CO M 'Si CO CO '5, CO (Si CO CO PA 'Si MD MD MD CO CO CO CO Si PA CO 'Si 'Si CO 'Si CO CO CO (Si CO I I I I I I 1 I I I I I 1 I I

• •

sO Cs CO PO ro ,- CA =4- r, m M ."4" PO .1" In In Cs 1.- sCi IN ,- co ,- ts4 ro a- ND CA CD CP.. C-../ co

=4- un No r, co N M M ro m CA CO PD PA +- sO OD +- 11- CO CD N Cl Cl N CO N CO '0 '... co =4- O — CO DO CO Cl ,- ro c.4 CA ,-- =-. In (Si CO PO In 'Si (Si CO MD +- PA =4- ,- rq CO ..-- No 'Si CA In In • • • • 11•• ••••• 4 • II • • •• • I.• • • • • •

M M M M M M M M M M M M MI M M M n M .-- M M LTD MI M M M M n M M ITD M I 1 1 I 1 1 I I I 1 I I 1

IN IN CO NO 1'0 n M 4- 'ql- CA co =4- -0 r.4 ro co =4- 0, ,- ND co 0, r0 CA N. CA CA ,r ,r ,- IN 'Si

N. CO No r, cs No ,- In ON ND =4- (N ro DO CA 'Si 'Si ,r ,- ro -o (Si un DO 'r$." N) =I- uo =4- IN CO CA

04 CO MO CO CD CO CO — CO (Si CO CA CA CO CD 'Si CD 'Si CO CD CO CD (Si CO (Si (Si CO CO CO 'Si Si Si CD

. . • • • • • 114•••••• • • •0••••• • •• •••••

CO MA CO CO CO CO CO CO CO CO (Si CO M CO 'Si CO CO (Si CO M M CO (Si M CO 'Si Si CO M CO 'Si i5,

I 1 I I I I I I I 11 1 I" I I I

4" N CD +- CO =4- un co CO CO ,- ,- Cr- c-.4 ri Lin (Si 4 cr. ,- (Si -o cr. N. CA -0 ,- +- N ,r ,- h{.,

O` In Oh ...... In CA CA CO CA DO .4- ,r- cs N. CO co ri CO co PL IN CA ,4" +.... NO +- In .1" Cs +- (Si PA

4X In CA (Si CA PO M .4- CO I'l M M CO Si 'Si ,...' CO +.- PO +- CO Si CO +- (Si In PA PA

a . • • * • •••••••••• • • • ••••• • •••••••

CO CO CO CA CO CO CD CA CO (Si CO CO CO CO CD Co-! ..-- (Si (Si CO CO CO (Si CO CO 'Si CO CO (Si CO CO (Si I I I I I I I I I

<E gc.4

1: -- 04 rt".1 4" :C .- r..1 N) .nr C. <C 40 C...1 im U_ CO MC D-4 ...J AC Nt Cl Cl Cl Cl Cl .."-. <C 44 C.) I.J.J U. CO MC 1-4 -4 AC NC CD Cl Cl Cl Cl 0 o U.. 14.

Page 179: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

179

r, PO CA ,t ,4" CA CD CD ,- 10 -7 Do CO ON M CO

r, No PO -0 N) m N) CA CO t r, ,T r4 7 -0 Cr r, 10 ,0 +- -0 CO +- CA 0, CD CA ro • ron c...! -0 o-. m If) +- +4" +4' h. CN CD CD 1.0 CD M CD CA CD +- CD 7 m -- PO OD +- CD m m m CA CD rl CD +- LID CD ro 10

•••••11•11•111111•11•11 111••••••••11111111• • •

M CD CD CD M CD M CD CD CD CD M I) CD C4 CD CD CD CD CD M CD CD M CD I CD CD CD CD CD CD

1 f 1 1 I I 1 1 1 I I I 1 I 1 1 1

PO CD CD CO ,0 CA 10 r. Cl CI AD CI ,0 PO CD PO r. CD "4' ,- ON OD CO r, CO IN '4- CO r, "4- r. 0,

ro N) 10 C. PO CI CD CD Cn. C, -7 co ro m m 0- PO CA -0 0, CO +- 1- N) N -- m m c.4 N -...1- -- CO

C5 CD Si CA 04 CA 7 M CD CA +- +- N 04 CD CO PO CD CD CA ,- rl rte) m m -- CA rte) N IN n Do ro • • • . . . . . . . • • • . . • • • • • • • • • . . • . • • . .

tst ta m m ta cst ta CA 1 Cad m 15) 15) 15) 154 CI ID ID IS) 154 ID C4 ID CI ID 15) 15) ID ID M ISI CD 1 1 1 1 11 1 1 1 I I 1 1 1 1 I 1 1 I

,T +- r, Cl 'AD rl CD 10 10 +- +- CD CD sO PO On 0, PO +- r, r4 r, rA CD PO -.0 7 0- (-4 co — ro

CA CO +- '4' Co 0, +- M CD C. 04 CO rte) cq ro m (N -7 CA n N) '4" CD ro to ND N) (5) N) 15) ,r ,_ If) O CA CD +- CD (N CD CD ..-- CD 15) 15) +- 15) C?- CD +- C.1 M *- M Cl M 0.1 M *- M M *- IS) .4" M M • • . • . • • . • . • • • • • • • • • • . • . • . • • • • • • •

IISI *A IfS4 m m m m 15) 154 m m m m is, m 154 si is, m CI 15) ID 15) MI 15) 154 IS) 15) 15) (Si IS) SI

I I I I I I I I I I I I I

a CA +- 0, IS) C?- 10 CO On +- a m ro sO CO +- CA ,1" CI CA +- CA C?- N) op NO CO CA CD CD CZ M 15) M PO CA (N +- CD CA +- CA 15 CI, CO CA CO

• CO ro ro ro uo No rq rq Do m vo r, ( PO if) 0, 7 ro Do -- C?- LP CD If) CO CO

MID PO N CD 10 CD CC +-

1111•••••••••••••• ••••111•111111111•1111••

m m m m m m m CO m (Si m m m m m m CO m m Is, m m est m s4 m cs, (54 m m s! si. 1 . 1 I I 1 I I i i I I 1 1 1

m C. CD CD 10 10 0, 00 ,0 CO DP NO CD CI ML? 7 uo 7 CO C, CA VD ,0 CD 0, C. 00 r N. N) CO

(N ,0 CD 10 CD PO ,0 '4" If) .!-- -so 0, -4) PO PO 0, 10 r C?- r, CD CA CO t 7 CD 1.0 C, r4 LP ,- 0,

CD ..-- m In M .,...- CO CA CD CA +- CD r, CD +- CA 7 ,--- m ro m -- m ro m CA CP CD "4- CO +- CD PO CD.

. • • • • • . • • • • • • • • . . • . • • • • • • • . . • • • • 1-,

ID ID m CO s! s! m CO cs! n CO CO ISI MI IS) M M4 (Si IS) MI CO M 1St M M M (Si 151 154 15) n (Si 0 1 1 1 1 1 I I I I I I I I I I I I I I I

r.r.'

a ON IN CA 0, +- CD CA CA On CO 10 ,t ,- SD -7 IN m 7- oo or C, UP NO CD CD 0, ,- CD sD to r4

CO J CC- (N — ,-- — ,-- (N o... No No — CO Co CA +- if) CA 7 ,- C?- or C, PO Co 7 m Do Do CN 1171 r-1

7gt M M M 1.40 C..1 Cl CA +- +- CD OD CD CA CD +- CD CD CD M CA CA +- +- CD CD CA CA ,- CA CD CD CD M . • . • • • • . • . • • . • •• 11114••••••••••11.11 ..f...1

M M CO CA CD CD CD CD CO CD CD CD M CD CO CD CO CD CO CD M CD CD CD CD 15) CD (Si CD CD CD CO ri 1 1 i• I I 1 t 1 i I I I I I I I I -,

N a co co m Lo a CO 11.1 a a, CO -- -- rte) Do m r, N ii, -- LI-) if) 15) (N CN CN .tr -- ON If) n Ill

m m m m oo No r, c-.4 -- OD C, +- CO CA 7 -- r4 -- No 7- r, 0., r, 1.0 c, so co Go 00 ,.. ,r co ri

AC (Si CO ..-- ...- 15) ,-..- M CA CD 0, 15) ,- ,- CD ,- ,- +- CD CO +- CO CO CO +- CO 04 (Si CO CA 15) M CD M

. • • • • • . • • .. • • • • . • • • • • • • • • • . • • . • • r m m m m 15! 151 m m m m 15! m m m 15) m m m m CO m 15) m CO m m CI s) m m m CO W

1 i 1 1 1 1 I I 1 1 i 1 t I It-

CO *- r", if) DP If) r. Cn On. 10 CA SO CN if) CD +- CA CO CO N) +- 0-. NO ND CD UP N CA CO ,!" if) 10 r-

V' CO V' C. r, 10 CD ,-. 7 -- IN 10 +- r. ON 00 Cl T.... ON 00 T." Ul MI 00 M.) CD CD +- On co -.41- u) CI

--I M M CO VI +- +- +- CD C, CD +- CA CA CD r4 ro CO M rq rl IN -- m m CD +- IN N — m r4 a 4-

. • • . • . • • • • . • • • . . • ••••••••••• ■ ••••

M CO M M CD CD CD CD CD CD CD Can, CD M CO CD 151 CO CO 15) CD CD CD CD CD CD CO CD CD CD CD CD >: 1 1 I 1 I 1 I I I I 1 1 1 • r-I

5-. .o.: CO 04 NO ,- 7- Do CI ,-- 0, m (....4 OD 00 Ul r. CD CD r. CA ,T CO CA CO CO +- ND 10 CO CO ,- r, CD 4-,

CI CO 00 C. M CD 7 0, ,-- rA +- ..q. ro m o-. -113 -.0 -4o C, OD 10 +- CI CO CD .c1- +- On bp '71 I-, co I7

E 1-4 T- M (Si CO CD CD M OD CD CA +- CD CO +- CO CD +- CO CD CO CD CD CD N) CO ,- CO CD 15) CO CD m r r- • • . • • • • . • • • . • • • . a Via M•Ig• • ••• • ■ •••

0 M M M CO M M CO CO 151 M !Si CD (Si (Si CD (Si CO CD CD CD CD CD CD CO CD (Si Cl CD CD CD CD 'Si 11! 11. I 1 1 I I 1 1 1 1 I 1 1 1 u

r

CO 7 N — n (..4 DP OD C. ..1-- MI C?- DD CD DD m — 00 CO C, NO N r, PO 0- rq 7 IS .c1- +- NO OD M

CO -0 IN CA PO DD CO 7- m r, -- ..0 o- m (5., -- Do m m 7- c..4 m Do Do r, ro r, Do r, O. 0- 0, •r 1

:c c...4 m CA 10 If) CD ON WA ,- SI r4 N -- N) m N ro m (N PO II) +- C, CD M +- CD ro — ro n n (-

CO

. . • • • • . . • • • — • • . — • . • • •••• • ••• • r5

M M M M CO M M M M M CD CD CD CD CD CD CD CD CD CD CD CD CD CD CD CO CO CD CD CD CO > I 1 1 I 1 1 1 1 1 1 1 I I ! I CI

LI

'41" CO CD -0 DP On CA DD DP DD +- Ill 0, CA CA CA OD Os CA UP 0, +- 7 IN r..., I."0 I.r.) OD .,:r ro -- r..!

tA CD CA 7- N NO DD CI 10 "CI *- !"0 C..1 *- N) ".10 M In *q" CO C..! Os r"... CA CA if) 10 CA "4" N) CD CO . .

CD ..- CO CO Cl C.4 CO 'Si CD +- ,- CA CO CA CD 7- CO m m ,-- N r4 7- m m m cl N CO r4 m 7 -- 0! • a • It a • •••••••••• ••••••••••••••• • r m m m m m m m m m m m m m m CO m m 151 m m CO m CO m CD CD CO CD CO CD 15) CO CI

II! I I I I I I I I 1 1 ! 1 1 I I 4-5

No 7- Do m 00 if) CO •q" 1.0 CD 0- 10 CD NO NO '/" CD 10 a rl r, 7 ro CO CD CO UP CD CA CD N) DP I-

7 CA NO +- CD N ro m r,. CI — m — 0, ro Do r, No rl 7- n Do Cr) oo r, m r, -o Do 7 LI) ,- 115 1.16. Cl ....- ....- CO'gr N ID M ,- M C.4 ,- C..I (-4 r4 m rq CO CO c4 No rq 7 CO m ca. • . • . • • . • • . • • • • • • . • • • • • • . • . • . . . • •

--------

CO M M M CO CO CO M M M CO 'Si M CO M M M (Si M CO 15) M MI M CD CD CD CD CD CO CD CD NO I I I I 1 1 I 1 I 1 1 1 I 1 1 <3.:

+- CO CA NO CD NO ,- +- DP 00 CA PO C, CD 00 7 7- ro (..4 ,..- CA CD DP 00 OD CI +- r, CO 00 m n al

N) 7- m N) — 7 CA C. N M IN LO rq CO PO N .q.- -- -CI -- -,..!- ot-. -- CA 1.0 +- LID CD CO CD ,- CD I-4 W CO M M or, 7 EN Do m ro — ro m ro m IN m — m m Do .1- IN 7- -- -- ro ro — ro — ...- rq .2:! • • •••••••••••••• ••••••••••••• • •• 117

CO CO CO 15) M CO MI SI CO CO M M CO CD CO CD CO CD CD CO CD 15) CO CD CO CD CD CD CO M CD CD F- 1 I I I 1 I 1 1 1 I I I I I I 1

a m ol, CA 10 CD CA ,0 C. CD r, CD +- r, CD CA +- 7 7 10 CO 1.0 NO 7- m m o- 0- NO CD ,- Cin

-0 CA ,0 M -0 CA r, CO "4" CD 0, CD N1 7 r, 0, n — r, (...4 r, n 1...0 CO 04 ..4- m CIO ,- ,- co !,,

C3 M M M M ,-- M CA M CD +- CD 10 CI +- rq Do ..-- m 7- m ,- m m M m m •-.. .4.. 154 N (N -,.4.. • • • • • • • • • • • • • • • • •••••••••11••••••

M ti) ...- M M M M fA M MI M M M M M M MI MI M M M n M MA M M M CO M M M M I I I 1 1- 1 1 I 1 I I 1 1 1 1 1

cl N) m CO 'qt CO 7- cl ,-- 7 or r, r4 -- m N) Do CD UP 0, if) 7 -- -- CA sO -O 7.- 001 Do On -Cl

CD NO CA ,t 04 CO NO CD CD CD ,4- NO 7- -- Do -0 7 — 7 No m Do ro r,. IN r4 r, 7- CA CD 1.0 10

04 CO CO- M M ..". M CO M CO MI M CO M CO 15) CO MI N) CD CD +- CO CD CO CD CD CO M +- CD CD (5:,

11••••••••••••••• ••••••••••• • • • • •

CO CO CO CO M M M 15) M CO CO CO M M CD M CD CO CD CD 15) CD CD CO CD CO CD CD CO CD CD (Si I I I 1 I I 1 I I I 1 I I I I

OS CA a .,-- ,0 -a CO N) CO CA a m a a m r, m ro 0' N '0 N. N C.4 ...10 1.54.1 C?- ,0 (....1 .0 On +- N CO NO 1"0 '7 M CO CI a ro CO rq c4 OD N1 0, 10 CD 7- 10 If) UD CO CA CO -0 N 7 (N m r, LID

.CE CO CD CO CD 04 +- (N +- CD CD CD +- (Si rA CO CO UP CD +- CO N CO CA CD +- ,- CO ,- CO cl. cD CD . . • • . • • . • • • . • . • • • • • • 4 II • • • •••411*•

CO CO CO CO MI CO M CO CO CO CO CO CO MI CO CO CO CO CO CO CO CO CO CO CO CO CO CO CA CA CO CO I I I I 1 I I I I I I I I

.eCC !MCI

E ...- c4 rya ..:r c -- C.4 NI ,:r r- <C aQ C.J UJ U. Co MC 1.4 ...1 A: = C) CD 07 Cl CD L <r OA LJ LW I.A. CD MG I-4 -A AC DC CD CN Cl CN CV O 0 Li. LL.

Page 180: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

180

,-- '-0 Os CO L) N CO MA In CO CA CO NO PO ON Os CA NO IN CO r ro cl ,-- cr. CO a- so a, CJ n ro •Zr In 1.171 r..m,- co CN CO in CO in amuo CON NN N.NNON CCI CO CO In in so romm a

I CO CO si- CA ,- ,- CO CO sr CO CA ro -- CO PO If) CO CO sr ,-- -- .,-- CO-- PO CO CA a •--. — a- co

•••••••••1•••••• •••••••■ • ■■ •••••••

CO CA CO CO MA MA CA MA MA CO CO CO CO CO CO CO CO CO CO MA CO CO CO CO CO CO CO CO CO ISA CO CO I I 1 I 1 I 1 I 1 I I 1 I I I

0, CI, ...... CA rl r- NO r, in In 10 po ,- ,- p.... c..) Ao Cs sr CA CA N ,4.- r, r, P, -0 AD AO sr ro m

po r, in co ,- in CO CN r, ..4- ,4.- Cl. — co -- — a- CO CO In NO OD JO CA CO N CO in CO CO CA -.T. In CD Cl CA CA .!-- ,- a- n n rq <5.4 m ro <34 m uo ro m n ro n — ro — m m CA rq ro -- m a a • • • • • • • • . • • . • . • . • • • . . . • . • . . . . . . .

CO MI CO CA CO CO CO CA Cl CO CO CO CO CO CO CA CO CO CO CO CO CO MA MA CO CO CO CO CO CO CO CO I I 1 1 I 1 I I I I I I I I I I I I I

AO VI PI OD PO NI -- w- a- Cr. so r, m co a. a- — r, NI NO NO CA MA CA sr CA ro m ro In .,-.1- cl

N CO CO .--M-4- InCIN NI OD ,--• in in In .ffi- A-- sr a -- ,- -.0 — No sr If) in m r.0 CO 0.1 -.0 N! CA

OD sr CO CA ,- PO CO PO CO CO CO CO +- CA sr CO •-• NO CO CA ,-.- NI CO N) CO CA -- CO CA CO Os CO -- •II•••••••••••••• •••••••••••• ■■ •••

CO CO CO CO CO CO CO CO CO MA CA CO CO CO MA CO CO MA MA CO CO CO CO CO MA CA CO CO CO MA MA CO 111111 I I 1 I MI I 1

CA OD AO CO CA a- a- CO CO ,-- CA PO in CA IN N. In NO CO -- CO PO CA NO ,-- -- NO PO In PO AD 0...

..-- CA CA r- ro in a- N. In Cs CO In CA MAMA N ,-- ,- a- r, N. a- N) NO in CA CA In (' <31 r..! n ro

cr. n — n N.) r- CA .-- CA -,-. CA CA MA LOMA CA CA ,-CA m po ,- c,) ,- m ,- .-4- — ts, — r..7..,,, — -- ■ ••••• ■■ •Issu ••••• •••••••• ■••••••••

M M M CO CO CO CO M CO CO CO CO CO CO M CO CO CO CO CO CO CA CO CO CA CA CO CO -!-- CO n CO I I I I I I I I I 1 I

NO 'A' ON N CO CO CO co N sr +.- CO NO, 0) sr CO In CO -0 NO 0- In CO In so co in N) CO -o -so

-q. .q.- co m AO CA UO Os +- CO MA V, In NO CA -- CO ro r, n m m rq co ro CA N CA MA v.:A MA -0

CD +- CO a- -- — CO N) CO CA CO - a- CO -- CA in +- CO -a- -- rq CO a- CO rq CO CO CO CA CA N) a- G.

.. • . . . • • . . • . • . . • • • • • • • • • • . • • • • • • . 11

Mt MA Cl CA MA MA MA MA CA CA CA M CA n MA MA MA CA CA MA MA CO CA CA mA CA MA MA MA MA M CA lo I I I I I I I I I I I I I I I I I I I I r-

Cr,

Os AD Os +- VI In at uo c....1 a a N. rq a- at PO OD CA -,-- Cr. MA PO ro -o r.„ in uo CO so ro so a-

N r, m uo N UO N ,--. MA OD a- uo a m CO uo a- AD Os In ..-- VI CA CA CA LI-A OD r, In VI In In ,--4

Z CO CO ...- NI +- CA CO CO CA CO CA CO +- CO NI +- CO CO +- CA CA CA CO CO ,-- CA CO M +- CO N CA M ▪ •••••••••• •• ■••• ••••••••••• ■ •••• ..I.'

CO CO CO CO CO CO CO CO CO CO CO CO CO CO CO CO CO CO CO CO CO CO CO CO CO CO CO CO CO CO CO CO in I I I t 1 I I I I 1 I t i I I I I .-!-:.

CO AD CO VI CO VI CA AO In Os CO Os AD NO IN CA PO AO +- ,- UD NO In Os CA V, In AD +- rq N. n ri)

AD CA a- — CO In PO a ro AO OD NI Os NO CO CO N .:1- CO a- r,. OD VI In 'tr 'AO In rq CO CO CO uo .--1

1: — CO (Si NO +- CA +- +- ,- CA CA CO +- CO CA CO MA MA ,-- NO CO ,- ,-- — (Si Cr, CA Si a- — V.,1 CO M ■ ••••••••••••••• ••••••••••••••114 P.

CO CO Si CO SiC CO MA MA MA CO CA CA CO CO CO CO MA CO Si CO CO CO CO CO CO CO MA CO CO CO CO CO W

I I I I I I I I I I I I I 4-

•O CA MA OD MA N Os +- MA CA MA MA In PO CO NO N MA ND NO Os AD CA MA MA rq r, uo ,-. a- r, a -

CA CA CA In N, CA V, ro n r, PO ar ,- so ,- 0, T... Cl N. MA PO MA CA OD CO sr CA NO CA In N MA 0

....1 +- MA PO +- CO CO CO CO NO CO CO N +- +- +- CA +- CA CA CA CO CO - MA CO CO +- CA - EN CO ro 4- • • • • • • • • • • • • • . . • • • ••••••••••• ■ ••

M M M M M M MI M M M MI CA MA MA MA MA MA MA MA MA MA MA MA MA MA MA MA M M MA MA CA X I I I 1 I I I I I I 1 I I 1 1 I I • r.4

5- 04 CA .).- *r co co r.... NO CO NO CA NO AD PO PO N MA a-- .,--. OD OD NO OD N MA MA Os AD CO AO CA N *-- -1-:,

CA N, CO CA OD N In OD CO VI OS a- in No rl ,-- 0-. CO +.- NO at In In Sr CO In CA CO In In CO CO M

E 1.4 CO CO CO — CO CO CO In Si — CO CO — CO CO CO CA CO CO ,-... •-• MA CO 00 CO -,-- MA Si CO CO MA CO r • • • • • • . • • ••••••• ••••••••••••••• ■• 0 M CO CO (Si CO (Si CO CO CO CO M CO CO MA CO (Si CO CO CO CO CO (Si CO MA CA CO CO CO CO n m n CI IA. I I I . 1 I I I 1 I I 1 I I I I I LI

r

N. 1..... AO NO NO a- r,. ro so uo uo ro ro IN ro — ro m r, c-,1 r, OD a- r, rq in ro uo 17.-.! n -a rq ri).

co No ro ,- in r, in PO CO N Cs (Si N) PO PO N a- ro uo CO a <34 ro uo rq ro <31 CA NO sr CA CCD •H

7C N (Si NO a- .4.. CO r, CO n m -- ND -- c..1 n cq ro m CA NO a- — cr. CO — — m a- *-- ro -- -- r-

. ••••••••••••••• •••••••••••••••11 IT"

CO (Si CO CO CO CO CO CO CO CO CO CO CO CO (Si CO CO CO MA CO CO CO SiC CO CO CO CO (Si CO CO MA CO I I I 1 I I I 1 I I 1 I I I 0

r, a. uo m I'— N N Os In ON C Os Ncarte C

-a O C

V, In MA Os NO Os CA ,- In Os Os CA sr MA a so — a r, a- a uo n uo m co uo n No ro vo o- ..

Co CO CO Si rq N' ,-- m — m ,-- m r-.1 m ro n CO m (Si rq rq o- — n m ,- CA MA CA CA po -- 0

•••••••••••••••• • • • ••••••••••••• 3

CO CO CO CO CO CO (Si CO CA MA Si CA CO CO CO CO CO MA CA CO CO CA (Si MA CO CO CO CO (Si (Si CO CO 1..' I I I I I I I I I I I I I I I I I

-I--'

*47 1:71 NI CA P, CIA NO PO -.- ,- Cs V, -a cl 0, ,-. r, NO -a — — CA V, ro a in CA AD MA AO MA PO

L7 MA MA N V CO CA CA In +- P, .-- MA Os .q4.- -- AO AO In sr Li ro Cr, a a- ro r, -- CO a- -- CO CA M

Ia. CA -,-- +-- sr AO CA In CA CA MA CA ,-- +.- CA CA MA CA MA MA CA OS rq a — m MA CA 1:-.4 — ro — — La

. • . . • ••••••••••• amtles•••••••••••

CO CO CO CO CO CO CO CO CO CO CO MI CO CA CO Si CO MA MA CO CO CO CO MA MI CA CO CO MA CO MA MA Ao I I I 1 I I I I I I I I I ! I I SE

r, a uo — rq uo r, a- ro In OD OD NO N ,- AO NO UO Os -a -- rl rl co -o -- a AD ,-- NI CA fA W

UO NO CA -- sr MA a co CO a a r, un ro co r..! N. ..-- N rq uo a- m -o <2, a- uo <5.4 r., so so rq --,

LLI MA MI CI In rl rq ro n CA -!-- CA CA PO MA - MA MA MA MA Os CA CA PO -,--- CA NO CA =- NO ,-- MA -!-- ..!D

•••••••••••••••• •••••••••••••••• IT.I

CO CO (Si CO CO CO Si CO CO CO CO CO CO CO CO MA CO MA CO CO CO CO CO CO CO MA CO CO CO CO MA CA V- I I I I I I I I i I I I I I

Or in a N a- N CO r%1 co cl a 'r CO ,- ,r. N) -- c..1 17.4 0, ,r a N. CO -O ,- ,- CO co N) ,r r,

a- a r, NO CA a <3.4 CN CS '0 C.4 CS MI S4 OS U7I L171 I.0 n r, .-4. r, U7 T.- r, n CS N IN ,-- In r,

C.) — MA a- CO CO — (-1 m rq CO CO ro CO — rq a — n oo CO CO CO N CO CA +- -,-- a- CO N ro a

. • • • • • • • • . • • • . • • . • • • .... • • ••• ■ ••

CO CO MA CA CO MA CO CO (Si MA CO CO MA CO CO CO MA (Si MA CA CO MA CO CO CO MA CO MA CO CO MA CA I I I III ! I I I I I I I I I

NO ro a- NI in Os OD r'N CO r.... MA In =- PO MA AD CA AO CA In NO ,---. MA .-- MA AO CA In ro r-, a so

CA — ,-- — so uo m so -- — uo a rd CA NO PO NO NO In -- In a ro n IN a -o ro a- — co c-.4

04 CO VI CA CO CO CO CO CO CO CO CO MA CO CO CO CO CO CO MA MA (Si (Si CO CO CO MA CO MA CO CO CO CO

•••••••••••••••• IlMom••■ ••••• ■ ••••

CO CO CO CO CO CO MA (5) MA (Si CO CO CO Si< CO CA (Si CO CO CO CO CO CO Si MA (5) MA CA CO CO CO fa I I 11111 I I 1 I I I I I I

CO VI +- sr NI OD .,-. NO CA CO N -a -1.. c, r, N. N. c.-.4 — so r, r, ro a N. NO CO MA In ,- No co

In a m a !...,. m uo so rq cl — uo N. a- rq --- m ro in r, so — a a ,..- r, a oo — a- m rq

<E li, M ..- ,- EN M In -.-- M ..- M ..-- M c...1 m •-• a n — n rq m ro n ,-. m m — ,--. ro CO CO

•••••••••••••••• ••••••••••••••••

CO CO MA MA CO CO CO M CO CO MA CO CO CO 'Si CO CO MA CO (Si M CO CO CO CA (Si CO CO CA CO CO CO I I I I I I I I I I

SE 04

E +- NI NO sr c +- CA Po ,r. 5- <C 04 C.) WW WI. Co MC I-, -.) AC AC cD C CA CA CA C. <r 04 C.) fA) I.A. CO MC 0.-4 ...) C MC CD CA CA CA Cu 0 CI W6 WL

Page 181: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

181

C51 ,- r, .cs- co N uo r, N co 4- ,o CR No PO -,-.. N. ,- CO SI NO ,..- Is. 'q" 0' MI 0. M4 el uo co -0

4- *-- -0 in CA N 0- n -0 N7 't C. ' -43 N rn N 4- Cn.I N 0, CO 4" N. M MI Mt N 4- MI ,CI N Co

cn -- m so rn m m ro ,- ro -- m NO MA *- 4- os -- m uo rn m rn ro -- 0.1 rn -- so m -- .v- so • • • • • • ••••••• •• • ••• • •••• ••••••• •

M M M M M M n M to n M t CR CR CR CR CR CR CR CR CR CR CR CR CR CR CR CR CR CR CR M 1111111 I 1 1 1 11111 1 I 1

M N. CA D7 N C) LO 1.11 PO NO r. '0 N CO ft- MO CO 00 Cl 'AO 0 CR VI 0 VO ro Ni C) ro ,r 0.1 CA

MI ,-- 'T M ON 0, CN r., IS4 .,.r co ,0 rq m ,- op po 4. .,..- m rq ,- r, ,- ro rn CA 't D1 MN MO 0, -0

Cr fl.- m vl ,- ,- v m ,- Cl m .- N, 04 C) os 4- rn m 4- ,..- 04 vo .-- m -- CA NO ro c- m If 4' 1111•••••• •••••• • • ••• •••••••••• • • •

CR CR CR CR CR CR CR CR CR CR CR CR CR CR CR n M M M M M M Ma MN MI MA MN M M MA M MA

1 1 1 1 1 1 1 I 1 1 I I I I 1 1 f

CO ND D1 ND r, CA ,o .q. -- uo rn r, uo 0.1 co ro -- ..,..r CA VO CA Cs MI rq c, -- ro el ,f1- CA NO ND

Cl Os M M M CA — Os CO CA -- n ,-- 'qt .7:- -- N co rn ,- ,- as CA .t3- n oo so rn 04 vo Cs) -- -..r

cn N m -- ,- ro C) Cl m rn rn — -- m cs rn ,- 04 rn ,- fiff CA M rl CS) m m rn -- rn uo rn rn • • • • • ••••••••••• ••••• ■ •••••••• • •

CR CR CR CR CR CR CR CR CR CR CR CR 154 CR CR CR CR CR CR CR CR CR CR 15) CR CR CR CR CR CR CR CR I 1 1 1 1 1 I I I 1 1 1 1 1 I

co .nr so ro CS) N (5) CA CA VI CI CA Os If) (N ON 1.11 CO 00 Os N7 N. r, OD M NO cr r„-- .q- 4- 0-

- CO r4 NO .-- ro IN el r, so CA 4- cr 04 -,r rn so -0 -- N vo .cr r, ro 0- ,- Ni r, uo 0.1 cl -0 — O CR CR CR 4” CA el CA ,- Cl CA CA .,- CR CR CA CR CR CR CR -cr 04 N ,- CA CA CA ,-. ff--. If) .f.- m rn • . • . . • •••••••••• ••• •0 I ••••••• II • •

M MA n M M M MA M M4 M MN MI ,-- M4 M4 M M M M M M M MA M M4 MI M M CA) n M

I I I I I I I 1 I I I I 1 1

CA 0 4- CA 4- r, NI N. CN OD N. CO 04 r, so so ho 4- ro r4 OD Os NO 121 CA CO OD ,- rl ND r, el

M r, co r, 4- r, co rl m co ,r 4- 4- ,- CA MN CO MN "ft CO MD M4 rl PO CO OD r, NI 'cr OD 0- CA O ,- CA 0 ,- ,- CR VO ,- CA ,- ,- ON ,- ,- tO ,10 ,-- CR 0 ,- ,- CR 4' ,- ,- CR CR -40 CS! ...... ro uo • . . • . • • . • . • . • • • • . • . • • • • • • ••••• • • CI

M M 15) St n M M M M (Si M M M M MI 45) MN M n Mt CR CR m m M., 15) M M rn In rn rn 7-., i i l i Iii I I I I I I I 1 I 1 I I 0

O4 r, -0 0- ro ro r, m uo uo N P. VO MI r., 4- c, N N. TT if) ,-- r. (N .q• .cr N N) so IN so r..! crr

0- -- ,- OD ,--- N. CN. CO In CO -nr .a- -4- m so 0, ,- rn so ,- CC- 0-- -i- LO 00 N. CO CO Cs ND CO --

= CR CR ,- V0 VO ,- CA ,- ,- CR ON ,- CA ,- ,- CR CR CR CR NI CA ,-- CA ,- M CA CA ,- -f-- M MN M4 f--, • • . • • • • • . • • • • . • • • • • • • • • • • • • • . • .• fil

M M MA CA MN M M M M M M M MI M M M MI M4 MN n M M MN M MN n MN MN M M n M4 H.:, 1111 1 1 1 1 1 I i I I I I I 0

O 4 1.11 N rn ro N. rn cs uo uo co N) uo so co CA .-- ON ON CO CO N. In M 1.0 NO D7 CO CA ,r .1-

CA N NO M r. NO CA NO MO CO 00 CO CA ,-- OD -4- r, ro r, CA -4- co -- cll., CO 0.1 uo rn -- .1- 04 rn u) • CR CR ,- ,- CR ,- ,.. ,-- C4 0, CR ,- CA CR MI .,.-- n n M -f--- MA M4 ,-- MA M MI M -- in m4 M ,— ..f...1 • • • . • • • . . • • . • . • • . • • • . • • • 1. • • • • • • 4

M M M M M M M M M M CA M M MN MN MI M MA M MI MI 15) M MI MN 15'4 15) M MN M Mt M 11", 1 1 I 1 1 I 1 1 1 I I I 1 1

::-

4- co vo rs .q. CA 1.0 ,-- 1.11 Cs 1.17 Cr CA ,-- NO CA CO MO CA r, ,- NO CA MO M7 ,.... -.:r rn so 0) CD MA ID

NO CO CO r. ON ,- CR 0 CR PO 111 MI -0 04 .q. ro rn uo CO M -0 ,-- CS) •••fr 1.11 CA ,-- NO r--. MN CO -- cf-

-J CR CR CA 1'0 ,- ,- ,- CR CR CR ,- CA CA M CA rte) M4 M4 CA MO CA ,-- M ,-- ro -- 04 -- -- ,- -- ,r • . • • • • . • • • • . • • • • • • • . • • . • • • • • • •• • X

CR CR CR M M M 45) to ,- 15) (5) M M4 (5) n M M M MA M MA MN M4 n (5) M n MN MN Mt M4 MN •.--1 I I I 1 1 1 1 I 1 1 I 1 1 1 1 :..-

14 ¢ rn e, IN m -- cs N. -4- ,- mf mf r, CA .q- 1.11 r. ND r, ,..- NI 1.11 CO CA CO NO CO r, ,-- CO 1.17 r, Cs fr.',

-4- 00 ,-- VO r, NO .7r 4' Ll PO CO CA l', CO CR -0 04 ro r, vo uo ,1.. uo -- to 04 ro rl pp n ts4 ,0 £

E 1-.1 CR CR ,- CA CR CR ,- 0, CR ,- ,- ,- ,- M4 ,- ,- CSI CR ,- CA ,- CR ,- -a- ,- -- rn 04 m rn ,-- ,- 5- • • • • • • • • • • • • • • • • • • • • •••• • ••••• • • al

0 MI MI M M M MI M MI M MN M MA CR CR CR CR CR CR CR CR M4 t4 M MA M SI C) M M n MA f4 U UL I I 1 1 1 I I I 1 1 I I 1 I 5--

ai

4- r, -- 04 -- uo r, r..... in N. N. 1'0 M NO b1 111 No .i- uo -- 00 m uo 00 0.1 m CS Lo (.4 ,0 ts, 0, .,-,

0, V0 R. -0 -0 MN MO 4- rn 04 cs Cr! IN lo- r, m T.; ;;, P '2' :*. ;i2 ;.1 "I' cq cl 6 ',1:). Cl 4 .4-1 Q7.1

f....

:C CA MA MO Ll L7 M M4 •-• ,-- ,..- CA NO CA CA M ro 175

111••••••••• ••• • • • •••• ft •••••••• • • >

MA M M M M M T- M in M M MA M M M M M n M n (5) M) C5) M fiff M M M M (5) MA M CO 1 I i I I I I I I I I I I I I C:1

-0 ,0 4' M Cs ro MO 0, CA VD MO Ns r., 04 . IN 0, LI VD 0- 0, r, MI SO 1.0 CO 0-. n Ol .1. M rn ..

-4- rn 01 0, 0, CA M ,0 ,- ,0 l',.. r, CA .s- 0, OS 01 TT CO ,0 CD CO ,0 ,- Vl 0, 4- r, r, -0 ..,.1- CR Ill

CD *- M M — ,-- CI, n M .-- — ,-- M04 rn -zr C) CS) rn — -- 04 4- m rn Si 04 elrn 04 is..! 4- IN r- • • • • • a • • ••••••• • • •• •••••••• • •• Me b Mt Mt M M M M MI M CR CR M M M (Si M M MN M4 M M CA M (5) MN M Cif M MA M MA MI CS)

1 1 1 tIll 1 I 1 I I 1 I I I 1 I I 1 44

PP) N. .-- 0 CO ON ,- ,- .T r N) .q. rn r, rq co 0, ND CO No .,-.. m 0, -- m 44- co 0, C'- m -- co ',7;

M 0- 0- ro ro CC- so r, 0- r, ,- ...r. ro CA Os N. U1 CA NO CO CO 'q• CO MO ,-- CO co N. CA ,- 4.- IT. O

U. 04 m — 4- ON ,- 0 MP .-- MA MO ,- CA MO ,-- MN CA M M CA ND CA -i- -- m rn -- -- -- .,t. -- rn • . • • • . . • • • . • . • • • • • . . • • . • ••• •• • v.

CO CR CA CP CR CR CR M CR CR CR m m m m m CA CR CR CR MI M M M M M M MI M n M MN -tr I I I I I I I I I I I I I I I I

CU

Ll .g• OD -0 1.0 M CA St N. r, Cs rl ro ND 111 *q" MO NO NO MN ,- r.... 0.1 so co N. CO CO TT NO CT, !-r) ,-.1

M MO r, ,0 VO 0, ,0 N1 r, MN CO r, -- m CN C.4 CI, 1"...1 NI CN MO CA NO ,-- -0 0, .1- 04 r, r, 4- - W CR CR C5) O. 4- — uo CA NO ,-- VD — '4" ,- ,- MA M M MN 121 U1 CA 4- N 04 01 ro (N el rn rn -- 115 • • • • . • . • • • •••• • • ••• V • ••••••• •• • • 1...

CA CR CR CP CR C4 CR CR CR CR CR CR CR CR CR CR CR CR CP C4 CR CR CR CA CR CI CR CR CR CR CA CR I I I I I I I I I I I 1 I 1 1 1

0 ,- 4- co ,- -I- -- CA NO 1.11 so 4- ,o vo 04 N. 4- co co so ro 04 ro ml uo Ni ro r, MO Mo r, Cs.,

04 rn r, r, cs 04 r, -- co so ,- OD ,0 MI 114 L7 .qr. (N -0— r, ro ro U1 ,-- ,1- rn 0, n ri —L!

GJ ,- CR CR CA ,- CR ro ,- CA •-• *-- U1 SI •-• tO ,0 .-- M D1 .- --- MN 'T ,-- PO M ,-. 141 M4 CA ro uo

••• •••• •••• •••• • ••• • ••••• ••••• • •

M4 MA ,-- M M Mt m m rn m rn rn CR CR CR n MI M MI M MI CR C4 CR CR CR C4 CR CR CR CR CR I I I I I I I I I I I I I I I I

iri CO ,... 4.- N so r, 0, CO .q- r, Ll. 4.. -0 r, ,- (N Cr- so -- ,- .q. ro uo CO so ro ,- so 15 r, ,r

C51 rn rn ro 0, ro NO CO CO r"... T.. N. C..I n .q- so 'T CA NO ro rn co rn co -- uo so 04 rn to r, <I-

x's; es! m m 1:5) n MI Is) M M CS) in MI Mt Mt Mt 15) PO Mt C) ,- n n Mt n Mt Mt n — n n CS) • • • • • • • • • • • • • • • • • • • • • • • • • • • • ORO •

MI - M M M MI M M MI MA MI M MN CM MI M M 154 MN MN St MI ES) M M MN MI MN MN M M4 n 1 1 1 I I 1 1 1 1 1 I I 1 I I

U1 1.11 U7 Ul VI -0 4- n 4- LI CA CA CO CD M CA 0- MO ,-- 0, V0 CR L7 4- cis Ci- C) 00 ro 4- cl, r,

CA M N M N) 4' CC- ‘-r- so 04 Cl- ro co Cl- — ..... CN In M r, 0.1 4- m ,-- 04 ro 04 uo os m ro uo ¢ 0, m -- m cl ,- CA CR (Si CR CR ,- CR CA ,- ,- 4' CR CA CR CA ,- MO M ,-- CA ,-- ,- MA -r -- m

. • • • • . . • • • • • . • • • • • • . • • • • • • . • • • • •

CR CD CM CA CR C4 CR (Si CR CR CR CA CR CR CR C4 C) M CA CR CR CR CR CR CR CR C4 CR CA (5) CR CR

I 1 I 1 I 1 I I I I I I I I I

Cl, IN11 •Zr

..

4CKt ILJ 11.1 1..1 J i 1:11 0i •Cr LU CO Of C23 0 0

Page 182: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

182

rN .73- cil NI Co NO ON 0, M 'r CA CA ON -o CA -o CA NO 00 0, CA CA In N) In OD In r- n C4 1- MI

c' L7 ,-4- kr) ,- 0, m co -O „-- es, ,- IN ,-- cr -0 co Lo In co Ls, - al N In In in -0 t Cr, PO 'AD M CN CA CA

In r- CA CA r- r- 1" .- to In Co Co cr '0 Co Co un Co ,-- - CA ,- Cl Co IN ,0 m -- .-4. m

•••• • • • ••• • ••••• • • 0 • 4 0 • • • • • II • • • • M M M M Co M M M Co M M M CA CA CA CA CA CA CA CA CA CA CA CA CA CA to M MCA M ,--

I 1 1 f I I I I I I I I I I

ON N. N, ON r- M M N, OD cr sO IN cr ro cq co m - r, co cq - co ro ..t CA cr so n ...- ,c1 cr

M r0 r, . .7 cr cr so Co CO c.1 co oN so "- Cr, CA c.J un - Co D7 r..... CO NI CA C'. In EN CA 1.0 Cl ..40 G .... Cl NO Co ..-- cr - - - Cl m ro Co Co In '4' Co Co cr Co ,- ro ,-. - Co - rq PO +- n C cr • • • • ••• • It • •••••• • • •••••• •• • II• • . •

CA CA CA CA CA CA CA CA CA CA CA CA CA CA CA Co Co Co CA Co Co CA CA Co CA CA CA CA CA CA r- CA I I 1 1 I 1 I I I I I 1 I 1 1 1 I I

1- Co b1 -0 Co 1' -0 In 03 CA CA sO cr CJ cr un ro CA ro so cr cr. cr Co cl - CO so -- C'. - IN

IN Co ro cl r, - sD CA Co Co .74• sO 00 rq Co D7 '0 NI ,- N. - ro so 00 CO "0 ,- CA Co 0, CA In ro

C5 'qt Co (.4 Co cr Co cr Co ,- Co Co - - un Co - ro Co IN - ro Co PO M IN -- n cl ts! n Co r- • . ••••• • ••• ••••• ••••••••••••• • . •

CA CA Co CA Co Co CA Co CA CA Co CA CA Co Co Co m Co Co M Co ta ta Co M Co n Cal to ....- ta M

[Mill I I i 1 I I I I I 1

NI -0 In r- 0- IN cq CC' sO 00 sO CA r- cr ro c..) PO cr to so 0-. 'q- SO "40 SO Co N r, N) ,- IS, Co

r- 0, M M N. CA r. C') N) N. r- cl... cr IN ro Co Co cr un L1 sO CA PO OD U7 CA r- MI 1' PO C'- CA CO

CN CA r- CA CA r- CA r- CA r- NO r- CA VD CA CA CA CA CA CA N1 r- CA CA CA ,- cr cl n n ts! - ts! • • • • • • . • • • . ••••• • ••••••••• ••••• •

CA CA CA CA CA CA CA MA M M M CA CA Co LA Co Co Co CA CA Co CA CA M4 Co Co M M r- IS MI Co

I I I 1 I I I i I I I I

CO r- N, CO 0, M U7 r- Co D7 NO r,„ CA n Co • CJ cr. CA IN P, N) CA -O Co CO P0 uo CA un cr

CD r- CA In CA r- CA 1- CA r- r- r- r- r- 141

PO NO ON CO NI Co cr ro ch. (-1 NO M N SO SO ,- Cr. cr ts, ro cr ro cr so in OD N cr n R.. n

Co so CA CA in CA r- CA CA Ce, CA CA r0 .40 • • •••••••• • •nle• M •01••••• • ••• ••• • • C.-1

Co M to MI M M Co Co CA Co Co M M M 15) M 13) M M M n 134 M 15) M M 15) t5) M M 15, MI LI 1111111 1 I I I I I I I I I I I I CI

r--

Co N) 141 CO CO 0, 0, IN cr -0 CA CO "4" ro ro r, U7 CO CA D7 D7 CO 1- r- CA N) cr N) N. co -4- un cr,

CA sO M cr co cr r... Pte) ,-. b7 CO N, N Co cr r, ro ts! CA N CO =- r- cr CO OD ,0 CO MI CA In ,0

X r- Co r- NA r- CA CA Co CA Co C4 t5.1 ...- M NO ,-- Mit! Co CA CA IN (N Co n - cq ch. 15) CA tst cl cl ri

m • • a . • • • • • • a • • w . . ••••••••••••••• 17,

CA CA CA to CA CA CA CA CA CA Co Co Co Co Co Co Co M Co Co M M to Co to Mt M Co IS, MI 154 ts! 1 1 1 1 1 1 1 t I i 1 1 1 I 1 I 1 1 0

„!..:,

cr. -0 N) r, cr CO 'Si CO ,-- PO cr CO sO ,- NO PO Co M -0 cr oo cr. '- un -- n ro cq Co - IN CC,

ND In cr CN CO 0, 04 CA CA CA N OD PO sO CA n =-. CA OD U7 CO sD ,-- cr - ts! CO U7 r- .- ON LO 414

X CA CA CA CA Mt CA +- ,-- r- PO CA M CA M CA MI - CA C, .1-:' • • • • • • • • • • • • • a . • . • m w • a m • • • • • • • • •

Co Co CA Co Co Co CA Co Co Co Co Co CA Co C) CA CA CA CA CA CA CI M M n r- to M M4 M M4 M iii I 1 I I 1 I I 1 1 1 I 1 1 1

-

0, OD In CO VA ro rd sD U7 M 1' CA M CN PO 0, NO ,0 ,- 0, 00 ,- CO M 0, ,-- CA 0, sD CA cr un n

CA r- r- ,0 r- NI CA NO U7 CO CO CO r- CO M MI Co N N CA U7 1" U7 CO -O r- CO -O CA ,0 CA U7 g-

...I r- Co NI CA Co Co Co r- N1 CA CA r- CA Co r- N r- CA CA 141 CA CA CA r- 0, CA .-- r- r- C.! CS, (-1 • • . • • • • .... • • •11.1 •••••• •••• • ••• • 1 X

CA CA Mt Mt CA Co Mt CA M to to M CA CA CA CA CA M M M M n M Co Co Co Co M Co Co n M •H 1 I 1 I I I I I I 1 I I I I 1 I I I r-

i-3 gA V' In VA ,0 r- ,0 CO CO tO In CA In CO CA un cr un ro co rq M r41 r- In M D7 ,- PO ,0 M PO PO rt-J

r- CO Ul r- PO ,- cr ,-. -4- c-. In N1 0, CA P.') 141 CA -0 r- CO ,- CO N, r'.. CO 1' cr - un co ro un 7. E 1-4 CA CA r- CA ..- CA r- cr - n ,-. ..-- CA MI M r- M M ,...- IN CA M 1.- CO r- r- M rq n n - -- r- • • • . • • • • . • • • •••• • •••••••••••• ••• aJ O Co Co Co Co Co m n Co 15, Co n to to GA CA CA Co Co Co CA CA ra ta Co to CA n M Co M MI Co. U LL. I IIIII I I 1 I I I I I I 1 I I I r

un PO ro cq cr. n In rq c.d r, r, -.0 P, NI ro r,.. ro - un rq is! sO OD r- CO ,- 1- cr sD cr co un •-,

C, m ro ,0 CO ,0 N1 In CA r- cr rq to 1' +- 1-, .,- ro ch. 0- ro -0 cr r, bn - ,- cr co co CO N r-

X PO CA 1- cr cr n CO r- to r- CA .1- r- CA r- PO PO Co PO NO U7 MI M4 r- M4 r- Co Ul 114 PO r- CI 10 • • • • • • . • • • • • • . • . •••••••••••••••• >

Co Co Co M Co M Co Co Co Co to to Co Co Co Co Co Co Co Co M to r- Co Co Co Co Co Co Co Co Co 0 1 I I I I I 1 ! 1 1 I 1 ! I 1...1

Co 1- CA IN, Co IN Co CO ,0 CO r- 0, N. 0, Co r- r- 0, CO PO r- ,0 -O ro - oh. OD Co cr Cr. - CA ..

cr CO NI CA cr CO 'CI cr - CO 0, MI r, c..1 N, cr un cq IN In 4O N SO co cr -0 - NO ro ,0 N OD CI

GO ,- MI MICA CA 1- MI MI +- M! r- MI CAC :MICA CM CA CS, r4 rq cr, m m GA r- CA CA rl n ro ,- -2: • I. • • • a • so 11 • . 1 • • • • • • •1•••••• •••• • 1. ../..''

Co Co Co ts1 Co CA Co CA Co Co CA CA Co CA CA Co Co Co CA Co Co Co CA CA to Co m Co Co Co Co Co i i 1 i I I I I I I 1 1 1 I I I I I 4.,

!....

NI r- 141 r- .- CI, CO In r- CO In co rn c...1 un -o r, r, op so .- ,-. IS, Co CO CO U7 ro 0-. cr IN CA i1

CA Co N tO CO CA CN In mD cr 0, N) cr 0, ,- Co 1' DO 03 N, NI ,0 PO =-. D7 CO CO ,./' cq ro uo - a.

Lt. N ..- ..-• in -o CA Li.1 4.-. (N Co N .-• CA CA CA Co CA Co MI PO Co CA In CA MI Co CA CA r- rte) r- r- • • • • • • • • . • . • • . • . • • • • • • • • •• • ••• • • r,

Co Co Co Co CP CA Co CA Co Co CA CA Co Co Co CA Co Co Co MI ..-- Co CA Co 'Si Co CA Co CA CA CA CA <r 1 I I I I 1 1 I I I I 1 ! I I

I,

CN ...- "0 n sD 0- ,- NO N 0.. ...... NI 0.. in "0 Co rn C'- cr n -.0 PO rl cq cr, cr un op ,0 '0 OD 0, ..-1 N rn ....... 0.• CO SO 120 V1 M CA ,- CO D7 ,-- CA Cr. CO IN n cl IN rn 0... 00 C..1 1.0 r..... ro sO -,-- Mt Co ..c!

LU Co Co .- U7 CA ,- cr cq ro - ro .- cr Co - n Co ts! Co ts! ro IN ro CA ro PO CA CA PO +- MI 'S fll . . . • • • • • . • • • • • • • • • • • . • . • .11.• ki•••• F-

VA CA CA Co Co Co CA to CA CA Co CA CA Co CA CA Co Co Co r- Co Co Co Co Co Co M n Co Co Co MI 1 1 I I I I I I I I I I 1 1 f

r- mD CO 141 CO NO In r- CA 0, N, PO OD cl cq CO ON 0, U7 cr CO CO U7 CO ,- ,0 (-1 cr, un vs! r, co

C so so ro -0 CO CA N CO !N. -o cr cq .- n N sD ,- OD Cr! CO N, 0, ,- N CO CA n D7 N .0.... CO

(e) rq is! un ts! n - ro ,-- cq Co m un m - cr un U • • • • .. • • • • • . • . • . • •••••••••••••••

CA Co CA CA Co CA CA Co CA CA CA CA M4 Co Co Co to MA Co Co Co M M 154 M4 Co Co Co IS, Co ts! ts! 1 1 1 I I 1 1 I 1 1 1 i 1

ro CN CO Prl NO In cr IN ro - cq cr co cr co - n -0 cn c, r, Cr, - ro -o Co CO ro 'T CA r- ro

U7 CA CA NO CA +- ,-- ro un MCA n - M r- CA 1" ,- ,-- CA U7 c..1 ro so r.... IN n cr un - un un

04 Co PO Co Co M Co 15, 154 M 114 MI Co M Co Co M Co CO Co MI Co Co MI Co Co M4 15) n Co M n 154 • • • . • . • • • . • ••••• • op •••• a •••••• ■ ••

Co M Co Co 'Si Co M 154 n Co Co Co Co Co M Co Co Co M Co Co Co Co Cl M n MI Mt Co M n M4 I I I I I I I I I I

Cl- CA cr ro cr• ch. so ,0 CO CA 0, bl D7 ,- CO N. -0 ts! CI" ro r, - ro un C.) Co un ro ro ro ts! cq

c0, cr cr ch un CA +- CA MI P, r- OD ,0 CO cr cr m cr -01 co -4- un - CA M r- NO CN cr ro (-1 un

.‹C 1' CA ..-- CA CA M PO M4 MI Mt MI r- M4 CA M ,- CO M r- 1 CA M NI Co +- r- M r- M NI MI MI . • ••• ■ •11•10 • •M••• •••••• • •••••• • • •

CA CA CA CA CA CA CA CA CA CA Co CA Co CA CA CA m m to Co m to to to CA t& m n mi m m n 1 I I I I I I I I I I

ex

•-•. C4 MI "4" Z •■-• Mu .1-

001 C.) W L.. CD MC I-1 .. AC NC CD CV CZ CZ C. ‹C W U. CD MC AC At CD CD Cb CV CI 0 0

Page 183: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

183

,10 CN OiN ^0 Ch In DCI CA CO r, r, No cr. Tr co co Ch c.4 ,o vs., in os Tr os m ,_ ,r 1,... m L.c.,

Tr ro m ,- Co Tr N1 N. -4- DO PO CO Tr cci CD CO r, 4 uo m Tr DO D1 Cr. CD Tr ro N. Tr r,

CD CD CD In CD CD ,- CA ,- CA +- CD In CD CD CA CO CD CD ,- ,- ro m N CD ,- In CD CD PC1 1.10 • • . . • • . . • • • . • • • • • ••••II0••••

M M M 'Si M M M CD 'Si CD CD CD 'Si 'Si 'Si CD 'Si 'Si CI CD CD CSI 'Si 'Si CD 'Si CD CD M C4 'Si CD

1 I I I I I I I I I I I 1 I I 1 1 I I I

NO CO N, In If) CO DO CD 1- PO ro In NO PO CO PO Tr tsl -o cc Tr co n No IN cci In CD CO In ND CA M +- ..- PO CA CO Ns ON ND In to Tr cci N un CCI cq In CA ND CO ,- CA In CO in DI CA

Cri 'Si CA +- +- Tr m - m 'Si CA 'Si 'Si CO CA CD 'Si • 'Si +- ,- CD CD ..-- CA rl CD 'Si Tr PO . . . • • . . • • • • • • •11 l• ••••••••••

M 'Si CD CD 'Si CD CS, CD CD 'Si 'Si CD CD CD CD 'Si 'Si C a1 CD CD CI CA CD CD CD 'Si (Si CD 'Si CD 'Si I I 11 I 1 1 I I 1 I I I I I I I I I

NI DO CA PO -o r, CD In IN M 1' CO DO 1' rl cis N ,- r-, r, N. oo cr Cs CO DO CD NO CM IN CO Tr m ro e,ry n N Si If) PO Cr CD CD CA ,-- CA ,0 0% 1' NO 'Si Cs PO MI tSri a CSI m ro - N1 CD CD +.- CD CD Cs CD CD CD CD CD CA CD CA M CD CD CO M CD Tr m4 M

• . • • • • . . • • • •9 ••••••••••a••••

(Si M n 'Si n t5aM (Si 'Si CD M 15, CD CD t5I CD CD CD CD CD CD CD (Si CD CD CD CD CD 'Si CD 11111 I I 1 I 11 1 I I

PO In PO IN CO If) +- PO Tr ON Tr Cr Tr if) 0, M ro -0 n tsi ro Tr - No No c., 0N PO CO CO Tr

CI No IN Cr Cr Si MI Cr. CD IN N. ND ro LI) CO CA CD sO NO N ND CD CO In ro ro No EN vo -- C 'Si m ro - ...- CA Cs CD CD 'Si 'Si CD ND ,- CA +- CA ,- ,- ,- ,- NO 'Si 'Si m! • • • . • . •••• ■ ••• •111111•••••••••••

CD CD CD M CD CD t5W CD CD M M 'Si MCI CD CD CD 'Si CD CD CD CD CD CD CD CD CD 'Si CD CD I I 1 1 i 1 1 1 I I 1 I I I

IN ,C1 CD In CM Cr DO CD Cs CD 1- M ND DO +- N, DO In Cr sO ON n rl IN OD PO +- DO ND M

CA In D- ,- ..-- Tr CCI In DO Tr - is.... bl DO +4- CD Tr un -.0 un - r, r, lo, cr ro un r, PO ND Tr O. CD M Tr IN - - ro - - - Cr - m IN UPI - n Tr CA +- M PO ..-- •-• M CD DO CD +- PO Tr 7.; . . . • • • • • • • . . • . • • ••••••• ■ •••••• 0

M CD CD CD CD CD CI CD 'Si CD '5, CD 'Si CD CD CD CD CD CD CD CD CD CD CD CD CD CD CD CD CD CD Cr. 1 I I I I I 1 I I I I 1 I 1 I 1 I r...

CI PO N) +- CD +- Tr ts! NO Cs Cr ND PO ND Tr Tr EN Tr N 17.4 DD +- MI In CO CD ND 0, -o -o r".1 D.-. CA CD OD DD Tr cl r) IN Tr n ro OD ,- ..-- CD D.- 11.1 CA 1- CO CD ND IN Ti' ND ON n o..i it

Z m - ro N m N ts! ,- ,- co - - CD M +- CD +.- CM CM ..--- IDA CD CD D.- DD. ,- 0.-- n trA 1-7', .p.) • •• ••••••• II• •••••••••••••••• 0

M n 'Si CA n CD CD CD 'Si CD CD to CD CD M CD CD CD CD CD CD CD CD 'Si CD CD CD CD CD CD CD 4-, I i I I 1 i I 1 I 1 I I I 1

Cl

DO In M N +- ND N co un ts! tT4 4 un 4 N. CI M Cr PD Tr CO OD Tr ro oN Ti- EN c..! r. CD ,-. DI

NO CO 1- ..-- Cr ND CA Tr PO DO N 0, ro co - No un - cr uo m! CO ,- IN C 1: m m..., -- ..-- C4 ..-- CD Cr +- ,- CD CD D- n'Si.- ..-- (C! C4 CD M +- PO CD +- CA M 1:71 CD ;ID ▪ • • • • • • • . • • • •••• • • •m• ••• • • ....4

Si n n CS1 M m m m m m M M n M M M M M M M n n (5i M Si CSI n SI n LI I I I I I I I I I I I I VI

00 4 AON C..1 Cr CA CD PO rl CO Tr un MI co n co vo r un N -0 - - cr 0, co un N - In 1.11 r-

NI CO In NO CA M 4 DO +- CA CM In 0, m! N. ,0 ND Cr +4- CI CO mD +- D- ro ro r, 0, ts! -4- c, o -..1 M CD N PI N +- +- M Cr CD ..-- ,- D.- +- CM CD CA CA Cl D- CD ..-- tO CD CA '5, +- CD +- CA .... • • . • • • • • . • • • • . • • • • • • • . . . • . • . . •

CD M Cu CD CD CA CD CD CA CD CD (5) CD CD CD CD CD C5a CD CD CD CD CD CD CD CD CD CD CD CD n n ...< 1 ) I I I I I I I 1 I I I •.--1

r Dr DO In DO NO N, CA In NO N +- rte) ,-- N -Cl C4 r., o- Tr No os Tr N .- N) MI DO r, 'S M:i CA ..p

R. CD Cr n ND r-, uo MITr co ro tra Tr Tr N Ca N Tr EN co No co No MI Tr un rl c..! No ,- m E '-I n T- (5) ....- n 'Si 1-, n ,-- n ,- ,- n ,- ,- ,- ,- ,- (Si tr,...! n Tr - C4 tS! (-1 m.! CD ..-- CA s- !D • . . • • • • ••••••• ••••••• ••• ••

O CD CD CD CD CD (5) CD CD CD CD CD 'Si CD CD CD CD M CD CD CD 'Si CD CD CD CD CD CD CD (Si CD Ill LL I i 1 I I I I I I I I I LI

4 !JO "'I'. r% VII 0,1 n 1..... C4 DO sO In NO OD eD DO .,- NO In PO 1.- OD Tr No cl -0 co No oN m

Tr M !Ds N No Cr r, 1- Cs NO r CD PO CO N, Cr CD CO CD DO IN Tr n m! In CD NO CD CO 10 +- •,•-■

2 ND CA Ul In 0% m -- m C.4 PO Cr CA ND .,-- +- Tr If) CD ND +- MI ,- C.! ND CA PO D- ,- r- • • •• ••••• •••• •••••••••••••••• III

CSI M M CD CA CD Si CD CD CD a CD M CD CD CD 'Si CD CD M CD (S CD CD CD CD M CD M CD CD 1 1 I I ! 1 I I I ! ! 1 0

(....)

CAN CA CA +- PO Cs ,- CA .-- D- CO PO CO '0 ,... U7I Tr 4 r, uo ro 1.-,. DO ND CO Cs CA PO CA

Tr 4 r, No 1- N, DO CD M +- CO Cr CD CA NI ND CA CA Cs CO PO DO CA PO NO CO 0, CA -.1- ta.! ..

CO +- CD +- +- CO CD +- +- +- CD ,-. +- 1- +- CD CD +- CD +- DO 'Si CD CD +- +-- M ,- CD .1- CA al • 1611••••••• 11.1•• 11••••••••11•••••• IT

M M M n 'Si Si CD CD M CD CD CD 'SiCD CD CD CD 'Si M CD SiC CD CD CD CD CD CI M CD CD 3 I I I I I I i I I I I I 1 II! I D.'

+- CO Tr ON ..." 1..... 1"... Cs ND +- In r, EN Leo Cr -- r, un 1.), un c.4 co Tr ,0 ...-- Tr un No uo N co r....

DO DO +- CA cr -o m rq co r, ..- oN Tr ro 4 CA PO ND D- ND ,0 In DO D- CO CD DO "4- OD CD Cs D5

U. CA D- Tr Cs +- U CD CM 'Si CA +- .- PO D- SiC ND D- 'Si ol so T.... Tr ,- 'Si m Cl ,- ,- ro ,- 'Si Ct • • • . . •••••••••• •••••••••••0 •••

CD CSI M (Si CD CD CD CD CD CD CD CD CD M CD (5) CD CD CD CD 'Si CD CD CD 'Si 'Si (Si 'Si m n n t54 co 1 1 i I 1 i I I I 1 1 1 1 wr

Ul M CO ro Tr CA +- ,0 CA M PO CD NI sO DO DO ND 1" +.-- CO ,- No al

Tr No un ,- No cq n DO Tr tra ,- . CD ,- CO CD 0- PO Tr CA CD CD CD 12 % 2 6:::: ',.74 r..1 4 ;:! ,-.1

1..i.1 CD M 0- Tr ,- un ,- ro - NI CA NO CD .- CD CA CD CD DO Tr to Tr Cl - C.! ro ,- N) ,- ts! CD • . • • • • • • . • • •••Il •••••••••••••••• M

M CA CP M CD M CD CD CD M M CD CD M CD CD M MCP CD M CD CD '5, CD CD CD 'Si CD CD CD c.a I- I I I II! I I I I I 1 I

In If) 0, CO CO N In ,0 0, In M ,0 in ro r, Cr Ml rte) r, Cl- -C, Tr r, CO Do op

,- DO DO DO DO r, cs c, In CO DO DO N ,- - CA CA N) m ,- r.... -0 r, ro - 1

T-7. n .1- R Cl;

C.) +- CD Cr CD +- M CA CD CA CD Tr m n cq un ,- m .•..1- -- CA CD PO M CA CD +- 1- 'Si +- CA ..1- • •

• • • • • • • ••••••••••••••••

M M 'Si Si (Si CD Ira m tsi m CA 'Si V31 C54 V34 CD 'Si CD CD CD M eD CD CD CD M MCI M n M M I I I I I 1 I 1 I 1 I 1 I

CA DO ro m N Cr un Tr No r, rte) ,- co Cr. In co Cr .ct. If) CD DO Cr Tr r, co No IN n No -

r, No m No 4 ro m co NO CA M In PO CD DO CO el DO 4 CA OD Cu CA ND CA DO DD Tr Tr CA

AP rra Cr CD +- +- +- +- CD M CD CD CD CD CD CD CA CD +- CD CD ....- CD CD CD CD CD +- CD CD • • •• le•IIII •411•• ••••••••411••••••

M CD CD CD to CA CD CD 'Si IS (Si CD (Si CD Si CD CD CD CD CD CD CD 'Si CD CD CD '5) CD CD CD 'Si 1 1 1 1 1 I 1 I 1 1 1 1 1 I I

CO In In +- Tr No co N. '0 ... co Tr -o No No - r T T Tr -4 No r, T cr No n n cr

-0 ,- r un 4 4 r, r T T Cr Tr N. N ro ,- OD

Cl M ND 0, Tr (Si DD M CD +- ..q.- DD +-

4 OO CD D. CD CA PO CD CD CD 'Si CD CD PO CD CD DO ,- CD CD N) +- Cl m .- m m m m N) M M • • • • • • . • • • . • • • • •••••••••••••••

CP CD CD CD CS1 '5) CD CD CD M 'Si CD M M M M (Si M (54 M M Si 'Si CA CD M M m m M 'Si I I I I I I I I I I I I I I

P4

,-.. CJ M Tr E Cl N) CE .5. C-1 W Li. CD MC 1-4 MC Mt CM CC cn CM CC .. <r 04 LU LI- CD MC s.-.1 J MC MC C) cm cr cr cr

LA.. LA..

0 0

0

Page 184: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

184

ON ,- t CA 00 CA ON IN to ,..- ....o (Si r ,o uo uo No co PO CO CA Tr vo ,.... vo 'Si N. rq ,-. uo -- Tr

Tr -- c-.4 CA Tr Cr- S -- -- Cr- Cr. CA T ,- ,- CA N. CA VO In r.., cs r..1 rl m m -- 0- c-..! m CO

el m m Tr n m CA *- CA CA KA (Si Tr n m PO In (Si (5, T (Si KA ,- CA ,- CA KA ,- Tr m m4 Tr co

111111•••• • • • • • • •• • •••••• • • • II• ••• •• M (Si M M (Si M MA MI M M M M M M M In M (5, M (5, M M M M n (Si (Si (Si 'Si to KA MA

I I I I I I I I I I I 1 I I

M sO M Tr r..1 ro so sO In KA sO P, PO KA CO PO -.-- CO MA In N. M co Tr P, Mt In PO ro --

ro -o -4- co m m Tr uo ,o Tr so n ro Pr! ,- ,0 Tr n -- r4 co Tr CA CO In sO 1.17 Cr, CA CA CA KA KA Tr-

., . • . . . • . . . • • • • • • 11,1••••• • 11013•11• II•

M (Si KA MA KA to KA to MA MA to 'Si KA MA KA MA KA (5, to KA M4 MM M M (5, M M MI M Si In I I I I I I 1 i I I I I I I I I I

(Si m Tr CA U1 IN OD MA Cr. In CO In KA N. CA sO Tr uo 'Si vo Tr N. CA Cr, N. PO In

rq Tr Tr Cr- r, co r..1 cs ro CA KA bl r, r4 Sr In P, Pr! Cl CA

CN .- r- r- ro mnnn n Tr n m-4 ro n ro n PO KA ,- MA KA rn 00 MA KA • • • • • • • • ••• ••• • • ••• ••• •• • le•••••

KA KA MA KA KA MA KA MA MA KA KA KA MA to MA KA 'Si 'Si MA KA M KA MA KA KA KA KA n KA KA MA

I I 1 I I 1 I I I 1 I I

sat CA In *- CA sO N. ,... r.4 N. ,- PO ON In Tr Tr Tr uo n oo ro CA P, ND ON ON Is, PO ,- Lo ,.....

,- sO ,- CA r CA KA CA ON CA PO P, sO N) CO NO PO CA -0 Cts ,- KA .4' sO T PO CA P7 r.. CA CA Cr'.

(3 n M MI NI r- CA KA =- CA ,- KA ND KA KA KA *- MI MA CA ,- ,- .- MI •- V) .- n Os MI KA (5) • • • • • • . . • • . . • • . . • •••• • ••• • • • II•

tn KA KA KA CA M KA KA KA KA MI M M M M MI M M M MI KA MA K KA KA CA KA KA KA KA

I I I I I I I I I I I I I I

ON sO ,- P, b7 Cs CA sO CA CA m! ro CN CO CO .,.." n CO r..1 Tr co -o vo r, r4 co uo r, N. -o IN

ON In h. Cr. In Os '0 LIO Is, ,- Tr uo ro ro uo Tr r.4 KA CN 'q- T.- In If) PO CA ..0 n ro -- r..1 o, CI

CD M MI .4" r- r- MI NO CA KA ,- ,- If) ,- KA CA Ln r- Mt Tr CA CA KA Tr r4 -- n o• n -- ro Tr • • • • •••••• • 14.1•• • •••••••• • •••••• 9

II

M tn 'Si M (Si M M n M M M (5) M M M M M MA (5, KA KA KA KA MA KA KA MI MA n m n

1 mil i 1 i I , iii i 11 1 I Cr.

OD Tr ro co so ro uo Tr co CO CA ON CA PO P, KA T In P, CO P, r, CA P, CO Os Os Is, P, ..--,

CA PI sO 03 CA T PO CO P., tom) Pte) Cl- (N n -- r.4 CA so uo (---A Cr- M r- In '.0 CA P7 0, Os a; M MA ,- P7 ,- CA CA CA MI ,- MA ,- KA CA ,- MI MI •-- CA CA N M M ,- CO MA ,- , KA -

• • • • • . • • • • • • • • . . . • • • • • . . . • . • • C)

'Si KA MA Ca KA M KA KA KA KA 'Si M KA M (Si KA KA KA KA KA MI KA KA KA m! r,...! ml M M MA -r=

I I IIII I I I I I 1 1 I I I I I

r, cs cs r-. Tr ro co uo r, No CN SO CS .r..... sO CO CA n n ID LI

w- '0 ..40 in in P7 CA IN Cr-. In MA ,- Os M. In ,- ,-- %CI sO OD P, -0 CA MA ,- If) N N. sO KA MA NO 0% ,- N) Cr. In -

N M St CA r- r- r..1 m ro -' -- n -- (Si (Si 'Si n CA MA KA *- CA *- ON ,- MI N) (Si n M 0.1 • • . • • • • • • • •• • • •••••••••••••••• .1-1

MI M (Si M M M M M M 'Si (Si MI M (Si (Si (Si (Si M.4 M4 M (Si (5, MI M4 MI n n M n M I-I i I I I I I I I I I 1,11

sO a- so m so In CO Os NO 1'0 CA sO Cr, KA Tr los -- CO oo Tr m r, Ii 1 r, so c...1 (-1 so N. r, uo f-

KA CA ro co .-- ro n -- CA (5, ON 00 Cr- OD 1.17 In (N P, co OU ro m, -- r..; -4- so so n ro

...I r- n CA r- (Si (Si 'S NO r- to ,-- r- MI n CA MI M CA CA M (Si 'Si 0, n (N Kt- II• •• • •• • • • • •12 • • • •••••• II •••••• •

M M (Si M M 'Si (Si M St M M 'Si (Si (Si (Si MI M (Si (Si (Si in n LSA M 'Si M (Si (Si (Si 'Si M

I I I I I I 1 I I I I I I I I I I • H 5-

ix, CA Cr- CO IN '4r. IN a CA Cr, a -- n -- N. co ch. m C'- If) Tr cs CA n in P. P, IN r.. Tr Tr ,- ..P

Tr M IN M In CA M r- In CO IN M NO IN Un CP, I,7 NI *- MA 00 00 KA MA NI -CI T In n n E 1'4 MI T^ 'S CA ,- (5, - - KA (Si r- CA to KA to (Si 'Si KA CA - KA *- CA KA CA KA 'S MA *- IC r- . • . • • • . • • • • • • • • . . • • 11.• • 11•••••• O M M (S I (S MI M 151 M Si (Si n M (Si n SA M M M CA M (5) rz! CSa Kn 'Si In (Si KS! In KS! (Si n_! U. I I I I I I 1 I i I ! I 1 I Li

c

Tr so IN ON 00 NO NI r- 00 sO Os Tr r, Tr vo Tr co co ...- in PO P, MA P, CO P. In CA In CO In IT.;

ON CO sO KA In in a m CR Tr r, r. Tr CA Tr ro -0 co ro cr• m-4 uo r, co No cr. uo Tr IN CO

c.. •r =CAMN aa M. KA KA CA PO-CAr10 NOM CA NOTKA NK A-KA Tr- m -- CA 5-

. . •• • M• • • • • • • • • •••••••• • • •• ■ •• al

(Si M 'Si 'Si M M MI (Si M M M M (51 M M M M MI M m M MI (5, KA KA (5 KA (5) KA KA 'S ›. I I I I I I I I I I I I I CI

L.) Q rn a sO CA V) In Cr. r- 00 CA sO ro r. co n ro n ro CO T •-• N) CA 15) ,0 CO ,0 1..0 n In Q-

r, CA r. m so ND CA CO 00 ..- IN - Cl 0, 111 sO CA CA Tr ro r, ro n co r, r, r.4 uo n Cr- cl CA • •

CO r- M M M— ul n -- m -- m (1 (S;! NI M ,.... M MI CA CA M M r- M M CA n ..-- n M ,-- C) • . • • • . • • • • . . • • • • • • • • • • • • • . • • • • • ,r.

(5) M M M M M MI (5 M n (Si M (Si (5) SA M M M 'Si In r- M M '51 (Si M SA M (Si m in .4.; I i 1 I I 1 1 1 1 1 1 1 i 1 I

,...' Ul "40 T' VI r. ro vo so Tr Cl CN n n Tr ,- CA

vp n Tr n .r.-. so a u--.1 o.. a so r. OD 00 In IN rq

CO sO CA MI bl In In so wo in (1,;., N7 sO P, NO NO KA CA N In =- ,-

U. NI r- CA 'q- sO r- In M CA MI CA r- r- CA ,- .r- ro m m rq m IN Tr -- in m rl (--1 -- P :1-7. Cr.

11 o C).

. • . • • • • • . • • • • • • I. •••••■ ••••• • • n e •

M (Si M M M MI M M (Si M M M (Si M M MI SA M M M r- (Si MI M (Si (Si SA MI (Si n (Si n co I I I I I I I I I I I I I I .1.:

r- Tr Cr, co c.r. Tr -- Tr IN ro cr U') r::! IN ml sO In CA -q.. cl — CO ,- T 00 ,- If) .=1... 'S .,zy. Li) CO al CA sO KA Tr ,- Cr n Tr Tr ,o so so IN CA vo OD IN Is. .. .`0 NO NI PO ,- CA In sD .,-.1-. ci- .-.1-. co !...... .--,

La MI M r- Ll CA M 'q- r- CA ,- CA CA NO KA MA ,- Mt KA KA Cs CA CA PO CA CA CA CA CI CA n MI Mt .in • • • • • . • • . • . • • . . • . . • • • • • • • 117

MA 'S KA KA KA KA (5) KA KA KA KA MA KA KA KA KA M KA (Si KA KA MI MA KA (Si KA (5, 'SI M In M m I-

I I I I I I I I I I I I ! I

IN 0, IN T.... uo Tr C - In CT m! Tr m ro co vo co -- Tr CA In sO n h0 CO U In CA in N, r. Tr CC! MI CA Cr- CA CA 10

C.1 M M "4- M M r- r- • • • . . • • • . • • • • . • .

r- CA r- ,-- 4- St M r- Tr -- •—• • • •—• KA OD MA n MA (N MA CA 'S ,- T to ,-

l'r.: -1-.

KA (Si n (51 m-4 (Si m m n KA MA KA (Si KA (Si KA KA KA KA (Si n (Si (Si KA KA (Si MA MA (5 (Si (5)

I I I I I I I I I I I I

,- CO rte) Tr N. uo n so Tr ro Tr N. N) r.....! ,1- rq Sr' -.0 Cr- CA 'T MA CO Os ,- in MA co Tr CA CO CO

,- 00 CA ON NO CA KA KA P. CA ,- Tr IN r..1 uo CA ,- OD CA P, s0 CA sO PI ,-. n CA M CA r- , - ro

174 r- CA (Si M .... (Si T.. tn to t54 n M M (5) 'S (Si KA OD (Si 'S 15) KA KA to KA M.! 'S KA MA m! n in • • • . • • • • •••••••• •• • •••••• • ••• NI. •

M (Si M (Si M M M M (Si MA n M (Si (Si M M M KA KA KA MA (Si 'Si (5 (5 In M (Si M In rs! KA

I I I I I I I I I I I I I I I I

sO M NO IN 4..... T... 00 NO In KA Tr -- NO T. NO 0% rl Tr n In Is, NO Tr PO Os n p, LS,/ )' ,:r---aO ,- In CA KA CA ro os (N N. co ,- n -- Tr c..1 cr Tr ,- m co ro CA N) Os In CA ,- CA NO CA KA CA

<E li) r- CA ro m ro -- m KA ,- ,- M M M n ON M T'" M4 PO ,^ V') (Si M M (Si ,-. — In n MI • • • • • • • • • • ft • • • • • 14 • • • • a • • • • ■ ■ • a • • M M M (Si (Si M M M M 'Si M M M M KA KA KA MA KA KA KA (Si Mt Mt M M M to M M M (Si

I I I I I I I I I I

E r- IN ro Tr £ ..- (N MI -43-, f.... 4:c IS) L) LO U- CO 2 0-4 J = nt CD CZ CN CN CN C. ..x 04 C.:I LU LI- CD 2 I...4 -I 3C nt CD CN CN CV CV O 0 Li- La_

0

Page 185: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

185

'1- CA Cs ,..- NI bl -4- CA ,i1 ".- N. ro r, „- ..-..4 if) -4- CA -0 ,-- CO UD U-.1 ri 0, N) ro .4- r, -O ✓ n v co n r, m r, co ro ro so PO I'l VO In CO ....... r- n In CO 1'... -4- PO r, .1- -4- v n .4- so cs

C — m uo m m — (-1 n ro — ts. LID CA M PO CO .-- CA LID CA t) CA PO CA N CA -- LiD MA ,-- N) VA ▪ . • • • • • ••••••••• •••• • • ••• ••• ••• •

M M M M M M M M M4 M4 M IS t5 t M4 MI 15) M4 M4 M MI 15) M4 M4 n M n M n M M n

1111111 1 1 1 1111111 1 1 1 1

In M ..-- Ch v. m CA U, CA CA Os -!-- -4- r, ro -- cr -- r, so PO -- CO ..- -0 -- r.. .1- Cr. OD LID

NO CD ,-- sO OD sO sO r, rq os -- co uo n — uo ,- 44- cr CA Os r, -- CA ,--- r, los co uo hD CA CD

C m m rq -- -- v m m CA CA ,-- r4 — m cr. ro -- ro -- m — — CA CA ,-- CA UD ND CA CA L-.4 -- ..• . . . . • • . • • • • • • • • . • • • ■■ ••• •011•1111

M M M M M M M M M M M4 M4 M M CA CA CA CA CA CA CA CA CA CA CA CA Si CA In CA CA CA I I I I I I 1 1 I I I I 1 1 I I 1

LID r, r, OD ,- ,--, Cs CA sO v — cs n so cr ,-- -4- uo -,-- IN ..- CJ r, co c.,. LID CA -0 °I- NI

v... t--%. CA CO CN PO LID CA !JD CA .-- CA Cr_, -...- ,40 15, "CI 0, ....- Ill +.-- r, 1.11 .4- N) C1 to v. co n uo n c..!

C PO Si CO CA N1 CA N) CA CA CA ,-- CA CA Cs CA CA CA CA CA 15) 1..-.4 15; N CA CA 15) 15) CA CO J. SI n • • •••••• •••• •••• •••••••• • •••••••

ta CO M M CO CA CA CA CA CA CA CA CA CA 15) CA 15) CA CA CA CO CO CA CA CA CA CA Cl CA CA M4 CA 1 11111 I I I 1 I I 1

VI r, Cs PO rq ro to to ro ro v cq r, n -s ND . cis

.1- — uo v CA CA CN CO M NO CO T... In ,-- LID PO CA N1 0, 1A1 CrCs. ,-- co r, rl ,0 s. .,.„4- rs... uo m CA .1- Cs NO PO CO LID Os LID Cs ..P PO r„. IAD

G m m m vo rq IN -- m cl ..- ,-- — os n -- n m m m ro ...- cq ts. .4- in m q54 • • . . . • . • • . • • • • • • ••••• • •••••• •Il • •

n 15) M M M 15; M ta to M M M 15, M 15) MI 154 MI M Mt 154 M 15) M 15; to 15) 154 IS! Si I I 1 1 1 1 1 1 I I I I I I I

Os CO IJD ..- -- ND N, 0, Cs cq CP- -- N. r, 1.0 NI ,-- CA CA ,-- so (.4 Cr. In DD ..1" CA CO 12D

-1- ro CA VO M) CO ID CO ..-- '0 ...- CO ,- CO CO in V) In CO uo v. -- ON OD ND OD sO v v co -1- .1-

cD — m uo ,-- — n PO CA CA ..- ..- OD ,-- CA CA VI ,-- CA cr -- — m ro n CA CA In LID IS! ..- ro .4-

. • • • . • • • • . • • • • . . . • • . • . • • . . . • • • • • :.:-. n m n n n n n n n 15, 15, m n 15) ts. 15) 15) n 15; n ts) 15; 15) ts. is. ts. 15, 15) 15, CA CA CA •s

1111111 1 I I IIIIIII I I I I 5- 4=

-4- uo .1- so co co r, cs IN ro CO CN .4- "-- cs -- v 1 co — cs co uo No uo N. IN co cq -- LID CO fr.■

..- MI .- ..- Cs PO sO sO r4 CA sO ,- CO CA ..- s0 so IS, so cl so v ro r, „-- csi ro cq so cs so 0)

CO rte:? n S.

.74' CA CA ,-- N) CA ,-- CA CA *- ..- CO '— r- ,- *-- CO CA CA CA cl N ..-- IN n m c.%! ca -- -- • • • • . . • • • . • .. • . • • • • .. • ••••••••• ■ • 0

CA CA CA CA CA CA CA CA CA CA CA CA CA Cl CA CA CA MA CA CA CA CA CA CA CA CA CA CA 15. CA In CA U

1111 I 1 1 I 11111 I I I I I I= 0.1

sO -- CA .qr -- CO LID LO 0- v N) ro to v IN -O .4- co N) ro uo N ON CO v r, co tal ro is) vo Cr, •,-I

CO CO Os PO IAD PI CA CO CO ...- sO CO N. IN v co r- IN N) — co v — v ro so n so so ,- 0, Ml

1= ts. m -- — m ...- — -- ts. cs ,- ,- ,... m n — CA CA CA ,-- CA CA ,- V.:4 r- VO CA ,-- CA CA MA In f75

. • • • . • • . • • • . . • • • • • • • • . . . . • • • • • [Si MI M M M M CO M 15) M CO CO M M M M M 15, CO M 15) M M M n MI MI 15) 15, MI M n

G 1 I I 1 1 I I 1 I I 1 1 1 U

M 0 sO Os Li? N) ,-. 0 -0 Ch. rl In ro --10 ri CA sD CA CA LO .q. n OD CA .-- CA ..- .1- s0

OD CA !JD ,- '1. s0 CI

CA CO N NI ,- ,-- ,- CA v ro cl n ts. IN ro ro uo so ro ro ro n cs -- "7, CO OD LI

-J CA CA CA NO CA -- ,- CA 0- CO T.- Cl CA CA CA NO 15; CA CA rA CA ...- CA CO NO CA CA ,- ..- IS ,- r.4 0 • • • • • • • . . • . . • . • • • • • • • . • • . • . • • f

CA CA CA CA CA CA MA CA CA 15) CA 15, CA CA MA CA CA CA mi CA CA CA CO CA CA CA CA 15) CA CA CA MA Cr, I I I 1 I I I I I I 1 1 1 I

r-

5 CA IN LID ,-- sO CA h.-) ro sCi VI ON T'.. P.75:4 rl LO .1- uo ro co to tio n OD C. .4- CO ..- CA CA r, CA •1-1

...- 0, 0, Cr. v. r, CO !JD n -4- so CO OD !JD Cr. CO in ts. — Cr. so to co v r, los n LID LID T1" Cr, IN .1.:

Z 1-4 CO M S4 M M CO M 1",.. n .— CA n CA CA 15) In MA CA ,-- CA CA CA n MI 1:74 ,- :.'.! ,- CO CO t!.;:! ,-- 4=

a et a •

f- • • • • • • •• •• • ••• •• •• •• • • • Nov • ,....I

0 M CA CA CA CO 15) CA CA CA CA CA to CA CA CA CO CA CO CO CA CO IS) CA CA n MA CA CA CA CA n n 3;

LL. I I I I 1 I I I Illi I I I I 1,

ro N PO vr uo 0- — uo ,- in N ,-- IJD Cr, CA UD CA 1.0 !AD rl -0 ,- sD ND 0- -4- v cq so uo CO 9- a,

cq co ro cq so -- c-q co -- — so uo los n r, r.% IJD .4- in n N -4- co s) CA NI NI CO UD ...- b-.! •..1- r.---1 = NI CA NI LID Ul CA CA Ch • • • . . • • •••••• •1•0

T T.... Cl 1..0 -- ro n c.4 t.~ ts! c.! v If! m r..,- n -- — ro -4- -- ,- 0 • . . . ,. • • • " • 0 t61 ts, m m ts. m — n ts. m m n n n 15, n ts. 154 n IS, ts, CO n n CO CO m n m) m4 CA n CL

I 1 1 ! 1 I 1 I I 1 I I I I

r, N. N. PO n Cs in NA CO 61 N) .- CO NI Cs. ,- N) CO CA CA N) -0 CO PO PO -0 NI CO sD 111

v• -- CO IN M *- 1.,.. '41' In CO CA In -0 M CA M In '4' M ,- +- n .4- cs4 r, ts. r, to os c

CO ..- CA '— ,- Os CA CA .-- .- CA CA CO 'q- ,- MA CA ,- -..- CA VI CA CA CA CA EN n cq n v — CI

. . • • . • • • • • • . •• • • •• • • • • It •• • • • •

In Cal CA CO CA 15) to CA CA MI M1 154 MA CA S4 M SI M M S4 CO M n M 154 M SI n CO MA In C14 -!;, I I I I I 1 I I 1 I 1 I I I 1 1 I I f-

17

LID ^O sO .4- .4- ro uo ,0 ..- CO CO CA ..- --4- -- m ,o -0 sD CO .-- -0 -- "0 N, ,0 sD Os ^0 n ...7.!

In IN rn h0 IN so ..4.-

,- .q- Cp, ...- In M COO, ND IN ro so N. co N. "71 NO r, r, IAD CA CO 0- N. P.1 0- CA 0,

1.L. CA M CA ,- CA PO ,-- CA CA CO CA CA sD CA .1- MA CA •,-- ,- ,-0-

In CA NA MA SI CA CA CA CA CA MA MI CA CA

7.1 ,- 15,

. • • • IN • • ■ •• ■ •• ••• N• •• • • •• • • .1"

COM COM M M MI M 15, n M/ COCA CA COCA COMi CA CA CO I I I I 1 1 I I I I I 1 I ! 1:14

L....!

0, sD NO -4- r., .11- ..- CN '1" ,0 Os NO CO CA ,-- 1f.1 ,- r,. PO UI CO ,.... sO CO ,-- CA N, ..-- .1" sD 0, ✓- so CO MCC, c4 cr N) — — ro -4. 0- co n uo 't.:1- MA 0, N N) CA PO N OD if) LID CO ...... VD CA it

LU CA tS, CO 0, v. LiD CA NO ,-- ro ,-. N) n — n — 15, n vt- v rq v. .,-- -- CA ro — c.4 — n -- r-

. • • • • . . • • . • . • . • • . . • • • • • •4 NI •••

M M M M 154 M M M M M M M n 154 M M M to CO CO CA CA IS) M mA CA In CA MI CA CA I 1 I I I 1 I 1 I 1 1 1 1

CA CSA ,0 ,0 "0 r, NO Ul sD M 1- ..- 0, N. ,- C CA PO r., ro Cr, so m. cs co rq co .4- — to CO MI

CA rn CA sO N. Ml Os IN PO ...- rl un co ND CO MI rl M n CO T.. n M OD ro ,- N) — — 0-

C1 '-CO.CA In ,- Cr1 PO M CA .-- ,-- IAD CA CA CA IAD ,- CO .4' .,... cq — uo n CA CA .1:

. • . . • • . • • • • • • • 7 % b: 74•1111•4•0. • • •

M CA .-- CO M to M M MI CA CO CA CO CO CO CA CA CA CO CA CA CO CO CO CA CA In MA ISA CA CA M I I 1 1 I I I 1 I I I f I

0, fn CA -0 ,- CA CA Ill -,-- LID ,- r, N. CA CO CA MA ..- CA ,- -0 CO C..1 Os "0 -,..- N. CO N1 N rq O^ ro "0 0,1 M CO CS CO N. n to cu. ts. .-- uo -s cq ro os so v ,- Cr! -4- cq --

CP M CA 0) ,- 15) MA CA ISA CA In MA M CA CA MI CA ro CA 15) -- 5) 154 Pu4 l5 m Isi m. ts) m. ks. is. ▪ •••••••••••••• ■ ••• • • • 0 •• •••••

CO CS4 CA *A CA 154 CA MI CA to MA MI CA M MA 0) CA M MA M4 CO CA Q CO ISA In ISA 1S4 M SI M n I 1 1 1 1 I I 1 1 I I 1 I I I

CA 0, N CN 1.11 r, NI CA CA -0 1- ,-- NO LAD PO .4- 41- Cr. v m so so ^O CO uo 0- ,- .4- so cq cr 0)

0". Cl c-1 v ro ..v. cq -- CO ,- -- .tr v CA CO CA 0, CA ,0 ..- IAD NI tr.M PO Cs IAD ,0 0 In '0 ..-- -0 • CO M ...- CA N ..- PO CO CO CO Mi .- MI NI CO ,-- .4- m ,- ta N .,-- is. 15) -- CO ,- M MI 151 ,-

•••••••••••••••• • . • . • • • • •• • • •• •

CO CO CO CO CO M *A MA MI CA MI CO CO CO M M M M In CO CA CO In CO CO 15) CA CA CA CO CO I I I 1 I I I I I 1 1 I I 1

ex 4:4 .r. ....- 01 r.0 .4" Z ...- 01 N) .4- 04 (...) LO Li_ C.N 5 P-4 ...I A: C CD CN CA Com:, CA 1- CE OP L) LU L. CD CC 6-4 --I A= At c:A cti cli cti Cr o 0 LA- U-

Page 186: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

186

CO IN Co 0% ,... -CI .71" rl No cs co 10 Os N) 1r -0 Cp. CA r, Tr -40 ON 1.- Co ri M 0, Tr -0 1,0 TV CI-.

Tr No — cs CA Os CP .4- CA Co CO Co Tr If) CA III Cr- C..I Cl Co IN Co IN Tr -O r, Tr N) Co cp.. Co ,- Co

0 Co Co Tr ,-. Co ,- -- ,- ro Co n Tr Co Co N) 0 Co CD In CD +- +- CA Co rl Co Cl In Co +- .4- Cr.

a • ••••••••••••• ONN•11•••••••••••

M n Co Co Co Co Co Co Co Co Co M Co Co Co Co Co Co Co Cu Co Co CD CD Co Co Co CD CD Co m Co

1 1 ! 1 I I I I I I I I I I I

0, PO CO '.0 CD CO Co r, Tr n In CO Tr Tr co r, 03^:r N) IN IN +-,- N) -.0 +-+-• CO ro co co Tr

ro -- rq cs un CA PO WO CS CO Tr No Tr r, m rl No — V- -at n un No co No No PO PO PO co Tr No ,-

CN ,- *A CA CD +- Tr ,-- n ,... Cl m ro m tSa un ro m tsa ro n — to — n n ,- rl to In MI oN Tr

••••••• ■ •••••••• • a * • • ■■ •011•0•••••

Co CD Co Cl CD CD CO CD Co Co Co Co Co CD Co-! Co CD Co Co Co Co Co Co Co CD CD Co CO CD CD Co Co

I I I I I I I I I I I I I I I I I I

rq co 1-0 Tr ,0 1,0 In cl •- tSi ,-- EN r, No I:, -4-. Tr OD r, In '4- r, In Tr rl — Tr o, un r, co to

rq -0 Tr -- ..- Cr- N ....... .1.. In CA CS Co N) If) If) Tr IN — -- un Tr No co Co Co M CD IX) PO If) "1- Co

Cl N) Co CA +- PO Co Sr' Co Cl Co Co — Co Tr Co — N) n rl m ro Co N) Co CA .”-- Co ..-.. CS! Cr, m +-

••• • . • . . • • . • . . . . . • • . . • ••••••••••

M Si Co Co Co Si CD Co Co Co Co Co Co Co Co Co Co Co Co CD Co Co Co Co Co Co M Co CD CD Co Co

I IIIII I I I I I I I I

%CI r, +- +- CP NO NO CA NO PO CA Tr Co Co Tr to co Tr un un Tr IT co Tr In CO .r.- Os 1171 0 ro ,o

...-. ro 00 +- CO PO CD 0 In r, r, ...ci Tr Tr CS! In CD CO Tr No — CA NO CD Tr rl r, r, CA CD PO co Os

CN n Co Co (N ,- Ci 4-- Co ,- 04 ,- Co "It Co ,- M Co Si Co ND ,- ,- ,- Si ,- I-0 ,- Co SI M ISI n • •••••••••••1111•• •••••• ■ •••••••••

Co Co Co Co Co Co Co Co Co Si Co Co Co Co Co Co Co Co CD Co CD Co Co Co Co Co Co Co +- CD Co CD

I I I I I I I I I I I I I

Tr — Tr ro No to IN -,..- .1- IS) CD In Tr In r, PO Co r, -- Co Tr r.... No in ro IN ro co cis c, co Tr

No — ro uo r, r, co uo In CD CA ,r Os CO OD Tr r, ts! to in CA +-.. CO ,r CD to co r, r.a co ro in

CD ,- n In -- ..-- CD to — — — — in Co M 04 in ,- M trl ,- N M Tr — rq ts! usa co n — PO 0

1, •••••••••••••••• • ■••••••••• ■ ••••• X

Co Co Co Co Co Co Co Co Co Co CD Co Co Co Co CD Co Co Co Co Co Co Co Co Co Co Co CD Co CD Co Co .,1

I I I I I I I I I I I I I I I I I I I !...- 4-,

T- IN CO CD NO PO Tr co — co IN uo n IN ,-- o- — rq -- to — -- CI- co uo Tr r, ro — Tr — IT p5

No Tr — uo cs c-..! ro n IN No ro No uo Tr cs Tr -- — co -ci r, 0-4 Tr — Tr Tr — co r, n ro ro r

Z Co Co .--. PO ,- N =-. Co N Co C..1 rS, ..- Co CA +.... CD CD +- Cl CA CA Co Co +- (N Cr.. CD .— CD CA CA • • • • • • • • • . • • • • . . • • • • •••••••••••• ID

M M Co M Co Co Co Co Co Co Co IS) IS) Co Co n Co Co Co CD Co Co Co Co Co CD Co Co Co Co Co Co U

I I 1 I I 1 I I I I I I I I I I I iTi

0- -O N 4-- VO Co .qr ,-- N IN N 0'.. -40 CO '0 Cl T.. CO ',I" C.4 C..' No uo cs co — .1- CI CO ,- ,- la ..-I

In -0 VI 00 CO ,4- VI O. OD +.... CA CO Cs PO N ',1" in n '''.r M 00 CA PO r, In r, "1- PO r„ ts! ro Tr r-

u: -- Co Co rq Co rl -- -- IS) N) N Co +- Co .— Cr.! Co Co Co In ta ,- ,-- ,-- n Cr. rl 15 to -- ,-- es, it • • . . . • • • •••••••• 1■ ••••••••••• ■•••• >

Co M Co M Co n Co S4 M Co Co CD CD Co Co Co Co CD Co Co CD Co Co Co CD Is, Co Co CD Co Co CD Cl

I I I I I I I I I 1 I I I 1 I.)

In 0, 03 CO -.0 IN Cl- Tr .-- Tr un IN ro r, — un m cs CI- Tr c...1 No Cr. -.0 op OD n PO 10 CA NO ro cl

CI- T- CO N. Co Co C..! IN .,.. M T.. I', ID CI N. N CO Cl '40 T- r'0 !'0 CO .- IS, if) 1' Co C.4 n -C N. -

J Cl n C4 ,- Co ta M Co In ,- *A N .-- Co ,- Cl Co Co N C.4 Co IS) Co ,- CI, n ,- CA =- Cl SI (N n

. • • • . . • • • • • . • . • • .. • • • . • • .. • • • .. • -

St Co Co Co la Co Co Co Co Co Si SI SI M Co Co Co Co SI Co Co Co Co Co Si Co Si IS4 Co Co ta Co CI -

I I I I I I I I i / I 1 I 1 I I I I I 1- .... iii M N 0, ,I0 ,-- ,ID PO CD CD CO -.0 NO r, CA OD UO Tr Cr. co n No Tr to -4- No cs CO n Tr Tr to co •,-1

M CI- Co ro uo ,-. Co Tr I:, 03 r, co co N) N PO CA CA .4. If) 10 Cr- Co Cr- ,- r, +- Tr Tr ts! -40 -0 3-

r 1.-1 M Co T- T- CoT- T- ro IS) n Co Co — tst Co m Co Co Co +- =-. Co -- r, +- +- Co +- Co Co n Co 1-, r- • • • . • • . . . • • • . • • . • . . . • . • • • • . . . • • • • r- 1

0 mmts!msinnnts4ntsitsinntsa Co Co Co IS! Co Co Co Co Co Co Co In Co Co Co Co Co .-. 1.1. I I I I I I I I I I I I I I 77

-it Co Co T- -0 N) -0 co co os if) ,...- co — .- co n r, Co — Cl- .- — N) cs n 0- -0 CO In .,-- a) .-

Co Tr m N r, .,-- co co Co Co N) oN to r -- Tr IN Co Tr Tr Co ro Cr! Co co N) Tr OD Co CO CO Tr 1...-1

:c ro n Tr Tr Tr n r... m is! — rl PO ...- CA +- PO PO CD PO PO In CD CD +- CD +- M Tr — PO .,-. rq o • . • • • • . . • • .1 • • • • • • • • • • • • • • • 1. • • • • • CI

Co Co Co Si IS) M Co n Co M Co Co Co *A Co Co CD CD Co IS) Co Co .r.... Co Co CD CD CD M CD CD 15 0_ I I I I I I I I I I I I I I I I

011

'0 "10 -0 CO T.. C..1 T." SI Co N CO Co r-- N N) ,- Co Cl- "Il- ,-- ,-- if) ,-- Tr -0 -0 -- r... Cr- r... =- Cr, Cl

VI NI ,-- VI .-- ..-- 'Ir VI ND ro Tr — ro un r, r, r, -- r, n r, co ro Cr, ro IN c..1 -- No -o No r,

0 ,- SA ta 04 N If) Co Co ,- M ,-- Si N Ca ND Co Co Co Co (N CA 0-. Co Co Co +- CA Co +- CD t":1 4=4

. . . . •• • IN••••••• 4NO•NOONN•1000•••• ,-

Co Co MA M Co Co SII Co Co Co Co Co Co Co Co Co M Co *A *A M Co Co Co Co Co Co Co Co Co Co CD -0

I I I I I I I I I I I I I I I I I I I-

-0,-- 0-- Si) CO N -O In •Izt- If) 0,N Cr-'— C-4 .4.1 Co -0 CO N. 0, ,0 IN CI ,-- -4- .CI- .1- N -.0

un 0,1 co r, r, n r-, No ro uo -o Tr IT r, CT- co co un r, n rq r, ts! un ro op r, CA rq Tr uo CD

Li- EN ,.... ..-- Tr ...ci EN un m C4 Co N ,- ,- N ,- Co C.4 Co Co N) St N In ,- M ta Cl Cl ,-- NI ,-- ,- ..• 1411•21•41/14•111111• • • • ••••••• •NMUMM cr

Co eD CD Co Co Co Co Co Co Co IS, Co ta M Co St SI Co Co *A ,- Co *A Co Ca n Ca Co Co ta 15, SI I I I I I I I I I I I I I I I a!

,-.4

*A Ci VI NI -0 00 CA r-0 In VI ,-- ,-In LO ,ID N CO VO C.4 0, ,- ,--. ,-- La Tr IN to OD in in IN Tr z!

,-- Tr s! cr. -0 Tr Co c, to PO CA In .4" +- Co n Cr, CA CD OD Co Co Tr uo ,- m ...0 1.0 .,.... If) Co C.., 111

LIJ MI M T- Tr c...1 .,..- 1- n IN ,-- rl ,- ro n ....- Ls.! ts! m Ca al ro rq ro -- r..1 to rq ....- ro m n CD I-- • • • • ••• ■ ••441411•• •••••••••••01•4111

M Co Co Co Co Co *A PS) Si Co Co Co Co M M Co Ca Co Co Co Co m tsi ts! Co ts.i m n n m n Co

I I I I I I I I I I I I I I

IC -- r, r-, -0 N) n CCI CA N) CO PO Cs ,r r, "4- NO +- n CA co Tr n 03 Cr. Tr — -- uo r, ro r,

,0 uo m m uo ID If) '- -O 0- .0 co n If) Cr m IC N) =- CD N r, Tr -4- No Tr co N) No -- No n

C..) ..-- Co LO Co Co ,- CA ,- Cl Co MI IC Co ts! rl un -- Co Cr- Co CD CD N) Co IN us, -- uo m cl ro If)

. ••• • •••..-...• • • toN4 • ION•NMNOUNN•W

Co Co IS, M Co Co Co Co Co Co Co Co Co Co Co Co M Co Si Co Co M M Co Co n Co n In 'Si n Co I I - 1 I I I I I I I 1 i 1 1

Cr- rs! to — -o — In to c...1 CO Tr I.r.I (,) ,- ,-- iiTi CO MI ,... r.O '<ION. r, oN Cr, co N r-, '1" CO -1- N

CA ....... Cl Tr r, ts! Tr Co un m Co N) Co -- 'Cr — N -IC N) CA In +- I'D IN IN t54 .,.. Co Tr ,- Tr m

Ix! m N) ts! Co ts4 Co. Co IS Co n n Co CD CD CD CD Co CO CD CD CD CD I5, IS) 15, 03 CD CD I5 to rrr I5

•••••••••••••••• N•11111•••••••••••

Co IS) Co Co Co M 'Si Co Co Co n m m Co Co Co Co Co n Co Co Co CA Co 15. CD Co M CD CD CD CD

I I I I I I I I I I I I I I I I I

IC n rq If) N) Cr m If) -0 Tr Tr IN m c, 0- cq uo co to co n Ca m IC 15) -- ,- m co Tr CO 0,

0- m N) un co rq In In N) CO -CI If) In 0- =-.. +- In rl Tr c, co r, rq CJ co If) — r, co r..! -- IN

[E IC Co .- -- rd n N) Co Co 15, M .— IS) Cl n T- CO Co T- Si C.4 Co N) Co Co n n T- In ro ts! ta

. . • • . 0•••••••••• IIMON1102•••••••••

CA Co Co Co Si 154 M M ta M Co Si LS4 n Co Co M sa Co Co Co *A Is) Co n n n CA CD 03 CD CD I I I I I I I I I

PC4

0,1 ,r z rA po .= 04 GJ W U. CO Z 1.-1 -J 1= MV CO CO CO CO CO t•- <r ix. GJ W LA. CO CC H -J 3= Mt CO CO CO CO CO

0 CI 146.

Page 187: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

187

00 P., 11-.... Cr- NO in CO S.1 N. NO NO PO -O Cr- '0 cr - ro -- mo n co ro -- No ro Cr, ro co If)

cr L7 ro m uo uo - co 0, - CA 0, OD r, No r, ro r, n ro c-.1 OD DO DO rn m cr cr n n 1.-..! a, '4

cn t51 m so m m - cl -- ro - n in n n PO Cr. M M in .n.... M M NI ,- CA t ,- ^O CD ,-- r NO • • • N. • •1111••••■ •• •••••••••••••••0

I M M M CD CD M M M Si M CD M ED M M CD Cl CD n M t n CD CD t CD ED Si 'Si M

111111 1 1 I 1 11111 1 1 1 1

M ,- CS Ln Cl CS NO n 1'0 OS CO PI r, DO Os r, cr r, CN In ,- CA CO IN ,- ,- r.. OD CO IN

PO M ,-- OD Ul r, r, No co CA - Cr. M cr m NO r, CA PO ,- 0-. NO -,-- GI Cr. 00 0, CA UO cn LS IN -- ,- cr n - IN - n ro ..--. Si OS ro CD Cl ro t! - rj ,- ,- m ,_ IN in

. • . • • • . . . •1111111311 • •••• 11•••••• 00 M MI n M M M MI MI ID M CD M M M GI CD GI CD M M CD GA GI CD CD Cl tai CD CD G

I I 1 1 1 1 1 1 I 1 I 1 1 1 1 1 1

CA -0 GI r,. PO NO PO M CO CD PO r, ED -0 r, ,0 ON CA PO r, co n cr a PO r, ND cr -- OD l'O CO

CA NO CA Os Cs ro uo - DO M CA Cl OD ,- O ED "0 ED ,-- DO ,- r, No cr csol 0.o• po cr co rc., uo n

cn ro n m n ro n ND CD CD CD +- M GI Cr- GI CD ND ID I M CA CD CA CD ml ED M CD CD Ezt" n Q.! • s • • • • • ••• 0116114 • • • • 11 0 4 II n • • a • • •

M M M Of M4 M4 M MI M4 M 'Si M n 15, M M M4 M En M En M4 &5, M MI M M CD M CD CD CD

1 11111 1 I 1 i 1 1 I

cr cr r, vo 0, co 00 DO DO GI

ED r, NO PO NO NO r, PO No cr .0 ro

in uo Crcr n ro ro

rq ro CC, -- -- N. ,0 CA 1.0 DO PO 'Si CD DO cr DO DO in CA

cn n 5, m vo c4 CJ CA CA CD CA CD SA SI CA cr CA CA ,-- CA CA CA +- y CD CD CD • • • •

• •

• • • • • •

• • • • • • • • M CA CA CA CA CA CA CA CA CA CA CA CA CA CA CA CA CA CA CA CA CA CD SA CA CA CA CA CD CD CD

1 I 1 I 1 I I I I 1 1 I

CD N. CO U7 CO ,-. CA ,- NO ND r, +- R. CO NO n Os -- CD CD CA CD CS n r, (-4 cr - ,- co cr

a ro ro co CA Os -.0 OS CS 1.17 b7 ND DO OD n 00 CA Cs! 'r.1. -,-- n r„ 0, OD -CI tr....1 ID ',CI Cr

CD CD CD Ul ,-.. ,-. CA 147 ,- ,- ,-. ,-. 0, ,--. n ro uo - n uo CA ,- CD cr - -- m n ^O m -- ro cr

. • . . • . • . . • . • . • • . . . • . • • • 11 • • • • •

MI M CA CA ml Cl CA CA CA CA '5! CA CIA CD '5 CD MI CD M ES) CD CD CD GI GI CD CD n M CD CD

1111111 1 1 1 1 1 1 1 1 1 1 I 1 1

sO cr r, m n co No No so in - PO ,- PO Cs ,0 CA CO CO OD CD M cr -- DO sO cr a No r, cr DO C

'71- CD CA IlD EN CA r. cr NO Cs 0, DO ,-- M Cs 0, 'Er CD CO NO r'', NO cr cr cr c.1 PO NO P. 'El CA ....

GI M ,-- in CA ,-- c4 - - m! co .-- IN .-. m n n n n IN EN ..-- CA - M c.1 CA ,- ,- n CD CD Cr,

. • • • . • • . . • • • • • • • • • • • • • • II • • • • • •

M M CA CA M CD CD MI CS! M n M 15, Cl CD CD '5 CA CD CD CD ED CID '5 CD n CD n CD '5 CD CD ,-1

1111 I 1 1 I 1 I 1 i 1 I 1 I 117, .4.=

In c, cr vo m! n r, n - CO DO NO CD M NO r, DO r, n c..1 No - r, Cr, in co ro - cr n OD 0

1.11 40 CA CA 0, 1.17 CA CA CA Os Cr, DO CO CA +- CA NO n CO 'nor NO cr -- PO Cr, CA N.. a CO (-4 uo 4...'

CA CD ,...- ...- CA ,-.. ,- CA CA 0, CD ,-.. ,- CD ,-. ,-- ,.... c;:! itA ,- tlIA CA .,-.. ,- CD vo m m cl n CD CD • • ■ • ••• • • ••••••• • •• •• ••••••• • • • !■

CA CA CA CD CA ED CA CA CA CA CD CA CD CA CA CA M CA CD CD ED Cl '5 CA CD CA CD CD CA CA CD CD 0 1 1 I 1 1 1 I I I 1 1 I I 4-

CA N. ro ..-- uo o4 DO CO NO ,- DO 00 n CD cr CA in N CA In in '.0 ,- CO In CO IN T.- NO X .1--

M CO co r, CA -- No CA ro a- ro n c.1 - DO -0 DO cr -4- n m cr co IN CO -0 1...... 'CI

-J CA CD CA 147 ri ,- ,- M 0, SA ,- ,-. CA CA IN 147 CD CD CA CA CA ,-- CD ,- ro n IN - .-- t5.4 - ro 5- • • NIR• 11•1111•11•11, • • • N • • • • • • • • • • • • +1

M M MI M CD CA CD CA CA CA CA CD 15, ES) CA CA CA CA CA CD CA CD CA 15) CA CA CD 'Si CD CA Ca CA I 1 1 I I I I I 1 I I I 1 E

tE ,0 ,-. CA ,- rq cr so cr DO n NO CA DO n NO CO PO Cr CO ro cr co uo co uo - cr 0, ,- n ,- 0

CO Cr, r. P.. CA ,-. N. m.! ,- CA *V Cr, CA in OD Cr rn CA Cr, ED ro so co -- c-.4 - - co ho n co cr "•,

E I--I ,- cD ,-. ,- Cl .,- CD ,-. CA CA ,-- ,- ,- M ,- -- CA CD +.- CA 15) M CD sO +- CA CD CA '5 n c...! r- . • • • • • • . • • . • • • . • . • ••••• ••• • ••• M O M4 M M '5 M '5 En TT MI M 15) CD CD M CD 15, CD D CD CD G M CD C? n 15) n•

CD CD '5 l 15) ..--1 LL 11111 1 1 I I I I I 1 1 I 1 r-

!•!.;

I", 40 V CO c, cr so r, r, No - co ro Cl, uo co ro uo r, co uo r, - ho m CO cr cr ..-- - Cr, -2..

NO CA ''',1- sO ,-- NO N. cq cl r, No -- so co C -q. Li

:IC 147 140 117 1.17 CA Cr CD +- +- CA NO CA PO M CA PO GI CA cr 111 CD r. ,- ED PO .,-- '4-,-. CD

. • • . • . . • • • • • • • • •••••••••• • • .

1$) 151 M Cl CA CA ,- M CA 15) CD CA CA SA CD CD CD CA M M M CD ES) CD n n CD CD CD 15) n n .. 1 1 ! 1 1 1 I 1 1 1 1 1 1 0

!::

N. CO r, N N.:1 Cr. cr CA OD CO CO ND CD ,0 Cr, NO CA -4- cr co c4 co n PO PD Cr. r. CO CN C)

CJ taa (-4 r, r, m ,-- - ,-u7 0.4 Cr- -- ho 1", -,- CD GI sD in n n - -- m cr cl co v,,, -o m

CD ,..- CA ,-. ,-. Cr, ES) ..,- ,- ,-- CD CA CD V - CA CD ,- ,- CA Id7 CD CA CD CA CA ES) CA ES) cr cq

. • • • • • . • . • . . • • .

15, M En M 154 'Si CA CD C CD C 15) CD CD CD CD M t

ES) M GI M CD '5 5 C5) '5

C•4 11 !.... n n N

1 1 1 1 I 1 I 1 I 1 I I 1 I I 1 I I I CI.

0, ,0 40 CO N CO CA - 1.(7 C^A CA I.17 CI*. PO DO CS n In sO hl cr uo co r, cr rq n n n IN n

DO sO CA VO r, -0 CD ..-- Cr- CA -- PO r, DO ,- IN CO CD r, -- n OD 0 NO '',4- n NO CD

LL CA 'q- Cr, ,-- DO CD CA CA ..-- CA ND ,- M PO M CD CA NO CA cr -- m CD ,- ,- ..-- cr -- <I:

. • • . . • • . . • . . . • . • • . . . • • • . . • . •

M M M CD CA M CA M tal CD M GI CD 15) CD '5 CD CD M 15) '5 CA CD CD GI '5 n n 15) n n Ill

I I I I I I I I I I I I I I rs-1

Cr' in CA M NO N cr - - ro n co DO r. Os ,- sO CD ,- -o--- CA CA 00 Cs cD DO ISD ro NI CA n CA 115

NO r, sO CA r, cr r, r, C4 DO CO 0, Os DO !SD cr No ro r, co so ro CD NO 00 DO ,- Cr- Cr. P:O OD I-

CJ CD CA n Cr- cr - ho - ro - in - in 15) - 15, ,- CD n DO cr cq cr c..! CA CA in CA CA CD n CD • • • . • . • • • • • • • • . . • • • • • • • 4 • • • • '5 M 15) M M NI) M M M MI n Cl CD CD n CD M tD 15) M M ED M CD CD M M CD n 15) ED CD

1 I I 1 1 1 1 1 I 1 I I 1 I 1

V CO DO CA NO 00 NO V4 in cr r, N. n CO NO CA Ca sO -0 NO DO NO 'Si .1- N. in co

CO CA cr n. No c.4 CA r, co n CA in uo a. co n co c..! CA .- CO -- Cr -o co IN cl vo n roz!

(....) CA CA CA M ,- CA 141 ,-. CA ,- DO M M CA NO D C CN • • • • . • • • • • • 4 • • • ••• •• • ••••••••

M M T... M M M En M M M MI M4 M MI M tA GI ED M CA M CA ID Cl Cl CD CD CD ED CD

1 I 1 I 1 1 I I 1 1 1 I 1 !

CA ,-. 00 DO ,0 ,- in ,-- r. Cr. n cr No n CO cq n m No -- IN T.. cq - uo r, PO -0 rn No

ro Cr- CA -0 CA CD OJ Cs CO ,0 NO ,-- CA ,- NO n +.- DO cr 'N ro cr Cr, CA NCO'E - ,- Co 'E CA ,--

PCe CA 0, ES) CD ,- CA M GI CD GA 15) CD MM M NO n M +- D C '5 n D

GI ID GI CD CD ID • • • • • • • • • • • • • • ••• •••••• ••• • •••

CA '5 CD CD CA CD CA 15) 15, C CA CD CA CD CA m m m n ES) m m m n 15) n m m n n n

I 1 1 1 I 1 I I I I I I I I I I

cr sa cr 0- 0- r, r, sD CA DO +-- 'I-4- CA 00 PO no - cr uo ,-- cr in =.1.. co 1.17 N In' n ,-. CO ,.-

CO in CO - DO CI in CO M un Cr uo in PO In Cs. CA c...1 vo CO 1.0 - CA ts1 cr uo ESA in co IN „-

cc Cr- m to m c..4 ,-- r7 .-- m ts! t5a m ESA In M ES1 In 1.S1' ,-- M CA •••• ro - - • • • • • • • • • • • • • II 11 0 4 • • • II • • • 1.S.1 • CSI

P.".•I 71 151

,Si CA CA CA CA CA CA CA CA CD CA CA CA CD CD CA CA ED CA CA CA CA CA CD CA CA CA CA CD CD

1 1 1 1 I I 1 1 I 1 1 I I

.4r "III

E - CA 147 .4- C ,- c".1 m cr cr co G.) UJ U. CO = I-, --I NC NM CO CU CO CO CO r_ eZ 1.74 LJ 1JJ u- C7 MC I-41 -J 3C Dt CO CO CO CO CO

0 0

0

U..

Page 188: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

188

r- No CO rA uo c.1.T. No co If) sr to CO In LIO sr ra co ra ra 4 sr -0 r, CA Cr, Cd sr CV CA CA U1 CA rl r' ra ra sr ra ra 4 -o

Al cA r, N. Co r, ro N. cl ra -O Cr. — N) T — in 4 -o to N') cr,

Ci L1 Co C.1 If) Co Co • • • • 4 • • • • 11••• • • • •US1111.1••••*11111111M

CA CA CA GA CA CA Co I) CA CA CA MA CA CA CA I CA Co I) n Co Co Co n Co Co Co Co Co ra ra Co 1 1 1 I I I 1 I 1 1 1 I I

OD PO V7 CA CA V,. -- n -.0 N 4 co ...o N7I CO CO M 'CI cl sr OD OD ro ra r, sr ra "I- CA -.ID 0, -0

VI V, CA -,- PO NO '4' 1.171 OD V, In 4 V, In CA '4" oN cA to co P9 In sr co to -4- to to r, op P7 r„ sr

CD GA CA CO CA ,-- '4' ..- ,--. ,- CA CA V7 CA CA In NI MA CA N, CA .,- N7 ,-.. -- CA ...- CA 19 In CA Cr. ".1* . • • ... • • ..... • • . • . . 2 0R11••••• •••• 11

M Co M Co Co Co Co Co Co Co Co Co Co Co Co Co Co CA Co Co Co Co Co Co CA CA CA CA CA CA GI CA I I I I 1 I 1 1 I I I 1 1 1 III 1

.,--: IN, r-, CA to [CI ..-- -,--: •,..: GI I`, ,--- 4 CO CO PO Cr, CA .zr .."4- In CO .- 10 CO ,- CA ---r o, -4- ,o —

CA co sr ra 0, M -4D ,- CA ,0 CA V, -CI 1.19 PO sr IN ro -- CA OD sr r.. OD 19 ,--- Co ra -..o ro -(1 to co

a ro ra cA ra sr ra sr ra m CA n r- M .zi' M ,- I'''') M Cq M N7 CA PO CA CA -- Co ,--- CA 0, GA 15:

. • • • . • • • ..... • 0 • 11. •••••••••••• • 11

Co Co Co Co Co CA GA Co Co Co Co Co Co CA CA GA CA CA CA CA CA CA GA CA CA CA CA CA t CA CA CA 1 111111 I I I 1 1111 1

GI -0 V, PA CA N. sr Cr, r, sr No — N1 -,-- V, 0, ,-- Co In ,- 0, LIO '1' N1 N. ,--... CO r, CO 0- CA Co

,-- N) CO ,--. 0, r GI 43 in Co CO ,1) In In CA in Co CA '4- -.10 PO CA IN ,- sr ro OD r, PO CA PA CO 0-

C5 CA CA CA CA .-- CA .-- CA ,- CA .--- CA "4- CA .c- CA ,-C, CA N7 ,- ,- ,--. GA ,- po .,... m ts, ts, t, m • • • • • . . • 11411•• 11M • 111111 11111•••••11111111011

SIGISIMMSIGISIGIGI Co Co m Co n Co Co m m 14 Co Co Co Co Co n m n ..-- Co Co Co I I 1 I I I I I 1 I i 1 1

L,) NA sr to cA a, sr sr co CS sr ro sr po

cr, CO IN Co '4' CO N n CD CA In CA CA to IN ra CA PO ,0

co r, PO CO CO V, CA Cc Oc V, "P sr CA N CA -0 •q- m Cr, !f) L1 NI r, 4 N7 -0

Cl C4 6.1 cr 64 Co Co N) in ••• •••• •••••••• • • • • le • •• •••••• ft • •

CoCA CA Co Co Co cA CA CA Co Co Co Cr Co CA CA Co Co Co Co CA Co Co Co Co Co Cr GA CA Co Co Co 11111 I I I I I i I I I I

r, r, ra uo r, N) CO ,-- PO NA sr N N. to -- -0 PA sr Co Co sr N N) r, — if) -o a- N) N Co r... Q

in sr CA In 0, CA V7 -- CA V, Ni ,0 1.0 41- o, sr ra -- co r, r... cA bo 7- uo sr -- r, N M ri F*0

= n 64 ..-- PO .,-. rA -- Co rq Co rq Co -,--- ra cA -- is, Co ....- cA cl cA Co Co ,--. CA C CA ,-- t5 cl cA F •••••• •••• ••• • • • • 111111•••••••••• • •

M M M M M M 64 n n n ISA n CA CA CA CA CA GA is, M n is, is, CA CA CA CA CA CA CA CA GA ,..„, I I 1111 I I I 1 I I I I I ! I

If) n PO if) V, N) Co In CO OD -.0 N N) N. cl — ,- ci- OD N CO N. 0- In ,--- IN n Co -- — sr to n

sr -o cA co co sr sr — OD CA CA CO CA N7 CO sr ro ts! .!'• m co CA PO Cr- ID N '4' P, CO CIA N71 sr 4.-'

*--. is, Co CA GI CA ,- rl Co N) N Co (-A Co .- is, Co 15) Co to Cr — — -- ra cr, cA ra n ,..... 4' n • • ••••••••••••• • • 11X 11•••••• •••• • •

CA CA GA Co n Co CA CA CA CA CA 15) CA CA Co CA CA CA CA CA Co Co CA CA Co Co CA Co CA Co CA CA I I 1 1 I ) I I 1 1 I I I I cp..

OD *-- In CA sr m in CO -- to uo ra — to CO Po ,--- V... ,- po No sr lo, ci- ho — — — r, co r„, r,

is, CA Co 140 Co Co Co rA sr cr, sr cr, Co 0- 0, PO V.„ "i- in CO sr If) No ^0 PA t) In in r9 --- "1- N) •ci

_.1 ,- m c..1 N m Co n ,- po i5 Co -t-- CA Co Co CA Co CA CA CA Co CA CA .,--- 0-- CA ,-- -)--.. -- CA IS:, cA r- • •s• • • • ilm• ft ••••• a • •••• • •••• • ••• ■ ..!..,

S) CA GA GA CA CA Co CA CA CA CA CA CA CA CA CA Co n Co n CA CA CA CA CA CA CA cA CA CA CA CA fI5 - I 1 1 1 1 1 1 1 1 I I I I I I 1 E

Pc.) sr cl No Cr- r, co — uo -o a, — LT. to 0, to ..-- sr cA sr sr vo No co m Cr. uo r, r, to to ra co ID

15) Cr, No ts! — — Co ,- ra Ni 4 i'-.. to C.,1 n N) In In ..--- 1.).9 N) cA Co ,-- -.0 LT. ,- if) cr N) PA ,0 LI E N4 •N" M T- Cl T- Co ,- -.0 ,- ," ,^ 4.-. C-I CIA ,-- .,-- -,- Co ,-- CA ,-- -- ,-- Cr .-- .c.- CA Cl n IS! ,- ,- r- f- ••••• ■••••• • •14 • • • • • • • •••••• ••••• • iti

O Co M M M SI n n M n n M M n n n Co Co Co i-4 tTrA tS4 t! ,-- v.:14 CA CA CA CA CA CA M •,-.1 LI_ 11111 ! ) I ! I 1 I I III r_

A'

N) ,- -0 CO 0) CAN CO IS) N "z1- n-o -4- — co CO .1-. — .,q- 0) LOCO CO 0-. 0-, MI CO -.4- r- N) N

,- sr cl- to r, — a, co ra ,- sr . uo sr ,- uo to — sr No — to n n -.ci N-3 n o, -- co co -,1.- o

= hO n ff, sr sr ra r, ISA n ,- C.,/ .7- ,- C.4 ,- NO NO CA ro m n n 6! ,- 64 ..--. 64 sr — ro — CA L) • • . • • • • . • . • • • • • • • • •••••• ••••111• ■ •■

M In M Co Co Co Co Co CA Co Co CA CA CA Co Co Co CA Co Co Co Co ,- is, Co Co n Co M Co Co n . •

I I I i I I I I I 1 I I i I 1 CI 4, L 0.4 in to uo r, Co in ,0 sr r, cl op PO CO co -c1 .....,

In ro — -o ra Co sr -o sr sr -o — uo -o No if) OD CA -0 NI ,0 CA PA CA in CA CA CA r, r, sr uo

a -- Co n CA CA n M n .,-- Co ,- Co cA ra to Co Co Co ra cA CA is, Co ,- CA ,-- CA CA ,-- Co CO . . • . • • • • •• • •• •• • • • • ••••••• •••• IR • r.-•

Co CA CA CA CA CA CA cA Co CA CA CA CA Co CA CA Co CA CA to Co :c- CA CA CA Co t5I CA CA CA Co Co 111 1 1 I I I I I I I I I I I I 1 I 0

,- ,- Co rA sr -4- co to CA AD CA CA .c... CO 0, In -0 Co *-- Cv CA V, CO In ,0 OD "T" r9 0- PO CO CA mA

CO CA CO CO OD CA CO r.-.1 sr -o r, sr v- N. Cr- CO ....- -.0 N Si sr -ID ..-- ro -4- OD r... C1 C.4 '4-- in .1.'

1.1). CA ,-- .,-. "A- ,0 CA In GA CA CA CA ,-- CA CA .-- Co N) CA Co N) CA CA In ,--. Co Co CA CA ,- co ,-. ,- wr ••••...• ••••••• • • ON• 1111•••••••14 • • •

Co M Co Co CA CA Co CA Co to Co CA CA CA CA Co CA Co Co Co ,-. Co Co Co is, Co Co M Co Co Co Co All

1 I I I I I I I I I I I I I I ri J.1

In -0 sr ,- In 1" r".. OD V, CA OD n N7 IN N. ....... 1...". 00 '0 r.0 0". n sr se r . - ----Co ,-- .,-- sr sr — ill

uo 4 — r, In NI rA to r, sr -0 CA ,-- ,- V, CA co sr PO 0, CA NI ,10 V7 co n p, sr 1,0 co ro sr 1..._

1..urats! ..-- IrIN •■•-• •Ir CAN r- N N 4 Is, Si ,- Co CA C.,A 0, rocN r.r.) CAN PONNPO tA Co tA

. • . • • • • . • • • •••• • • • •••••••••••• • I/ CA CA CA CA CA CA Ca to Co m m n n m m m m m m m m n n M Co PSI M n M M M n I I I I I I I I I I I I 1 I

er Co cA -- -43 CA In Cv If) CA CO ,-- IN N) r,.. N) -0 4 to -0 — ro — sr __ co Co u-s, u--., ,r c,.., N. N if) CA N) .1. ,0 If) Cr, In CO Cr) ,-- CA In ,--. CO CA cA N) NA V, ,0 4 *-. PO sr CO -0 -0 r.s, co ro

L.) ,- Co In CA CA ,- CA ,- CA CA Ca In CA Co PO In ..-- CA Cr. CA CA GA to -- cl is, — uo Co cA N) IC' • • • • • • • • • • , . • Y • • • • • • • •11•••••• • /1 •

M CA CA CA CA CA Co Co CA GA Co CA Co CA CA CA Co CA CA CA CA GA CA CA CA CA CA GA CA CA Co Co 1 I I 1 I 1 I 1 1 III I

• CA -.0 CA In CA ro ro CA r, oo Cr. m cA sr ,- op r, sr co Cr In sr cl r, o- sr r, ra rd -o CA

CA ,-. CA ,0 V, CA In CA ,0 CA CA -- CA .c- NA CA CA "1" CA sr No cA — uo sr m -- Cq sr ,- ro

04 CA CO GA Co CA CA Co is, CA CA CA Co CA CA CA Co GA CC) ra 15) Co Co ra Co ra is, ra ra Co Co ts! ra ••• ••X •••• • •••• • • • • 111111••••••••••

Co CA CA GA Co Co CA t.St GA CA CA M M M CA CP Co CA Co cA CA CA CA CA GA CA n M M n in Co I I I I I I I I I I I I I I

If) CA CO ^0 CA 0, CO *-- .1- IC) CA Co ..-- 0-- IN cr ,0 CO --Cl ,-. ,Ci CA CO sr ,- ,- N) 00 — Cr, Co 0-

0- n CO sr ,-- Co N. to uo N) "4" CA ,10 CA CA N. V, CA Co CO *-.. CO PO PO N NA Co CA Co NI cA --

CE In CA Co ,-. PO CA NI CA CA -/- CA ,- Co CA CA Co 0, Co +-. Co NA Co Ni ....- Co Co 15) ,- ...- ro n Co • • • . • . • . • • . • • . • • 0 • • ••••••••••111/ •

M Co Co M n 64 Co Co Co Co Co Si Co mA Co CA CA Co CA Co CA CA GA Co CA Co Co CA CA GA Co Co

I I I I I I I I I I I

SA

,- m r- CA 1,1 -4-

5- • Cr Cra C) W L. CD MC P--# -J AC MC Co CN Co Co S. ‹C 04 Lu L CO MC H --I 3= Co Co Co Co Co O 0 LL. U-

Page 189: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

189

so Tr r, Tr so so r, NO Tr ro co so r4 un .- Tr Ii) ro ro 174 un cis co co Tr Tr un cn n cl Tr PO -0 ,-- SA CA -o .•••••• •• • ••••••

KA KM SA n SA n Sti M n CC. CA M

11111 1 I CO Cl sO r.4 Tr un n Cd CA CO co r, 0) "r n

M -o Tr n 174 -o n r,, r, os Tr -o ND ro so PO CD CA n sO CO CA CA CA CA .4- sO PO ••••••••••••••••

n CC, fAtAfAn SA n tM n KM n tM n 1 I 1 I I I I I

0, CO Tr u' -- to os n n ro ro In N. c4 .tttr -4- N NI

cn n ro -- so -o n ro n so -- CA ••••••• • • • MR ••••

MMMMMMMMMMMMM CA SA MI

L7 ty N N Cr^ in in co 0- N. 0) if) VD tsi Cr re) M Ni co ow, PO ro co N.

cn n -0 CA NO Cl N) V) Tr N, ts4 M ••••• •••• •••••• MMUMMMMMMMMMM M M M 1 I I I I I I ON ro ro os CA -0 ro Tr un c-4 to -- so N. CO D' ro c4 r, Tr n co sO CA 0, 00 0, CO In

C) r4 IS, VO CA sO NO Tr n Tr c-4 r--.1 0) uo • • • • . • •• NII•••• • M MMMMMM Mi n n

1 11111

O Tj

..-..

-0 r, 0, In N. CO 0, CO sO 00 Tr- n ro ro c4 co al

cis Tr- 0- CI) *-- 1'0 ,-- SA N. ,---. N. N) 0, .4. 0, N .-A

Z n M M CO r, Tr Tr -- Tr un -- m -4- -- In IN 17. • • • . • • • • • 4 N • • • • • ,.

M M M Si Ma n n MI MI rA ,-- la KM SA MI la 115 I I I I I I I I • ■-1

r--

Tr ro CA In r, Tr to uo so to CA ND SA -- M PO 4-D Tr C) cl 0. m cr. co c-4 un so 0, r, r, Tr iv) v-.1

5- • • . • • • • • . • • • .... ID

eM la KA Ma n MI M 0) MI Cl Ca. MI M la fa M CI

I 1 I 1 I I CL 7.,

M CO 1.11 ,10 CA Tr N) so Tr ,- If) ,0 CC, M ND In ....... N. 0, in n so ,- CA -- N In 1..) N In r, In IT.

.....1 la CA sO ,0 ro --- m KM eM SA In sD *I- +- ND 0- i.." • • • • . •411•1 ••11•4 II 0

M M M M tM M SA M ,-- la MI la MI M M Si •.-t 1 I I I I I I I -iD

m

IN -- los un -- Tr Tr so LO r, 0, ,0 ,-- OD ,10 M r-1 c..4 n '4- CO 0, Si CN ,C1 ,-- -4- mr to -4- m ,- iii

1-4 ,-- rn Tr cl cq .,-- rq oo n ro ..-- Tr IN -- c.4 un !."'

11 2 N • . • . 4 4 • V V •

la la la n la la Si n M MA la M MI Si n S4 8 I 1 i 1 1 1 I 1 1 LI

CA 0- CA N. VD ND on co co so r, ,eo m oo so cl -rn

= Ul t

IZ! in N N S4 MI tO SI r- in CO ..-- i.r) MI r, Ii' • 4 • • • 0 II • • • • • . •

IM n la CI M M ,- KA n MI M IS, St IS, IS, SA .... 1 I 1 I 1 I 1 ai

1-1 °,1tr OD CO ,-- CA r, co ch so un un cr..

-- ,-. so un (NI -- „- -- -4- c-,) co .- Ni riLl 2 co ,--

CD ND w- ,-- N CA In ,-- ,-- ...- •••V In r' un -- -- ro 115 • • • . • . . . • • . • . • • •,-1

la n M M MI ,-- SI la Mt SI fA kM MS1 n .1...- SI 5- I I I I I I I I I I 4-'

n 0, '4- Lf-.I ,0 r, V) 0-, CA ...- CO N. 0, KM ,-- 5-

VI ND if) .4. NO CA CA CA .4- -- ND 0, N. *0 PO so al

L. N) CA N. Tr ro n ro Tr ,- cis ro NO In CA SI Mt • • • • • • . • . . • • • • • 0

n 1M KM SI ,-- CC) ,-- M IS, M SA fa 0) St 0) n .---, 1 I I I I I I I ......

N N r.0 N r''', .1- cis if) 0- Ni so IN ....... T... Cr. to ai

CA ,- CA ND 0, .,..I- ND Ce. 0- OD •..0 -4- '1- in UO In u

LU -- M CA CA C'-. N) 0, CA -0 '4- KM .,4' ,- ,--- SA

••••••• • •••• • 4111, .;

M f 74 M T... M M M M n St ,--• V-14 KM Si trid- n • e-I

i I I I I I i I r.-

r-, PO 0- -0 ,-- CA OD NI N:' uo -,4. In Cr, -0 0- n ml -0 OD UD ,4 la CO 0- CA cl -4- OD ,I- CA r 0

CD ND la NO CA 40 CA CO •.i" r, la *-- Tr m ro Cr' Lin u • • • • N • • • N • • • • • • M M ..T. M M M M M n IS, IZA ,-- KM IS, la

I I 1 1 I 0 -i-.)

.1- -0 ,0 '‘). .1" CO 'kl- 0, n sO CO m) -0 CO r, ,0 Li

'1' If! KM - CA SA ro ts, ro c4 to ro - CA ND V) it.

Xi MA In SI KM •-• ,- la CA CA ,- Si KM M la Mi la LL. • • R 1 • • • • • • n . • a

MI MI MI KA KM CC eM MI n SA 0) fA CM n n 15' .• I III! I I I T^

CA N n Si N. CA N) CI ,z

M ro Tr ro m N r,. M •q- 0, 15' ON

ri

••••• 4 4 ..• • •21.•

to CC) M M CC, n tM fA IS, IS, la 0) eM I I I I I I I I fro I-

,-- CA Po .4- otr ,Z CD U4 L CD 2 1-4 ...i 3: DK CD CU CU CU CU

Page 190: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

and correlation

190

CA 0, CA r- M r OD NI N. ro ON NO T... ...... ro un

=4- 4- -- CJ No rl PO CO 4- -O rq r. MI r- -O N)

CV r- M 0, r- M -4- -- un -- (....4 co m m No on • • • . . . . • • • • . . .

C m m m m m m m m m m el m Cl -- mil I 1 1 1 1

CE T-. NO M C.4 NO 1 0. r, 0, r, co -- ro un E) In

PO r.- rl No -- m M IN -4- -- ND NO r- ND r- NO

Cl -- m 1.0 M CA ND r- CA r- .q. -40 ci m ...- co ...-. • • • •

elle ••••• rr, M M M M M rsl M M M M M M M M r- M

1 1 1 1 1 I 1 1 1 1 al LI

ilD NO CO CA 121 NI LID PO r- Mr =3- -- ro 0, -- -- ,-- ._

IN IN =r 141 0, =- ND N. SI c..4 No CI-. =1- F.-.1 -,-. W

a N. CA r- r- NO CA 1.11 r- M r- PO ,- M -(I M M .,-1 • • • • ••••••••••••

MM'U

MI MI M M M MI M4 M Ma M M M M U) 1 11111 1

W

CO Cr- Ma NO Ma M DO CA ND CA CO CA r- CA r- ..-4

,- CA PO 0.1 NO 1.11 0, M CA OD CO ...r. ro .17.; ro CT- m tZ

CN r- Ma M IN VD V1 1- ,- 1471 LID 1.0 CA SI r- Mt E . . • • • . . • . . • • • • •

M M M M M M M M M M M M M M M M !:- 1 1 I I 1 1 11

4- 1471 ..4- -4- ro co ,1-. co CO ro MI NO ..4. co co m ,-

OD CA N. -4. m ip, CO CO 4- co uo t -- =1- ro c, :..:

co cl -- c, .=1.- r0 r..... ND NO CA r- Ma CA CA r- N ,...... .,i • • • • • • • ft • • • • • • r■

SI M n M M M M M M SI Si .e... Si n M 4... 4....' I !fill I I I ID

E CA Cr- N. Cr- VA N) 1.21 n CO MD CA M M ND LD

NO 11-) CA N. U-.1 ,r r, ro n CO M NO IN ND '0 CO Z CA r- +- CO N N) 1.11 M NO .4" Mt M .nr ro -4- 'N • . . • • • • • • . • • • . . . m m I m 4,-... m n m m 'Si ..-- n m m m el. i 1 1 1 1 i i I I

Cr- co co -4- r, 0, ,- 04 CA

,- 4-- CO .4- '71 i'.4-5; Pt 117- rl 12 ;:?, r L„, „

• „, ,...... ...... „.. ....... „ „ ,,, „ N. 1" r- I- ...- -- -- • • • ••••■ • •• 11•1.••

M M 'Si 'Si M 'Si M MA MI MI St n M M M4 M 1 1 1 1 1 I

1D, ,- CA !JD ND .4- M r- ...- 1- -- =1- CO N. m N.

S =I- m r.. N m c.,1 N) -- un r, -- in .1- J -- IN 1471 Si) NI CA r- ro Cr- -- un r..1 4- m c... No

• • • le • • • • • le • • • • •

MSIMMmrSIMMMMMMMisiMMA 1 1 1 1 I I

M No m -0 m in -- N N. 0, CA N. un OD M ND CA N... NO CO CO NO Cr- CA CO 'Si ND CO Cl- N. IN M

1-4 'SI el Cl r- ,-- 'Si ."- 00 CA =1- m ro m n -- IN • . • . . . . . • . • 4 • el • •

M S'S SI M M Si Si Ma Ma M M MI MI M M 1111 1 1 1 1 1

,0 -0 CA M -0 Lo ro C4 Ma ,- ,- 0- CA N) M IN ,0 un r.. =1- 4- 'SI 0, OD CA N) Ma CO ,0 Ma ,r -0

:c un CA 1.0 N. N. r- M r- r- rd ^O N ro in ro uo . . . • . . • ••• • ■• ■ ••• Si SI M Si 'S SI ..... M M n 'Si M SI n M r31

I 1 I I I 1

NI 141 n. 0) N ro N) -- ,o c, in CO ND '0 LID -0

0, CA Cr- 4- co No N) l',.. ND r 10 ma NO N. YD Co r- M r- CA CA -4. -- 'Si rl IN 4- IN 4.. cl m ,F. • • . . • • • . • . • 4 • • •

'Si M M M 'Si ...... M Mt MI Ma Ma M Ma M ,- M 1 1 1 1 1 1 1 1 1 1

CO iiD NO r- Ma .q. ro .4. ml No ,- n O. 'Si No 'S rq un n 4- N) ro -- ND N) -Cl N)

Lt. 071 CA 04 -CI LO .4- Cr, IN ,r -- O. ,r ro ,0 N CS4 • • • • • • • a et . • • • • *

M 'S M4 M ,- Ma M M M M M M4 Ma Ma Ma Ma 1 1 1 1 1 1 I 1

'S. NO 00 =1- =r- n ..zr op -,-- h7I N. CO 11'.1 I...., ID F.0 ....- NO MI NO En '0 n o- =.r. IN IN NO NO CO ,.* N''.1

W N CA CA ro 0, ro 0, -- No -Zr m un N. -- M 04 • • . • . • • . • • . . • • • 12

CSI M M r- M n M M Ma 'S r- M Ma Mia M Ma 1 1 1 I I 1 I 1

M r- r- 0, NO Ma r- CO CA rl Ma IdD N... NO ND 0, IN ro -- N. un =4- Si ro co m ro N. -- ro 0, Zr,

(...1 CA M Ma 04 CA CA NO IN -4- -- ..- m m uo .-- . • • . . • . • • 4 • VIO0•• M M r- M MI M M IMI MI Ma 'S r- M4 'Si M4 =-

I 1 1 1 1 1 1

N) r .,-- ch.. =4- ,-- (-4 ..... N. ,o 4- ro ro 'Si Cr, -0 IN NO 04 M4 =1- cl 00 N. In r- n Cr- L.... =1- ,-

Pq r- a- m CA CI 'S ...- ,- ,- .....- ,... 4$4 'Si ,- M Ma . . • . • • . • . • • • • . • •

'$4 M M 'Si M M MI 'Si m m m el m m 'Si M I 1 1 I I 1 1 1 I

r.. -- ro --• ro n CCI OD 0, .c.. uo co .1- No n ,- ro m PO r- N-.1 Ma n r- OD r- CA '0 ON U7I U^a 0

<7 N ..... 04 NI 14-1 CA In M M 'Si EN CA Ma UD r- . . • • 0 II • • • • 0 • • • • •

M M 'Si Ma M M M M M M M Mt MI MI M M I 1 1 I 1

04 ro

<X 04 C) UJ LA- CD MC 3: 2t C2 CM CV CD CV

Page 191: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

191

M n r.1 a -- r, m ,- -- a No PI CO ,.... CA co .7r CO 1"1 In ON 00 ..-- CO ,- Cr- n ,- CCI ....- C 0, Cr,

ci ....- ..-- in- -.-- N (-1 m r, 6 MI CO ,- rq NO N • • • • . • • • . • . • • • . 4 M M M M M M GA M M M M M M M4 M ...- I I I I I I I

Ca

Cr- IN .I. C..4 CO 0' .,.. C.4 '0 a c4 cy, r4 r, in r, -.1

N, MI Co EN r4 cl, r, Cr. rA .4- ..-- in 117 0, ..-- •40 N. 0

CV ....- ..-- ,C1 I.) In IN t...4.4 *4 .4- a- -ci in ro m co *4 ii... . . • 4 • II • • • • • • • • • • Cr, M *4 *4 M *4 M M M M GI MA n M M ...- ..--

I I I I I I I I I 0 4)

a- ro OD 0, 00 .....-. C4 a r, -- co a- or -o or m 5-

r4 tr) r, -4- .4- CO rei -0 -- N, co .,.- m ,0 CO ,- 0, M

CN NO M t :';'4 a m 4- rl i- ....- G4 MI ..- NO GA -....-. • • • • • • 110••• • • 11•• 4.1

M M Sr! t M GA n M M4 G4 n n G4 MI G4 M .--4 I I i I I I I I 1 M ..7

... N. ...0 04 '0 CO n *4 117 Cr. l', rA to N. in rA a,

-- ro No ro No ca a ..r 4i) N 47 trj ,D ..40 m rq m (f- cr. m -- *4 ,o vo In CA CA a -ci If) GA Cr. -- v., ,- •..•• . .. • • • . . • • . . 5-

M M et, M M M M M M M4 M4 GA GA M4 MI SI 0 I I I I I I I 4-

..:

-

-- --0 G. ND .4" Cr- 0) MI 1.0 -- *ch NO OD 14-4 c.4 •ri

- t I IN 4! V) t5.1 471 ,- m r4 m cl 00 ....- 5- • • •••••• • • • ••• 4.3

M M M M m m m -- m m m -- m

i 1 I I i i I z

o- -- -- in r- Cr- co ro vo uo -- c-4 r, a cl. r, ....,

ro --o r, No m ro -- a- r, -q- ro -- r4 ,- in ,1-. 13.1

.7tt Cl GI ..- N In NO Cq c4 a. r, ON t-S! 10 M OD 1.'4 ..--1 • • • . • • • • • .1 • • • • It • Cr,

M M GI GA M GI M Gt M M M GI M rit M M r

I I I I I I I I al •,-1

in or CA P.... r, co in r,1 -- -- r4 uo 00 .r In 10 5- CA CA CA M IN CA Cr, M Cr- *q" ro m a r, in m • • • • . • . • • • . • • • . • ,..

m m m m M M M M M .-- M M GI M M GI al I I I 1 i !a

L

.-- cr IN -- r4 co ro c.4 co ,- No ,o m r, n _,

ro ro -4- co m co *-4 in o- co co o- a -ip la uo -- ....I rl M N. NO c- tr... M VO NO ci r0 .4- PO rA In IN . • • • . • . . • . • • • • • .

M M SI M M M M4 M4 M GA M GI M4 GA M4 M 0 I I I I I I I I .1-4

.1..,

M No r4 a Cr.. tsl co co co -- rq Cr-. N op Cr- N m

I-4 M M M NO CA ...- ...... 00 CA 1.0 rl M CA ....- GA G4 04 • ••••••••• • • • • • • r■••

1 I I I 1 I I 1 o 1..1 NO NO C, IN, 0, -- NO NO CO If) -.0 ,o in co uo a

r4 -- co in ,- -o IN in n No a r, r, a a N T1

x a la a No r, -- ro -- m N N r, C4 a ,- ro r- • • • • • • • . . • . . . . • • m I I 1 I 1 I I I

a!

No -- a- r4 in IN CO m r, r, a co a n uo co ,4

CA M IN OD Cr- IN 10 NO M NO a -o re) r0 in re) Cr. CD .--. ri ts! a a- co (-4 -- r4 co co is! r, m a- ro

• • ••••••••••••• • M

M M M *4 M .I. M M M MI M M M M4 ...- M4 ..-1

I I I I I I I I I 5- CA CO M 40 in If) GI .3" 0, 40 M a ..qr v CO 1,-,1

14. ."4. .I.. .... .40 ID op m r4 r4 (.4 No ro ro uo -o -- 6 • • • • • • • • • • • • • • • • ",.

M M M M .- M ....- M M M M MI M M M M 0 I I I I I I I .-1

.....,

IV •ft •• ••• •••• • IOU It

*4 *4 M ..... M *4 *4 M M m m t la m m m m

I I I I I I I I •.-t ,:-

r, c-4 -0 co a m r, in -- co m -- cr, in co r, m o- r, Cr, uo m -- IN op co 04 OD OD rl ..-- CA XI

C.) Cl GI ,-- M N ,- ,0 GI ,0 GA ..- ,.... M ri 0, ..- C1 • • . • • • • . • . • • . . • • U

GI M .-- St M M M M GA M M -,- M M M ,- I I I I I I i S-

CI

GA 0- 00 OD 0- r, -0 M rA 0, -- .1- -- CA 0, CA -,,

04 ,- N I:13-4 n ,-- CA MI GA M M M4 ,-- ..-- GI CA -- M • . • • • • • • . • • 154 33 M i M I$ GA G14 14 37 M M GA

i I

a a- m ro ro ro -o cr n No r, co r, ▪ N. co a -4- N) a -c! c4 GA rA -o CA N, • M CA M a -- I) ro .1- • . • • . • . • • •••• NION

GIMMMMGIMMIGISIM GA Mi ra GA I I I I I

cl IN) .4- ¢ 0.4 C.) UU L. CD MC I-1 NC z CD ci CN ci CN

ro No a a ro in -o a m Cr,

-0 a a ON cr r4 r4 co a cp r, co .4- r4 No 14!

Lu m m m a cr r, or a No co co IN m in r4 I1

Table

A13

:

Page 192: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

192

If) :q• CA CO CO ON NO N, CA CA CO N. NO CO CA r, r, un CO r, co c-4 ON NT- NO PO CO ND

4-- CO CJ Csa l57 CSC CO CA ON N Vl CO

cq ro

..4C AO (-7 LU U- CD MC -J A: Z CD I= C7 CN CN cq ro

..4C AO (-7 LU U- CD MC -J A: Z CD I= C7 CN CN

CO ,-- CO CA CO CA CA CO =-

I I 1 1 I I 1 1 0

.4- '0 Cr. '0 ON CO r, CO PO In V, If) CO N.., CO Cl

ND N -0 .4-• CO N) ,- ,-- N -.0 CJ In CO CA IJO 5:-.

CII CO CO In ,-- CA V, 4-.. CA In Ni ..r -4- n -- ..4. CN Cr, • • • • • .1 • • X • • 0 • • II

CO CO n CO CO CO CO CO CO CO CO CO CA CA =-. CO ai I 1 1 1 1 I I 1 I I LI

1.171 CO CO in NO .4- CA ...1. N) ..-- N, LO CO CO In CO 64

CA CO In CO CO CO ND =- If) NO ....... ..a0 CO If) 0-. ...- CO ....1

NO ..-- -.4- -- -o m N C.4 C..1 SI -.:1- CA N, -CA 1-.1 • 4 • • 14 • • • • • X V • • • Lit

CO CA CO CA CA CA CO CO CA CA CA CO CO CA CO CO

1 11111 1 1 111 ...--1

!,-.3

• • • • • • • • 4 4 • • 4 • ft • al

in CO M M CO CO CO CO CO CO CO CA CO CO CA CO ,--1 I 1 1 1111 1 I I OD

C.:

0-. VD V) IN OD OD CO N If) CO IN +- ,- +- -0 0, M

ro ,- -- o- -- r, to -ip CJ cr. r, ro -- -,:r .,-4- .1-1

X M M SI -0 +- ro IN in m V.) NO M N SI .4- ,- r-

. • • . • . . . • • • . • 4 • • 1..:.

M CO CO M M CO M CO CO +- In CO CO CO CO Cl I I I I I

al

cr. ro so r, PO MI r0 0, CO ,0 OD OD 0% CO CA IN 9- -- CO ro CO .- un N '- c...! N CO n 44- PO CO CO

.J CO MI 01-. •‘40 CA N CO CO CO CO OD CO OD CA -0 ..r ..., • • • • • • • . . • • w • . • ..... CO CO CO CO M CO CO CO ,^ CO CO M CO CO M 1--

I I I I I I I !=.• CI

0, M PO M ,- CA IN M 0, V') C..1 .q. In ON CA -0 ••-t CA Li NO ,0 V) 0, h71 PO 0, 4.,

1-.1 .- ..- CO..-- COCOCO0, COUl COgl CO ;F.4 P! ,- ai

„„ • • • . • • • • • • r—I

CO CO CO CO CO CO CO CA CO CO CS, CO CO CO CO CO W I I I I I I I I -

r-

co -r ro ro co NO ND V, CO Cr, Cl

ON CA CO 4:1 -4- -.0 CO NO CA CO ',lc tom,

R *1-7, F;s1 t, :c n -- 1-0 ,0 N CO !J71 CO CO N) 4 ,0 MO r... ,-- or • • • . • • • . • • . . . • -1-1

CA CO CO CO CO CO =.- Mt CO CO CO CO CO CO M t-- I I I I 1 I I M

L .-- ..r CA h0 N -0 c-.1 -- CA .-4- CO CA ro -,.r un co al

CD ....... ....- .,..- .tr N) co -- -- N) -.1- CO CO ro M ..-- PO I—I • a • * • • • . • • • 4 • • • Cr,

CO CO CO CO CO +- CO CO CO CO CO CO M CO ..- M != I I I I I I I I I iri

• H ....... ..q. -.0 r, .1- -- cl co fir) cr. cq uz! -4- r, -- ...-

CJ m NO if) Cl- CO CO In ID -CI N) Cl CA CA ro co 4-, LL VI CA CA VD N) VD M M CA CA If) .1- OD IN rte) n • . • • • • . • • • • • • • 5-.•

"S., CO CO CO T— CO =-. CO CA KA CO Cr,! CO CO CO CO al 1 I ; I I I I I 2:

CO co CO o- r) v..; CA CO ..-- LO CA ON CA CA ==. ,.r 1 n-4

CO VD CO CO c:'4 CO ON OD In NO == -0 CA 1.'7:: N OD .... La CA M M +4- OD N 0,.. TT r... CO M -- r, c.-4 cq CJ

. • • ••••11.1•R 11 4 4 ID

CO CO CO -. CO CO CO CO CO CO ,- CO CA CO CA M ki 1 1 I 1 i 1 1 I if=

-4- N) co CJ -- -4.) N CO CO CO un .=;- N CO ND CO •,-1

N) -o -- ,-- -o CO c..; r, cr. N NO -0 N, == In ,-- 5.-

C..) CA CA ND ,- PO CA -,1- m m t.,...! -- c.4 m -,4- r, Vi 11) • • • • 4 • • • • • • 4 • • ::-

CA n =-. CA M M M CA ..- M CA =-. M CA CA =- 0 1 1 i I 1 1 1 I LI

ND NO In .1- NO ,0 VD Mt CA VD CA +- IN CA CO.. ,T ,.:-

VI V, In 1.0 CO CA +- .,-- ts! ,o Ls! IN 00 m ,0 m 0

Po CO n CO CO -- -- -- '- CO M CO CO CO =-- CO CO .-1- • • • • 11.1.1.11•• •4••• LI

CO CO CO CO Si CO CO CO CO CO CO CO CO CO CO CO 61

I 1 I I 1 I I I I I LL

,-4-. Vi in N N. 0- .,-r ro o, .r CO. ,- .cr <1.- co N) On

N. C..1 "0 r..., n CO C..I IN ....... .q. SI In 0.) n C..1 ..... ....1-

cr.• CO c...1 rl -a -- N ..-- CO CO CO po ...- -.0 CO ..- ,- • . • • • • • • • . • . . • • • CL SISIMMMIntn SISIMMSISISIMM

I I I I I a! r—ri -CI ii, h-

Page 193: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

193

CT- 00 r, PO ,r m PO CO In<A CP -0 r-. CO C4 CA CO ,r cr. r, No CO CO CO ON CO N) CO NO

cn CO CO cr. CO CO cq ,r CA NO +- CA CO CO NO -0 II • • • • •1111 41111 1411•0%

CO CO CO CO CO CO <A CO CO CO CO CO CO CO CO ,- I 11111 I 1 I 1

CO CO .7- vD In Ck -4- ,0 1...... 'IT C4 .4- N. vD 0, .:1- ca. .

M c?.. cq CO P, C?- ,- ,0 DO CO 0- In CA 1.3, CO r, cl, LI

Cl CO CO In CO r- CO Cl C4 C4 r- ,r No ,-- m CA CP Cl •••••••••• • • el• • • r••••

M M M M MM <A <A <A <A <A <A <A <A .-- <A Cl', I 1 1 1 1 1 I 1 1 1

......4

II) Ok CA In CA CA -0 CO N r cr. — m — cr• uo m

cq cq ,r r, CO rn CA CO 0, 00 -- In ,r 1') No p, ...1.. ..!..:,

C5 NO r- C4 r- NO CA NO <A <A ,- CA CA Cr, NO <A +- 0 • . •••••••• • • •• • • 4-, CO CO CO CO CO CO n <A <A <A <A <A <A <A <A <A I 11111 1 ill

.--1 PI +.- N In N. ,-- DO OD ,- .r- .....- M In CA ID, PO 0,

,-- Ok CA <A cr. ro m co ,r. — CA M.! -- PO +- <A z

a <A <A <A NO ro ,r- CA <A In DO CA 0, <A CA <A

.. • • • • • • • . • • • • • • !:-

<A <A <A <A <A <A <A <A Ms! <A <A KA ta <A CACI 0 I I I 1 I I 1 '+-

-O CA m ,r -lo ro -- -o r, N) ,-- CO C4 ,- CA +-

'q- N. CP. CO 03 N ro ,r ,t- r, ,-.1- co ,4- ro ro No ..-1 C0 C4 CO Ok N) Cl ,--- NO PO N) ,- <A N) CA Cl CO PO

• • • • • . • • • • • . • • 1-)

CO CO CO CO CO CO CO CO <A <A <A ....- KA I) <A ...-. M I I II I I I 1 i Z

Ok CA .7- -- 'q- ,-- rA ta NO .,-- rA <A r' 1,O 00 DO ..-..

CA DO .-- P, PO 00 <A KA PO <A <A .1. rA N CA CA al

.7-e.. ,-- CO -- co r, N) DO CO DO DO P, CO 'Zr — 'Zr N r-I • • • • • • • • • • 11••••• 17'

M M M CO M is, is, is, is, is, is, is, <A is, is, ta 0-- I I I I I I I I M

• --I

CA ,- ,-- PO DO CO PO <A ,r r, cr. m No ro m r r

— cr, --o -4) MI 03 .1- CP -0 — CA -0 -0 N. N CO 4,-, <A +- <A '1' ,-- CA CA "1' <A NO ro — ,r m --

. • . • . • • • • • • • • • . . <I CSI MI <A <A <A <A <A <A <A <A <A <A <A I:5 is, <A el 1 1 I I I 1 I CL

Ca.

<A 1-0 r.1 1'. r- 0, vD OD c‘.1 ,r ro — ND vD <A +- ...1

ro uo r, 0, ci- -0 o- -o -to uo co ,r CJ r, uo ,r ,... _..1 r- CA DO In 1'0 ,-- <A <A ,--- <A ,.1- ,1-- DO <A In CO • . • ••••••••••• • 1.• -

CO <A <A <A <A <A <A <A +..- <A CO <A <A ta <A ta 0 1 I 1 I I I I •,-1

-r-

,0 N CA CO CO N) — CO -0 CO CO CA N. -4:1 CO N. 05

CP PO In DO PO OD PO P, I.N, cr. m ci ,r r, m! h71 .--1

I-1 ...... CA PO +- ,--• <A ,-- <A <A ro m ,1- m m ro ro al • • • • •••••• • • • • ••

CA CA is, CO is, CO is, is, •-• <A is, is, is, is, is, is, is, 5.. 1 1 1 1 1 I I 1 0

,CI Ch CO OD CO ro DO .-- If) PO N. CpI- COC•!1-0 CO0:,

In VO ,r ro uo Cr ,r uo — vD CA <A DO PO <A I

. • • . . . • • • • . • • • • . m CO is, CO is, m CO r- is, is, <A is, <A CO KA <A CO

I I 1 1 I I .... iii

LO Ul r- ,- ,r m cl ci, cl co ,o cl, CO CI CP ro 1-4

CA Ul CO In 1.41 CA PO <A PO CO <A In CO PO r, ,r cl-,

C7 CA CO +-- CA CA NO +- .-- CA CA 'Zr cq Zr c%) — ,r r- • ••••••••• IS • •• •• M

M M M CO is, +- CO is, <A <A <A is, is, is, +- 's, •,-1 I I I 1 1 I 1 1 1 I r-

4.) N) bo ,-- to "Zr to C-I CI- CO No co Cs r- Cr- r- ,17.1

1'... <A r- <A <A CO Ck In <A CA CA Ck CO <A NO PO ,....

Li. 'Zr C4 Cl r, ,r ro cl, — 1.0 ....... 1,, 1.■0 1......1 ,o 04 is, ID • • ••••11.1 ••• • •11•11 ...4.

<N <A <A <A ....- <A <A <A <A CSI <A <A <A <A CA <A C1 I 1 I I 1 1 I 1 r-I

....0 O. OD C?-DO cr C,,. ,-4- m No cr. m N. Cr- If) ,-1- --

No ro m m -- uo CA N <A Cr, MA 0- CA vD Cr.. NO al

Li! C4 r- C4 Cl 0, MI 0, '- 1'... ro co 'Zr r, — CO — l..J

. . • • . . • . . . . . . . • • .7.: CO <A CO +- C: <A CO <A CO <A <A <A CO CO <A M

I I I I 1 1 I 1 ••••1

M .4" r- r- l',.. 0, co .1- r, — ,r r, DO CO N CA ,0 M

ID. <A In DO N. DO 00 <A CP DO <A <A <A ,:t. ro No :.---

L) r- CO C4 Cl C4 C4 No ,r No CO — ro CO CA N N) CI • • . • • • • . • • • . • . • • u <A CO ,-- CO <A <A <A CO CO is, <A +- CO CO CO *-

I 1 I 1 1 I 5- CI

1.0 ON. ...... ..:1- -.0 0D No .c -!-:, PO <A CO -4" PO CA N.

<A NO <A la NO Zr- m -ci co is, ro in ,--. CO CA <A LI

04 ,.... -'Zr CO -- -- CO ,..- — 11.•• T... CO CO CO is, CO is, M • • . . . • • . • ...... • 14L.. is, CO CO is, is, CO is, <A is, is, is, KA <A <A is, is, 1 1 I I I I I • •

In N. OD NO CO ,-- +- CO CA N) 0, CA DO 'Zr -- NO ,- Cr- vD CO CO NO In CO 0, PO CO CO N. OD CO CA 4

4 CO CO C4 C4 Lil Cl 1.0 ..-- ,- CO r- 04 CO ,4- CO T. •••••••.111.1 • • • • • • al

<A <A <A <A <A <A <A <A <A <A <A <A <A <A M4 <A ,-1 i I I 1 1 I .o

IV r-

ro 'Zr ¢ CO 4.) LW Li- CD MC r~ -.I NM MC cn CN CI CV CN

Page 194: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

194

ro ro i i Os CO In -4- ON 0.4 IJ 04 ▪ ro uo 0 to IN N. N. uo 04 os c, so n CI n ts! rq rq m OD n 01 CO 01 ,10

• • . • . • • • • • • •

tn I 44 M M M n 44 n n

I I I I I I I Li.

ro .4- os m us uo r, co In 0, ,- 44 mo m LID Os os Cl

CILI 44 4:1 sO 01 ro r, m .-- '73- -4. so uo MO n sO 44 I-

.• •••••••••••••• 17! 40 ra tn 41 44 M tn 44 40 ta 44 44

I I III III 11.1

P, In Os M1 01 In PO In OD LO CA N. Os n -!.. IN la r',.. 0i In .mt -4- -- In PO .cr ro 1-. o. If) .cr el cn m ro n uo m! uo m ro — m PO M r,.ts1 r.4 .4-,

. • 111.•• 4 •••0110 4 •

SI n SI SI SI Si SI M SI n n M n 44 n CI IIIII

ID Cl •-• M -40 01 cr 04 44 '4- sO If) 44 0- 44 PO ,- 4-

• 4 V • • • • • II n • • 4 •

M n Si Si 'S n M n M SI ta 44 44 44

I I I i I I 0 4-

.tr ,- CO .1- If) so r, CP- .q.- CO N.

cr- los so r., r-, to ..4. so ct rui n rq so .c.r n n x

CO CA M 0- -- CA M If) ,-- In ,..- t3 PO M CO CO T.0 •,.., • . • • . . • • • • . • • • • II r....

M n SI n M M SI IS M SI n ..... M M

IIIIIII I I I al

Z

ro -gr —N. c.,4 so r, m uo co -o t... .1- so c4 ,- In -4- -o ro IN uo r., cis so -4- o, r.14 rl Isp bo ,-

al

••••••••••• .•••• r.-4

44 44 4.4 n 4:I! M 44 44 n to 44 tn tn 44 44 41 Cr,

I 1 1 I I I I !-'

c..4 -4- c...1 so In IN VO ,0 l',.. r,.. ,- o-. ,1- .•.:- so ro •,...,

• ••••••...•••.•••

m ts4 4! Si 44 tn 44 44 44 44 44 44 44 44 44 44 5-

I I I i I I CI

• • • • • • a • • • • • • • •

SI n n SI n n M M M SI n SI SI MI n SI ..:::

I i I I i 1 1 C) •,...

LO N 00 CO In T- L) -.0 M . r ro cp.. i,--.1 .tr ,- 1....4

• ft • • • • 0 • • • II NI II • • W

Si SI SI tri t SI in P — ts, l M tSI1 MI M M M r-

I I I I I I 0

co so IN -Cl 'N 04 co M M -.1.. -40 .,-- CO 11- Os

- so N. .--N ,-- MI NI (-ANN Ir.) T- r0 In

. • • 11 • • • • • • • . . s r=

44 4.4 tn 44 44 44 ,- 44 0:i M n 44 4! 44 15) n N

I I I I I I I

NO 0- .4- ts! to un n co co to so vs, ..4- op “-1 or, 0

0- N T- lxi .gr t) N. ro cl uo uo In -...1- -4- n to ...A

•••••••• b9111.0 • • • :::::

M M MI M M ..- 44 la la 44 44 tn 44 n .- 44 IC

I I I I I I I i I • ,_, c-

MO IN 00 CA sO Cs. PO ,0 Is) 0:i 0, If) Cl ,0 4111•••••11•11,•••••

1

Ill

15)

SS . n 15) 5) S S S S 15) 15) 0:,

I I I I c

-O1-corrruo

rqmo tuto

!

„ m! -0 -o -4- co .q- ro OD -o ,0 CO CA ,- CA MO N.

0 • • • • ••••••••••••

Li

SI M M .... tn n 44 M tn 41 44 44 44 44 44 tn I'''

I I I I I I I I N . ,-,

LN los -- os uo ,- co LI, rq IN co os -,:r rq OD r-

ro IN IN r, rq uo IN r, cs ts! ...4- rq to m • • • R •11W••••• U• NI 0

44 la ,- la 44 40 n la tn la 44 ...... 44 tS) IISII ,- LI

I I I I I I I

.4- ,0 sO PO Cl-. P, CO PI ,0 .4- n uo If) 44 CA If) 0

N) -Cl CA -0 in N. in P, ...- PO MO CO ,- 415) Cr.. -1-5 P4 M) --0 n 15) ,-- M Mt M n n 0:i to =.- 0:! ,- M tj • • • . • . . . • . IIIIIIIIn • N

44 44 la tn 44 44 M M n M M n M 15 n n L!...

I I I I I I

n N) Co I.11 ro N CO If) Os N rl ro n co 04 ON T. Cr- If) CO Cr- uo .q4

tt M CA M m m „- „- rq •- • • • • • • • • • . MMMM)MMIMM)

Mi M tn 44 n I

-0

ct

13

I—

1.41 ‹C Co C..) LW LL, CD MC P.M cn CN cn an cn

0

Page 195: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

195

▪ ro tt Cr, Cr. CO Co Cl hl CA CA N) MI CO Cr'.

uo uo bl t Co r- ^ O CO r- ,- Co Co CI Os ro

Co Co r4 ro r4 n 0- Co (-4 No so

. • .

. ••• • • ••••• MMMMMS/MMI t52 M M C2 to M

I I I I I I I

co -- m oo -- r, so No .:...r .a- uo m4 No cl r, ,- o.

M in Co IN r, co Co r. r, -- ro ro cr. -o No -4- ,- .7., G -- -- so ,- IN r, -- -- ND NO IJ7 1:7 CI Ms IJ7 ,.... 0 • . • . • • • • 11 • II • • • • •

Co M Co M Co Co M Co Co M tM Co In Co ,- ...... 0%

I I I I I I I I I 1-1

.t4- un m r. Cr, 4- ro b7 CA V, sO n OD CO Co N, 111

CA ,- (-4 so ND Cr, ,- b7 CA C,1 b7 OD N7 CA N) -1-0

CV sO Co rouo Co b7 Co PO ,- Co Cl r- CO Co Cl 0 • . • . . • . . • . . • . . • 1-1 Cl Co Co C2 M Co Co Si M Co Co M Co Co MI M

I I I I IIII I iii 4.)

C7, r., ID, C4 n PO CA ID r.... 00 In oo -4- .cl- ro

ID co CA r, CA CO NO PO sO CO Cl CO CO Co N, Co M

CV Co CM Co ro 4- (-4 ro ..4- b7 b7 ..-- sO ...- Cl Co • • • • • • • • • .1 • • • • 5.■

Co Co Co 6r M Co Cl Co Co Co n M M Co Co Co 0

I I I I i I I q- 4' so -- r, so co to so ,- Co so Co Cr, N. so ..,74

••=1

co -q- r, to r, co n 4- 1..... U7 1 s0 N, Cr0, b-

CD Cl CI Cr- N) Cl n -.0 ro ro -- 15) ro -- cl co N) e.- • . • . •. . . • ••• • IS 4-1

C2 m m m Co SA MI MI in M M r- MI n MI ....... 65

I I I I I I I I 1 I E

so ro -- IN r, ro -0 Cr- N. N. uo r4 Cr-- -- r, N.

r, uo n IN ...- uo ro co n ro n -- ro In No .4- 0)

z ..--. Co ..-- co so b7 N) Cl b7 .0 M ,- .4- (5) No r4 r-1 • 4•••• • • • • ••• ••• Cr,

Co Co Co M MI M (Si Co M MI ,-- M Co M M MI r^ I I I I I I I III

•ri

sO CA -0 n CA CA CO 1.17 -q- b7 MI CO CA CO ..-- .0 f-

CO Cr. ..-- CA .4- -0 co 0- N. -- 4- ..4-. -- Cr- uo -- 4-) • rq M MV, ..-- 1" ,- rl M r, b7 *- 1- Co V7 (5) • • • • • • •••••••••• 5•-•

Co (Si Co MI Co MI M M M Co Co Co Co Co Co Co Ci

I I I I I I o o

r... r, ,-. tcr.! ,- lo- r74 -40 In 0., ...- C?.. Ln r, M T.. ....' ....1

00 VO 1- Co ,0 CA MI .4- cr uo co rz) ,0 IN N. -q1" ,... -.I *--,- .0 N. (-.4r- n ro co n -4- .4- PO CA V7 N, • • • • • •••••••• •• • f•-•

(1.4..1 Co Co Co Co M M Co Co Co Cu Co M Co Co Co CI

I I I I I I I .m.1

-0 C.4 N. C2 N. OD sO ND ...:r uo ,ID rA PO -O M

-- rs, n uo .tr MI 1/7 0, r- CO U7 CA SD r...1

F4 n r.."^ r(:,1 C.4 n c.-4 Co ro CA CA 1i' rl n CA ND 13.! • • • . • • • • • • It • • • • r_

Co Co IS, C2 M Cr M r- Co MI MI MI M Co Co Co ;:.. I I i 1 I I ! i b

Li

,0 CO N, ,- CO ND r, cr OD ,- OD 117

U7 CA PO CO rA 0, M PO CO 0, CA CA 1.17 117 127 "-I

= -,,r m Lio r, ro ro n -- ro co CA b7 CA b7 r • • • . • • • • • ••••• 115

'5) M M Co M .... M Co MI Co Co Co Co Co Co I I I 1 i 1 1 ......

el

In ro ro r4 0- 4 CO -- ro ro ro N, w- .0 ,--1

C'- 0- ,-. .0 V, Cr, r- ,0 (N .<:1- PO MI ..-- r- N.

CO r- M =- ro ro m..! n -- uo r, ....- b7 *- PO ro !F.--.

. • • . • 110•/1111111 ••• M q74 n n M M r M 454 M M M MI M M r- M

•:1 I I I I I I I I I

• • • • • • • • • • I • • •

MIMMM MIMMan tS1 tat n CSI C2 M I I I I I

113

..- CA ND 1- • CC1 C.L LU 4. CD MC 1..4 -.I NC DC CM C11 CN CV CM

41 N. Cr- IJ7 r- .q- Co co -o (-4 r, -0 .t.l. No lo... -q-

0, /.- n sO Os CA 1. .cr ts: .1- uo 0, r4 or. IN -- r-

I.J... NO r- Cl s0 .q- s0 Si N7 VO NO L7 -q- 1., al • • • • ••••• ••••

Co! n n M ...... M T.-. n M M M M M M n n 0 I I I I I I I ..-4

,...

1" CA b7 CO U7 CO Co ..4- ND r, PO -0 N) N. *0 h7 CA CA Cs VI Cs "1- PI CO ,- NO CA ,- 1- urI ID

Ud M MI n CA CO b7 CO U7 r, ..4- Cr. -4- o n rq lz1 LI • • • • • •••••• 119 • r-

m n Co -- Co Co m n Co Co Co Co (5) n Cu.Co M I I 1 1 1 ' I .--1

1., N, PO N, M -CI U7 -0 1.0 ro ro r, uo m n R:i

0, ..4- v- -- co -o m -- -- -- sO CA -0 -0 '40 0"

0 CA n CA r- CA .,..... PO sO m! =- IN M VO CO ro ID • • • • • • 4 • • • • • 0 • u m m -- m m m m m LTA m m -- MI Mi M r-

I I I I I r- o

.gt rUs 0... T.... M In T` CO try n '4. r.... .4. N 4."'

V7I V-.1 r'.I L171 M CIN. (-4 ro 0, uo .,4- .1- uo -- Cr. wo U

ing q;1 lil M Co M M r- MI SI MI M M M In MI 65 • • • Id • • •••• •• • L.

Co Co MI M C2 MI M M Co Co Co Co Co M Mi Co

I I I I I I I •• r,

-cl r.. -0 uo CA OD b7 ..,4- 9- ro o uo ro ,-

uo ro n r..1 q .q. Cr. -- o N -0 (5) .,r *-- CO CO .r.Z <E CO C2 VD Co c4 .q.- n -- rq -- ro m! uo --

Page 196: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

196

ro N ON n -ci sr ro ro r, uo ro es ,-. ro sr sr sr m CA r- M N) '.0 mo cr. sr (y, (y„ 1,0 '.0 sr (N Cl M MI Cr'. ,- M IN N) CA D7 ,..- ,- CO M S4 ^0 "0

••..n•g 1111••11011••

M M M CA n M M M n CA M tn n n ca ,- 1111111 I 1 1 1 Ci.

.....

N) 'q' CA CA N r, so mo mo cs r.., '1- ,I0 in 0

N) ro bo so sr N) CO CA bo m oo sr co -- r C.4 n ....

CD M M b7 tat CA N CA CA ro ,- sr ki7 ,-. M CA Q", ..• ..•

.• . • • • 11711119 M M MA M n M MI MI MA M o; tn n tn ,-

Si ,--1 I I I I I I I I I I M

.i...

CA CN r- OD sr 0, 1.17 r, sr m n sr sr IN ro OD 0

cq sr es mo n in co r..1 -to r, (N N. Cr, sr EN -,-,r -0 4-1 Cn T- Cl T- '0 r- NO V..;:l T- Cl r- r.S1 r., n n . . • . . . . . .. • • • 11 9 11 at

MI M CO CO MI CO CO CO CO CO Mt M CO CO MI CO LI 1 11111 1 I f=

al

mo U7 M Lea sr es OD 1%0 N in cq -0 r, Co CO M •,-i

r, .0 r‘i .,:r ,- 1.17 b7 0, CA CA CA ND PO CA sr Li

Cl CO CO CO C, ro N) tov3 bo mo r..1 CO n -- n IA . • • • • • • • • 11••••• CA CO CO CO CO CO CO M CO CO MA CO .1.... tn CO CO

1 I I I I i I I 0 C- 4 sr ..-N CA ,- n ,- ml OD U7 sr CA CD sr os so uo es so OD CO rl n mo uo -- uo co m so

O ,-. M 0. r0 rqM S: s to - n rq C••1 r, ri •.-1 • • • . . • • ■ • • •

MI M MI M M M CO CO CSI M CO 9..... CO CO CO 9.... 4.'

1 IIIII I I I M Z

CO Ca ,-• -0 M r-, Cl =- -0 a.. ,- in M r.".; .:I- r'..!

r- N ND t5.:1 NCI tS) T- Cl ND so sr PO tn bl ▪ m -- es -o mouo m uo mo n m 127 CA 1./7 r.1 Iii

. . • . • • • • • ••••••• r...1 n CO CO MI MI CO MI MI CO CO T.... CO CO M CO M 07, i i 1 i i i i i E

P5

,- 'cl- Cr- CO r, „,r -0 r0 -0 ,10 OD ,10 ,:t CO sO CO •,-f n ro CO OD .4- 1-0 U7 U7 M N ,- sr ro ON N. 'AD f- • • . • • • • • ••• 1111•••

MI M M (5 (5) CO tar) M CJ'.! CA MI ml M M CO CO f-

I I I I I I 0 Imo.

sr uo .,.- r, -o ro so sr 1.2. N 0' V.'...:.! al ,q- t,-.4

-J CO ,-. -CI -0 (N CA CO ND CO (5) in in In ..— PO N. ...... • • • • • • • • • • a • • •

M M M CO CO M CO M M CO CO M CO M M M r” I I I I I I I I Cl

,-1

UP N ON .4" sr sr n m n -- n rl r, sr os co -o r.... oo cr. sr NO +- N. OD CA 'I' CO hP ,ID n

1..4 V".. ,... 9.... Cl CO M .1.. 0' CA N) n sr -- m c.4 ro ,---1 • • • . • 1111•1111011•11•• W

MI 15) MI CO CO M SI P5.1 (Sra M n tn tn CSI n St f- 1 1 1 1 I i I !:-

Ct

,- ,- ,- • M ...- -0 ..40 b71 CO LI Cr- Mt ..0 Cr, IN ,.. In 0, -40 -0 .q• sr m sr co If) Cl" N) -- n OD CA

:C 1.0 •-• sr -..ci r, CO MI ,- CO (N in N 'nr *-10 Cq LO /-1 . • . ••••••••••••• r"..

M CO M M CSa M M (5) CO CO I I I I „.,

CO cs CO r, co "C mo r.., CO CO 1.**, L7 OD CO CO (11

M U7 N U7 Si CO '<I U7 CO -.0 '0 ...- N MI Cl CO ,I

O C..1 CO .,- N) 1'0 -0 CO CO C.4 Cq *0 =- sr IN -- N) Cr • . . . • • • • . • 4 11 11 ••11 r-

tn in CO tn n ,- M CO CO n (5) Si n CO +- Si M I I I I I I I I I .,--1

5-

ND CO CA Ch. 1"0 CO tn U7 N sr ro r,. ,-- oo rq 9..... +"

9..... n 9..... n .1... N, sr cr, -4- N. M 0... to sr uo ,-

Lt. U7 CA CA U7 CO Ul +- tn NO ,- 0-. l'.".1 '4" N N) ts! ,- . . . . • . • . 111.11*M MVO

Ill

MI CO n CO .,.. n ..... CO CO n CO MA tn CO n CO 2: 1 I 1 1 1 1 i 1 CI

.--1 Lo 'r O. -o sr 117 U7 CO CA CO ND C' .4.- sr IN -, r, r, CA co U7 CA -0 +- if) CO ,1- CO in M 117 -CI

LU +... +- CA CO U7 OD ro -ci -r m sr co -- CS) ,--- at • . • . • . • •11111111M • n• LI

tn CO MI ,...... CO CO CO 5) tn MA .r..... MA MI tn in MA v=":

I 1 1 1 I 1 I I I Ti

• 3-4

ND .4' CA a) m sr OD sr rl cr. ...lo -- mo r- CA CA CO U7 Cr' VD if) Cr' CI" CO NO CA CA -,:r ro czi

C...1 CA Si M +- CA CA U7 ,- L7 Mi ,- in tn CA ,C1 CA U R 111•11 •••• • • • • • 11 CI

CO MI 9..... CO CO (Si CO CO CO MA CO ,- CO .rm CO ,- LI 1 i i i i i 1

5-

ID --r.:1 CA CA ,- on mi 0, 117 Sr ro tm -o a ,- ,r r+7 ro un -- o, ,r ,r ,r -- .,--,- 1:,..:.! .1-;,

Pci ,- sr -- CA n ,- tn tc in M n CO CO LI R • • • • 0 0 • • 0 Ill

CO MI tn n Si Cal Si in tS4 (5) ISI in Si CO CO Lt.- I I I I I I I I

• • Cl ,0 ,.... In r`:1 Cn .:-. ,- ro ,"4"0 1'0 ,- r, Co -I- -0 r, o- ro sr n m co -o sr ro sr ,-

..4r r-, CO ,-. In Cl L7 ,- M n ,-. ,- MI U7 CO (Si ,z • % 4 9 •• %a 1.119••

CA CA CO CO CO CO CSa M CO CO Si M CO CO M Si CI I 1 I I I i r--1

ED N

1--

r."J 4:r

+a-

w C 1 LIJ LI. CD 2 I-4 3= 2.: CD CV CD C1 CN

Page 197: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

197

G1 to -71- -- m n cci m .rr ..4- IN 7- co C•.I -- -o .qr LD ..T. un In M 7- un so to NO 1.0 CP, 7" CO In ti!

cn ,- m o., m m c,.1 PO Y- r,. St CA OD M ..-- NO 7" ,...1 • •••• • ••••••• .1•• 1.11 M SI M M M MI M M M M M M M SI SI ,- I 1 I I I I 1 I ,:-.1

M

r..I tri cr, si -.- =4" ,-- ,-- Y-- CO 0, ,- CA ON CSI LID f- PO CA ro co CO -Cl 0- os No r, N. Cl CO 7- -- m

• . . • • . . ••••••••• 0

M M M MI M M Si M SI CO CO M CO CO ..-- -- -.:, I 1 I I I I I I I 0

CO r, CA Cl N 4- 4 CO CO In ,-- CA NO In CO N. :7i..,

CA "CI NO PO M CIN CA N '4) ro CN 7.- cs. -5.. m r, Ct. I

or, No m to -- un ..- un n IN -- ,- c4 n OD SA !"..: • . . • • • .• • • . • • . • •-t

CO M SI M CO CO CO M CO CO CO M M M CO M 4: I I I I I I I 4=

• r-i

N. CA In SI PO -T to -Cl -0 r, os rq r„-- (N -.. -- IN os -- r, -- ro No N. -,..r c...1 n PO SI PO ! e.:1

cn n m n NO r.) .1” Cl -- -tr so un -- op n r..1 n -1-1 . • • • • . • • • • 11•• ••

Ill

CO CO n CO M M SI CO M CO M CO CO CO M SI — I I 1 I I I Cl

N. N 0

In CA 'r/- ro ,o e.:1 ON N. OD M NO SI CO ,- a SA ,- CO OD CO OP. 0, N NO In CA PO PO 0, CO Cr`

CD PO M ON CA rl SI In CA 7" ,-. SI CA =- CA CO .,-- r- • ••••• •• • • •••• CI

CO M M M CO CO CO M CO CO CO ..-- M CO M ,-- 4- 1111111 I I I

In N VI CA NO N CO ,- =-. ro No uo r, co un IT. •,...f ro vo ro !I-) CA CA N CA n ..-- CA CA CO M NO In Z.:: M SI ,-- CO ND in ro -- if' ND N CO If) if) CA 3 • . • . . • . • . • • . • • •M

St M M M CO M M CO CO CO CO CO n M CO CO E I 1 I I I I 1

,-,

,- CA CO .cr In CA OP. *- CO PO CA r. c..! op, co co al

In ,- 4- Ni 7- No -- ro n co -Cl •q" Cr. Cr, M NO .-...1

1: ,-- ,-SI In ,-- PO CA 7- M N .14- -- ..q- CO ro CO cr., • • • • ON •••• ••II• •• r-

M M M M CO CO M M SI CO M M Si CO CO CO M I I I 1 I 1 I •.•I

!-- NO CO ,-. ,-. .q.- SI N M 0, N NO NO In N (N CO

CA OD CA CO CO CA CA CA CO PO CA n ,- Cl-.

-J M .-- NO NO CA ,- CO CA CO CO .q-. If) -i- (N -.4- CO r-. • • • • . . • ••••••••• SI

M M M M M M M M T'. MI M M M M M M El. 1 1 1 1 1 1 1 I EL

:1 ON NO OD .nr r, 7- ro No un -- un PO In NO In 00

CA N .......

,- OD CA CA CO "4' SA In ON OD ,cr up oo r...

1+4 (Si +- CA 04 CA Si =-.. CO CA if) CO CA =-. CO ,- ,-. .- • . • • • • • • • • • • • • • • Cl

m m CO CO CO m (Si M CO MI St M M (Si (Si Si •,...1 I I I I I I I -1-,

M CO In LID In In SI In CA -q- ro os PO NO N t'=! .--1

CO Si CP. PI ,-- 0- MI IN N. so r, Cr, N. m al

:c In St In NO N. M PO ,- St CA PO N 0.1 In CA LO r- • • • • • • • . • • • • • • • • r••••

M CO CO CO CO CO ,-- CO CO 'Si CO (Si Si CO M CO 0

I I I 1 I I ! LI

Cr- CA If) CO N 0-. ,-- N. CO N rq ro 0. 7- c...) CO .7,

co rq ro ,o ',,z- !In IN IN ol op -,.r to so Kt M V, r:

CD ..- M .!-- ro r) ..q. M CO CA PO If) =-- v ,-- ol ro m • • • •••1 •••• ••• 11••

M CO M M M Y-- M M CO CO (Si CO CO SI ,- CO .--. I I I I I I I I I GI

M CO CA ON 7- CO ,- CO ON N CO, CA SI CA PI St Cr,

vl r. -- Lo CO N. N. r....1 CA mt SI NO CA =.- N. n r. It. 4gt. --

IN No ro v os CA NO ,- .0 M in ..40 M. ;.., ro . • • • • . • • • . . • • ••• •,..1

MI M M M ,.... M M M M M M n M M n M r-

I I I I I I I i-,

if) CN OD NO =- N -O CO N ON CO PO ..zr Cr- .,11. ND et LLI .-- M SI

. • .,- OD 1• OD CA No 4- r, N ,.CO SI C SI tl • • . • ■ •••••••• 0

M SI M .!-- SiI CO M M M M M CI . SI M SI .---o

I I I I I I I

P.7..I CO NO CO PO r, .1- n, Cr- Np wp ro LID PO n CO 0

OD ,- P., ON NO N. ro ,- Ni LI (...) CA SI ,- SI CA ,- NO CA N SI ,- ..-- irl'T! rq

CS'! 71 • • • • ••••••••• a;

M M ,- SI M M SI M SI n MI ,- n SI M ,- ...-1

I I I I I I r- 03

'q- N ND CO CA CA ro --

-- n os r, os cr, ,- so co 'so ro >

so rq CO CA N-.1 =- NO 7" NO --.1- c,

04 St NO tSI SI =- Mt n ....- M M M M M M n St U • • • • It • ••••••••••

M 'Si M M M M CO M n( M CO CO MI CO SO r- I I I I I I I o 7- so ro No n n IN 7. -- un c...! n o., r, -- ClL

ON T- ID 1'0 h0 0, n CA NO ,- CA CO rl 0, rl In M

<E NO CO CA ,- '4" ,-- In M (SI ,- SI CA CO .t.t. -- LI_ • . • • •••••• ••• • • •

M 'Si M M M M M M CO CO M M St M Si St ..

I t I I I I Cr,

4:c

S1

..tr 04 U W IA- CO D.: 0-.1 ....I 3 CD CZ CI CS CS h-

Page 198: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

198

IN '.0 U7 Os +4' ,- +A" •-- Mt KA Cr ,- sD Tr cl, Tr in rn O. — ,o cl, n m Tr co .4' +I" ,- rn G m m MI M 0.1 r7 C-.1 ,0 M 4.- 0, M .1.- ,o r, • • ••• •11•••• ■ •11

CA to KA KA KA KA M MI KA t A CA CI KA ,- I I I I I I I 1 I

Os KA sD +- -o tr. sO ' Os -0 o •-• N. CO N) r1 '54" ,- N. 03

KA ,..- rl N •A :t CA Os CO Os ,-.- 117 t7 U7

Cl CA N N) rA +4" U7 CA T4 -.CI ,-. O. • • • s • 117•••••• .;

CA KA n KA M KA CA KA KA KA KA KA ,-.. ,- 0 I I I I I I I I I ..*-..

CP CA CO Ul CO N. r, 117 N) ,-- rq 1 un ,- ,- Tr —

c., rn -ci ..-- I.17 Os CA sO n M") CA P7 Tr -0 a- if) ,ci .-1

a ,c1 m rn m U.1 +- U7 KA CA +- ,-. CA KA sO KA ,-. 011 • • • •I• ••• ••• • •• • .I.:'

M M M t M n n n n n M KA KA KA KA n 0 I 11111 I

„.... pp, Ig t,,„ IN 0... Os 1)7 PI +- ON sO sD 'ti" U7 CA IN c.-

+- I-7 OD sD (7, CA ro OD N. U7 KA KA 0 G KA QA QA sD ND +4" CA r, .4" sD 67 ,-. ,- KA rn CM (.1- • • a • • • • • • • • • • • • •

M M M MI M M M M M M M4 M T- MI t M I I I I I 1 • 2-1

Tr Tr Tr —

N. Tr co Tr r, CA sO N. r, cl, CO la7 ..'!-;

C• ,-.. Ma Os ro cq si Lin rn rn ,--m cl c,i CA CO ND E • • • • • • • • . . . • . •

Ma QA KA KA QA KA KA KA KA n n ..-- n n n ...- --- 1111111 I 1 I Ci

....1 r', ,- -- Cr, rn in, In Tr -ci m Lin rn c..! rzi r, or!

r..1 — co m e ...2 . .%

— in

z m, m -- OD -0 M7 tgi b7 sO M .D+--ti CA (6 • • • • . • • • . • . . • • • 4.i

M n n n n iS CA KA MI t +- CA KA QA QA KA r.... I I I I i I I -I-,

NI t7 rn r, m h7 Tr Tr co Tr Lin ,o c..1 ,o cq -. r„ — ro rs! -ci r, ro cl, r.-.4 co — c, rn ai

3= M -- MI 471 ,-. rl CA NI KA CO U7 +- sO KA NI KA ctL

4 ••• ••• • • ••••■ -L.

KA KA KA Ma M4 QA KA Cu KA KA KA CA KA CA KA iS, :3 I I I I I I ---

r., sD KA IN m Tr 75t cq c-.1 co un ,- rz.4 mi ..o .c. un rn rn ,c1 co CO sD ,- CA n CA r, — cs, VZ! 0

--I KA .r. sD -.0 CA ,-- KA CA Os KA U7 NI U7 c-4 Tr r, •r-I • • • • a • • • • • I • • • • 4..,

n M M M MI M M n not ti4 ml n n n KA KA III I I I I I I I ,....1

ci

rn Tr -- c..1 Lin rn o, co IN .-ci rn rn — rn rsi rn r...

OD Tr ci-.. Tr mi -- cq cr• rn iS, m O. m m co Tr !....

1-4 cl — IN rn ,- m — cl rq Tr cq Tr ci CA ND '4" CI

••• ■ ••41•11•• •••• • LI KA KA QA QA QA CA iS, ,- KA KA KA KA KA KA KA MI

I I I i I I I I 7-1 r

-O sD CO U7 U7 Os Tr -o C. N) — r, co m — un ir, R- cl, co r, rn -- C) r„ -CI un CS! r, -o sO OD U7

T Tr m +T..' sCI N. KA Tr — Km C.4 M7 r„ rn in 1:-..1 un ..-.

. . • . • • . . • • • . . • • . al

iS, m m iS m mi -- m m m KA QA KA n iS, CS r..1 1 I 1 1 1 1 I cr:,

r

Tr un Tr 1.7 KA Tr NI CA U7 .r. CO P7 CO ND un rte) r

U7 CA Tr un un o, ro c..1 un O. ci, in co un un Tr •r•4

CO +..- to +- rn rn m m iS Cl Tr r, — ,o — Tr +4" 5- • . • • ••• ■ • •• •• ••• H.A M IS, M m m tea mn iS, KA KA KA KA KA KA .— KA

I I 1111 I I I I 5- CM

CA +- KA sO KA rm r, r, ND 1.":1 4:1 Lri ,0 sO sD :3

In Cr- 0, tt-- ,0 -,.- ro -o N M CA KA CO Os CO IN 0

4- 'V .- +- -.0 CP, N. rl +- N) CA Os Tr Tr -O Tr Is! ,H • • I. • • • a a • • • • • • a •

n n KA KA +- KA +- KA Cal CA KA KA KA KA KA KA I I I I I I I al

LI

Cl "-J .”- r- ON. cl r, N. Tr P7 ON 0:I l'''') M "CI ,- r

CA ..- ,0 ,CI Tr n OD Ch N. ,- CI '54" sD CO Cl"! IT.

14.1 KA U7 KA sO CA Tr r, un — • ,-1

M4 KA Ma .a.-- ,- iS, ,..- C4 KA CA ,-- UK".:i t:21 P: ...K.; :;).1 515 • . • . • ••• 4 •• 5.4

I I I I i I I I

La .1... cl, r, 00 U7 CA ,- Tr un IN CA N. Ca, N CA CO +- NI Tr co cl cq if Tr rn •— un iS c-A 1"."

1-1

C-1 ,-. M T- T- 01 C.4 ..10 ND sz.1 m..! ,..- KA C.4 CO Pr' r- • • • • • • • • • • • • • • • C)

KA CI +- KA KA KA KA QA KA KA C4 ,-. KA KA KA ,- -1.-, I I I i I I U

513

c..i CA Tr n Lin un ci- rn KA KA Tr rn re) la, un ro u_ c..! r, -- o, co CA 0, KA ,CI PC1 ND I'D

134 KA Tr Ca m -- qm t.T.4 --- n m rm m n qm m n .. • 4•1411••• ■■ •••••• KA

M KA IS, KA KA IS, KA KA KA IS, QA IS, KA 15, KA IS, CA I I I I I 1 I ex

+4" 1.11 Os In Os Os -,--. N. +4" M7 CO ND IS, -.0 .4:1 CO -CI ,- CO +-- IN

CI

CO ,- If) -CI Tr 1:,.:4 Tr ,- N) ,ci ,-, <lc cl, m ,o cq un N) m IS, m c..! MI U7 KA M .f.)

• •••••••••••••• a 115 KA KA KA KA KA KA KA KA CA KA KA KA IS, IS, KA h-

1 I 1 I - I I

C') N-J 04 C.) UJ li CD MC 1-4 J riC 2: CD CD CD CD CN

Page 199: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

199

REFERENCES

Page 200: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

200

Allport, G. W. & Odbert, H. S. (1936). Trait-names, a Psycho-lexical Study. Psychological Monographs 47, Whole No. 211.

Alston, W. P. (1976). Traits, Consistency and Conceputal Alternatives for Personality Theory. Chapter 4 in Harrg, R. (ed). Personality. Oxford: Basil Blackwell

Argyle, M. (1976). Personality and Social Behaviour. Chapter 6 in Harre", R. (ed). Personality. Oxford: Basil Blackwell.

Barton, K. & Cattell, R.B. (1972). Personality Characteristics of Female Psychology, Science and Art Majors. Psychological Reports 31, 807-813.

Becker, W. C. (1961). A Comparison of the Factor Structure and Other Properties of the 16PF and the Guilford-Martin Personality Inventories. Educational & Psychological Measurement 21, 393-404.

Bern, D.J. & Allen, A. (1974). On Predicting Some of the People Some of the Time: The Search for Cross-situational Consistencies in Behaviour. Psychological Review 81, 506-520.

Bock, R. D. & Petersen, A. C. (1975). A Multivariate Correction for Attenuation. Biometrika 62, 673-678.

Borgatta, E. F. (1962). The Coincidence of Subtests in Four Personality Inventories. Journal of Social Psychology 56, 227-244.

Browne, M. W. (1972a). Orthogonal Rotation to a Partially Specified Target. British Journal of Mathematical and Statistical Psychology 25, 115-120.

Browne, M. W. (1972b). Oblique Rotation to a Partially Specified Target. British Journal of Mathematical and Statistical Psychology 25, 207-212.

Burdsal, C.A. & Vaughn, D. S. (1974). A Contrast of the Personality Structure of College Students found in the Questionnaire Medium by Items as Compared to Parcels. Journal of Genetic Psychology 125, 219-224.

Cattell, R. B. (1933). Temperament Tests: II. Tests. British Journal of Psychology 23, 308-329.

Cattell, R. B. (1944). Parallel Proportional Profiles and Other Principles for Determining the Choice of Factors by Rotation. Psychometrika 9, 267-283.

Cattell, R. B. (1946). The Description and Measurement of Personality. New York: World.

Page 201: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

201

Cattell, R. B. (1950). The Main Questionnaire Factors in Questionnaire Self-Estimate Material. Journal of Social Psychology 31, 3-38.

Cattell, R. B. (1952). The Three Basic Factor Analytic Research Designs - Their Interrelations and Derivatives. Psychological Bulletin 49, 499-520.

Cattell, R. B. (1957). Personality and Motivation Structure and Measurement. New York: World.

Cattell, R. B. (1965). The Scientific Analysis of Personality. London: Penguin.

Cattell, R. B. (1966a). Handbook of Multivariate Experimental Psychology. Chicago: Rand McNally.

Cattell, R. B. (1966b). The Scree Test for the Number of Factors. Multivariate Behavioural Research 1, 140-161.

Cattell, R. B. (1972). The 16PF and Basic Personality Structure: a Reply to Eysenck. Journal of Behavioural Science 1, 169-187.

Cattell, R. B. (1973). Personality and Mood by Questionnaire. San Fransisco: Jossey-Bass.

Cattell, R. B. , Eber, H. W. , & Delhees, K. H. (1973). A Large Sample Cross Validation of the Personality Trait Structure of the 16PF with Some Clinical Implications. Multivariate Behavioural Research, Special Issue, 107-132.

Cattell, R. B. , Eber, H. W. , & Tatsuoka, M. M. (1970). Handbook for the Sixteen Personality Factor Questionnaire. Champaign, Illinois: IPAT.

Cattell, R. B. & Sullivan, W. (1962). The Scientific Nature of Factors: A Demonstration by Cups of Coffee. Behavioural Science, 7, 184-193.

Cattell, R. B. , & Tsujioka, B. (1964). The Importance of Factor Trueness and Validity versus Homogeneity and Orthogonality, in Test Scales. Educational and Psychological Measurement 24, 3-30

Cattell, R. B. , & Warburton, F. W. (1967). Objective Personality and Motivation Tests: A Theoretical Introduction and Practical Compendium. Urbana: University of Illinois Press.

Comrey, A. L. (1970). Manual for Comrey Personality Scales. San Diego: Educational and Industrial Testing Service.

Cronbach, L. J. (1951). Coefficient Alpha and the Internal Structure of Tests. Psychometrika 16, 297-334.

Page 202: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

202

Cronbach, L. J. (1975). Beyond the Two Disciplines of Scientific Psychology. American Psychologist 30, 116-127.

Edwards, A. L. (1957). The Social Desirability Variable in Personality Assessment and Research. New York: Dryden.

Eysenck, H. J. (1947). Dimensions of Personality. London: Routledge and Kegan Paul.

Eysenck, H. J. (1966). Personality and Experimental Psychology. Bulletin of the British Psychological Society 19, 1-28.

Eysenck, H. J. (1972). Primaries or Second-order Factors: a Critical Consideration of Cattell's 16PF Battery. British Journal of Social and Clinical Psychology 11, 265-269.

Eysenck, S. B. G. , & Eysenck, H. J. (1968). The Measurement of Psychoticism. British Journal of Social and Clinical Psychology 7, 286-294.

Eysenck, H. J. , & Eysenck, S. B. G. (1969). Personality Structure and Measurement. London: Routledge and Kegal Paul.

Fahrenberg, J. (1977). Physiological Concepts in Personality Research, Chapter 25 in Cattell, R. B. , & Dreger, R. M. (eds. Handbook of Modern Personality Theory. New York: Halsted Press.

Frenkel-Brunswik, E. (1942). Motivation and Behaviour. Genetic Psychology Monographs 26, 121-265.

Fruchter, B. (1954). Introduction to Factor Analysis. Princeton, N. J.: Van Nostrand.

Gray, J. (1973). Causal Theories of Personality and How to Test Them. In Royce, J. R. (ed). Multivariate Analysis and Psychological Theory. London: Academic Press.

Greif, S. (1970). Untersuchungen zur Deutschen tbersetzung des 16PF Fragebogens. Psychologische Beitrage 12, 186-213.

Guilford, J. P. (1956). The Structure of Intellect. Pscyhological Bulletin, 53, 267-293.

Guilford, J. P. (1975). Factors and Factors of Personality. Psychological Bulletin, 82, 802-814.

Guilford, J. P. , & Guilford, R. B. (1934). An Analysis of the Factors in a Typical Test of Introversion-extraversion. Journal of Abnormal and Social Psychology, 28, 377-399.

Guttman, L. (1966). Order Analysis of Correlation Matrices. Chapter 14 in Cattell, R. B. (ed.) Handbook of Multivariate Experimental Psychology. Chicago: Rand McNally.

Page 203: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

203

Guttman, L. (1954). A New Approach to Factor Analysis: The Radex, in Lazarsfeld, P. F. (ed.) Mathematical Thinking in the Social Sciences, Glencoe, Free Press.

Hakstian, A. R. (1972). Optimising the Resolution Between Salient and Non-salient Factor Pattern Coefficients. British Journal of Mathematical and Statistical Psychology, 25, 229-245.

Harre", R. (ed.) (1976). Personality. Oxford: Basil Blackwell.

Harre-, R. , & Secord, P. F. (1972). The Explanation of Social Behaviour. Oxford: Basil Blackwell.

Howarth, E. (1976). Were Cattell's "Personality Sphere" Factors Correctly Identified in the First Instance? British Journal of Psychology, 67, 213-230.

Howarth, E. , & Browne, J.A. (1971a). An Item-factor Analysis of the 16PF Personality Questionnaire. Personality: An International Journal, 2, 117-139.

Howarth, E. , & Browne, J.A. (1971b). Investigation of Personality Factors in a Canadian Context. I. Marker Structure in Personality Questionnaire Items. Canadian Journal of Behavioural Science, 3, 161-173.

Howarth, E. , Browne, J.A. , & Marceau, R. (1971). An Item Analysis of Cattell's 16PF. Canadian Journal of Behavioural Science, 4, 85-90.

JOreskog, K. G. (1970). A General Method of Analysis of Covariance Structures. Biometrika, 57, 239-251.

JOreskog, K. G. (1971). Simultaneous Factor Analysis in Several Populations. Psychometrika, 36, 409-426.

Kaiser, H. F. (1976). Review of Lawley and Maxwell's "Factor Analysis as a Statistical Method". Educational and Psychological Measurement, 36, 586-589.

Levonian, E. (1961). A Statistical Analysis of the 16 Personality Factor Questionnaire. Education and Psychological Measurement, 21, 589-596.

Levy, P. (1973). On the Relation Between Test Theory and Psychology. Chapter 1 in Kline, P. (ed.) New Approaches in Psychological Measurement. London: Wiley.

Lord, F. M. , & Novick, M. R. (1968). Statistical Theories of Mental Test Scores. Reading, Mass.: Addison-Wesley.

McGaw, B. , & JOreskog, K. G. (1971). Factorial Invariance of Ability Measures in Groups Differing in Intelligence and Socio-economic Status. British Journal of Mathematical and Statistical Psychology, 24, 154-168.

Page 204: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

204

Mischel, W. (1968). Personality and Assessment. New York: Wiley.

Mischel, W. (1973). Towards a Cognitive Social Learning Reconceptualization of Personality. Psychological Review, 80, 252-283.

Nunnally, J. (1967). Psychometric Theory. New York: McGraw Hill.

Pawlik, K. (1973). Right Answers to the Wrong Questions? A Re-examination of Factor Analytic Personality Research and its Contribution to Personality Theory, in Royce, J. R. (ed. ) Multivariate Analysis and Psychological Theory. London: Academic Press.

Royce, J. R. (1973). The Conceptual Framework for a Multi-factor Theory of Individuality, in Royce, J. R. (ed. ) Multivariate Analysis and Psychological Theory. London: Academic Press.

Saville, P. (1973). The Standardisation of an Adult Personality Inventory on the British Population. Bulletin of the British Psychological Society, 26, 25-29.

Saville, P. , & Blinkhorn, S. (1976). Undergraduate Personality by Factored Scales. Windsor: NFER.

Sells, S. B. , Demaree, R. G., & Will, D. P. (1970). Dimensions of Personality: I Conjoint Factor Structure of Cattell and Guilford Trait Markers. Multivariate Behavioural Research, 5, 391-422.

Sells, S. B. , Demaree, R. G. , & Will, D. P. (1971). Dimensions of Personality: II Separate Factor Structures in Guilford and Cattell Trait Markers. Multivariate Behavioural Research, 6, 135-185.

Timm, U. (1968). Reliabilitat und Faktorenstruktur von Cattells 16PF test bei Einer Deutschen Stichprobe. Zeitschrift fur Experimentelle und Angewante_ Psychologie,15, 354-373.

Tucker, L. R. , & Lewis, C. (1971). A Reliability Coefficient for Maximum Likelihood Factor Analysis. Topics in Factor Analysis II. Technical Report, Office of Naval Research.

University Grants Committee (1975). Statistics in Education: Universities Vol. 6. London: HMSO.

Vagg, P. R. , & Hammond, S. B. (1976). The Number and Kind of Invariant Personality (Q) Factors: a Partial Replication of Eysenck and Eysenck. British Journal of Social and Clinical Psychology, 15, 121-129.

Page 205: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

205

Van Thillo, M. , & JOreskog, K. G. (1970). SIFASP - A General Computer Program for Simultaneous Factor Analysis in Several Populations. Res. Bull. 70-62. Princeton, N. J.: Educational Testing Service.

Page 206: A RE-EXAMINATION AND ATTEMPTED REPLICATION OF. THE

cc 0

ta

.

HIGH SCORE UZSCRIPTION

A OUTGOING, WARMHEARTED, EASY-GOING, PARTICIPATING (Affectothymia, iormerly cyclothymia)

C

E

F

G

H

I

THE PRIMARY FACTOR SCALES OF THE 16PF

LOW SCORE' DESCRifri ION

RESERVED, DETACHED, CRITICAL, ALOOF

(Sizothymio)

LESS INTELLIGENT, CONCRETE- THINKING

(Lower schoicstic me-Eal capacity)

AFFECTED BY FEELINGS, EMOTIONAL-LY LESS STABLE, EAE1LY UPSET

(Lower ego strength)

HUMBLE, MILD, ACCOMMODATING, CONFORMING

(Suornissiveness)

SOBER, PRUDENT, SERIOUS, TACITURN (Deskirgenc.y)

EXPEDIENT, DISREGARDS RULES, FEELS FEW 03LIGATIONS

(Weaker superego strength) I

SHY, RESTRAINED, TIMID, THREAT-SENSITIVE

(Threctia)

TOUGH-MINDED, SELF-RELIANT, • REALISTIC, NO-NONSENSE

(Horria)

TRUSTING, ADAPTABLE, FREE OF JEALOUSY, EASY TO GET ALONG WITH (Alakia)

PRACTICAL, CAREFUL, CONVENTION-AL, REGULATED BY EXTERNAL REALITIES, PROPER (Praxemio)

FORTHRIGHT, NATURA_, ARTLESS. UNPRETENTIOUS

(Arrlessness)

SELF-ASSURED, CONFIDENT, SERENE

(Untroubled adequacy)

CONSERVATIVE, RESPECTING ESTAB- LISHED IDEAS, TOLERANT OF TRADI- TIONAL DIFFICULTIES (Conservatism)

GROUP-DEPENDENT, A "JOINER" AND SOUND FOLLONER

(Group adherence)

UNDISCIPLINED SELF-CONFLICT, FOL-LOWS OWN URGES, CARELESS OF PROTOCOL (Low integrction1

RELAXED, TRANQUIL, UNFRUSTRA. rED

((-ow ergic tersior.)

MORE INTELLIGENT, ABSTRACT- B THINIKING, BR!!".34T

(-IicEer rnento: capacity)

EMOTIONALLY STABLE, FACES REALITY, CALM MATURE (Higher ego strength)

ASSERT:VE, AGGRESSIVE, STUBBORN. COMPETITIVE (Dominance)

HAPPY-GO-LUCKY, IMPULSIVELY LIVELY, GAY, ENTHUSIASTIC (Surgency)

CONSCIENTIOUS, PERSEVERING, STAID, MORALISTIC (Stronger superego strength)

VENTURESOME, SOCIALLY BOLD, UNINHIBITED, SPONTANEOUS (Formio)

TENDER-MINDED, CLINGING, OVER-PROTECTED, SENSITIVE ;Prernsio)

SUSPICIOUS, SELF-OPINIONATED, L HARD TO FOOL

(Protension)

M

N

0

Qi

Q2

IMAGINATIVE, WRAPPED UP IN INNER _.P.GENCIES, CAE:EL ESS OF PRACTICAL (Autia) ■.'ATTEIRS, BOHEMIAN

SHREWD, CALCULATING, WORLDLY, PENETRATING (Shrewdness)

APPREHENSIVE, SELF-REPROACHING, WORRYING, TRCALED (Guilt proneness)

EXPERIMENTING, LIBERAI LNALYTICAL, FREE-THINKING

set)

SELF-SUFFICIENT, PREFERS OWN DECISICI4S, RESDURCEELIL (Self-sufiiciency)

CONTROLLED, SOCIALLY PRECISE, OLLOr. ■ NO SELF-IMAGE

ah se;(-conceE.• contra')

TENSE, 7RUSTRETED, DRIVEN, -_VERWPOLIGHT ;I-kgh ery,c tensi:n)