1

Structural validity of psychiatric scales

Jouko Miettunen, PhD

Department of Psychiatry

University of Oulu

e-mail: jouko.miettunen@oulu.fi

2

Topics of this presentation

Exploratory factor analysis Confirmatory factor analysis Structural equation modeling Cronbach’s Alpha Latent class analysis Examples

Rutter Swan

3

Exploratory factor analysis

Based on correlations between variables Dichotomous, ordinal or

continuous items Data requirements

Tests for data N should be about 5* n of variables Depends on skewness of variables

4

Exploratory factor analysis

Communalities Variables effect in EFA based on

loadings Selection of N of factors

Eigenvalues > 1 (or 1.5) Scree test Theory?

Interpreting loadings E.g. > 0.40

5

Exploratory factor analysis Factor analysis vs. principal

component analysis EFA maximizes variance

explained by all factors in solution PCA maximizes first variance

explained by 1st factor, then by next etc.

Rotation techniques Orthogonal or obligue

6

Confirmatory factor analysis For testing presented models Test statistics

Chi-square test Akaike’s Information Criteria (AIC, CAIC) Root Mean Square Error Of Approximation

(RMSEA) Goodness of Fit Index (GFI, AGFI) CFI Tucker-Lewis Index (TLI)

7

Confirmatory factor analysis

????

8

Structural equation modeling

Combination of factor analysis and regression

Continuous and discrete predictors and outcomes

Relationships among measured or latent variables

9

Structural equation modeling

Caringorientation

Expertiseorientation

Lifeorientation

Catalytic-co-operational

nursing

Controllingnursing

Confirmingnursing

•male, p=.002•older, p<.0001•no children, p=.048

•Swedish, p<.0001•older, p<.0001•no children, p=.036

•Finnish, p=.020•younger, p=.0003•sairaanhoit, p=.020•no children, p<.0001

•older, p=.034

•Swedish, p<.0001•older, p0.002

•older, p=.030

+ (r=.64)

+ (r=.11)

+ (r=.27)

+ (r=.27)

+ (r=.47)

+

+

+

+

+

+

+ (r=.22)

+ (r=.44)

+ (r=.18)

+ (r=.19)

Orientation to nursing

Orientation to learning nursing

Vanhanen-Nuutinen et al. (manuscript)

10

Structural equation modeling References

Bentler & Stein. Structural equation models in medical research. Stat Methods Med Res 1: 159–181, 1992.

Bollen. Structural equations with latent variables. John Wiley & Sons, Inc, New York, 1989.

MacCallum & Austin. Applications of structural equation modeling in psychological research. Annu Rev Psychol 51: 201–226, 2000.

11

Cronbach’s Alpha Are the items measuring same

phenomenon? For the whole scale? For subscales?

Based on variances between items Varies between 0 and 1 Improves with more items

Validity of mean of the scale not validity of one item

12

Latent class analysis

Specific statistical method developed to group subjects according to selected characteristics

Classifies subjects to groups Identifies characteristics that

indicate groups

13

Example: Anti-Social Behavior

Damaged property Fighting Shoplifting Stole <$50 Stole >$50 Use of force Seriously threaten Intent to injure

Use Marijuana Use other drug Sold Marijuana Sold hard drugs ‘Con’ somebody Stole an Automobile Broken into a building Held stolen goods Gambling Operation

National Longitudinal Survey of Youth (NLSY) Respondent ages between 16 and 23 Background information: age, gender and ethnicity N=7,326

17 antisocial dichotomously scored behavior items:

Reference:http://www.ats.ucla.edu/stat/mplus/seminars/lca/default.htm

14

Example: Anti-Social Behavior

Damage Property

Fighting Shoplifting Stole <$50 Gambling. . .

Male

Race

Age

C

15

Example: Anti-Social Behavior probabilities

16

Relationship between class probabilities and age by gender

Females Males

16 17 18 19 20 21 22 23 (age) 16 17 18 19 20 21 22 23

17

Summary of four classes: Property Offense Class (9.8%) Substance Involvement Class (18.3%) Person Offenses Class (27.9%) Normative Class (44.1%)   

Classification Table:

1 2 3 4

1 0.854 0.031 0.070 0.040

2 0.041 0.917 0.040 0

3 0.058 0.021 0.820 0.100

4 0.038 0 0.080 0.880

Example: Anti-Social Behavior

Rows:Average latent class probability for most likely latent class membership

Columns: Latent class

18

Latent class analysis

References Muthén & Muthén. Integrating person-

centered and variable-centered analyses: Growth mixture modeling with latent trajectory classes. Alcohol Clin Exp Res, 24, 882-91, 2000.

LCA in ADHD ?????????????? http://www.ats.ucla.edu/stat/mplus/

seminars/lca/default.htm More references and examples

Homepage of Mplus software: www.statmodel.com

19

Rutter items Rutter 1. Child is restless, does not have patience to sit down for along period of time Rutter 2. Stays out of school without reason Rutter 3. Wriggles and is restless Rutter 4. Often ruins and brakes his/her own or other's things Rutter 5. Fights every so often or quarrels often with other children Rutter 6. Other children don't particularly like him/her Rutter 7. Is often worried Rutter 8. Has tendency towards being alone, is quite seclusive Rutter 9. Is irritable, takes offence quickly, flares up easily Rutter 10. Seems often low-spirited, unhappy, weepy or anguished Rutter 11. Child has twitching in his/her face or compulsive movements in his/her body Rutter 12. Sucks often his/her thumb or fingers Rutter 13. Bites often nails or fingers Rutter 14. Stays out of school for unimportant reasons Rutter 15. Is often disobedient Rutter 16. Is not able to concentrate on anything for a longish period Rutter 17. Is often scared of new things or situations Rutter 18. Is meticulous pedantic Rutter 19. Lies often Rutter 20. Has stolen things once or more often Rutter 21. Is passive, slack or apathetic Rutter 22. Complains often of aches and pains Rutter 23. Child has had tears in his/her eyes when coming to school or has refused to come into

the school building Rutter 24. Child stutters Rutter 25. Gets annoyed or behaves aggressively when corrected Rutter 26. Teases other children

20

Example: Rutter

Eigenvalues1 10.982 3.793 1.734 1.625 1.126 0.94

Northern Finland 1986Birth Cohort 7-year follow-up (N=8228)

PROMAX ROTATED LOADINGS 1 2 3

________ ________ ________ ITEM1 0.910 -0.184 0.076 ITEM2 0.028 0.048 0.923 ITEM3 0.858 -0.037 0.042 ITEM4 0.850 0.001 0.069 ITEM5 0.963 -0.061 -0.084 ITEM6 0.610 0.356 -0.047 ITEM7 -0.093 0.819 -0.039 ITEM8 -0.252 0.782 0.067 ITEM9 0.684 0.310 -0.200 ITEM10 -0.002 0.825 0.107 ITEM11 0.306 0.463 -0.123 ITEM12 0.217 0.215 0.202 ITEM13 0.230 0.265 0.206 ITEM14 -0.031 0.132 0.870 ITEM15 0.978 -0.104 -0.019 ITEM16 0.753 -0.015 0.140 ITEM17 -0.178 0.869 0.035 ITEM18 0.046 0.584 -0.276 ITEM19 0.718 -0.024 0.195 ITEM20 0.660 -0.067 0.170 ITEM21 0.009 0.529 0.237 ITEM22 0.090 0.323 0.249 ITEM23 -0.009 0.581 0.258 ITEM24 0.068 0.356 -0.019 ITEM25 0.752 0.236 -0.226 ITEM26 0.959 -0.122 -0.014alpha=0.25

21

Example:

Swan

Eigenvalues1 11.382 1.683 0.62

Northern Finland 1986Birth Cohort 15-year follow-up (N=6643)

PROMAX ROTATED LOADINGS 1 2

______ ______ITEM1 0.000 0.875ITEM2 0.088 0.813ITEM3 0.364 0.438ITEM4 0.056 0.824ITEM5 -0.013 0.864ITEM6 -0.044 0.830ITEM7 0.061 0.840ITEM8 0.210 0.566ITEM9 0.259 0.533ITEM10 0.729 0.133ITEM11 0.816 0.099ITEM12 0.873 -0.031ITEM13 0.875 -0.003ITEM14 0.908 0.018ITEM15 0.794 0.051ITEM16 0.678 0.195ITEM17 0.841 0.049ITEM18 0.564 0.193

22

Example: Swan

Confirmatory factor analysis (N=5987)

CFI 0.880TLI 0.985RMSEA 0.140SRMR 0.044

23

Example: Swan

Alpha

Whole scale (21 items) 0.45 ADHD (18 items) 0.33 Attention-deficit (first 9 items) 0.25 Hyperactivity (items 10-18) 0.26