friday harbor psychometrics 2012 scientific summary uc davis / senas (spanish and english...

Friday Harbor Psychometrics 2012

Scientific SummaryUC Davis / SENAS (Spanish and English Neuropsychological

Assessment Scales)

Friday Harbor Psychometrics Workshop 2011

Acknowledgements

• Funded in part by Grant R13AG030995-01A1 from the National Institute on Aging

• The views expressed in written conference materials or publications and by speakers and moderators do not necessarily reflect the official policies of the Department of Health and Human Services; nor does mention by trade names, commercial practices, or organizations imply endorsement by the U.S. Government.

Overview

• SENAS Overview• Demographic variables and longitudinal change

• Ethnicity, clinical diagnosis, and cognitive change

• Age, education, and relationship to brain• Decomposing demographic and brain effects• Intracranial volume and cognition• Factorial invariance• Final thoughts


SENAS Overview



Purpose of SENAS Project (circa 1992)

• Create matched English and Spanish language neuropsychological tests for ages 60+ New scales based upon neuropsychological model of cognitive functioning

Scales psychometrically matched•Within English and Spanish language versions

•Between English and Spanish language versions

Distribution of item difficulty appropriate to elderly population


Approach

• Concept of item and test bias central to project

• Development and validation based on modern psychometric methods Item response theory (IRT) Latent variable modeling

• Empirically based Data based approach to making decisions about item selection and scale construction

Empirical evaluation of reliability and validity


Guiding PrinciplesInitial Scale Development

• Scales targeted to assess neuropsychologically relevant cognitive domains

• Verbal and non-verbal measures• Non-timed• New scales, not translations of existing scales

• Examiner administered• Item generation

•New items•Broad range of difficulty


SENAS Scales and Abilities Measured


Model for Item Selection


Test Information for 3MS and SENAS Object Naming

Mungas et al., 2004

Scientific Applications


Summary Conclusions / Challenge for Conference

• Demographic variables have robust effects on baseline test scores Especially ethnicity and education

• Demographic variables have minimal effects on longitudinal change Brain and disease variables account for change



African AmericanN ≈ 425

Hispanic EnglishN ≈ 450

Hispanic SpanishN ≈ 900

CaucasianN ≈ 700

English

Hispanic

Community

African AmericanN ≈ 140

HispanicN ≈ 130

CaucasianN ≈ 235

Longitudinal

Clinic

N ≈350

N ≈ 175

Cognition

Clinical

MRI

LifeExperience

Morphometric

Measures

Ethnicity, education, and cognitive change


Purpose of this study

• Understand how demographic variables relate to cognitive change

• Demographic variables of interest Race/Ethnicity Education Language of test administration


Independent Variables

• Model 1 Verbal Memory Form Previous Evaluation Spanish Administration

Prev Eval by Spanish Interaction

• Model 3 Age & Education Gender

• Model 2 Race/Ethnicity

• Model 4 Recruitment Source Clinical Diagnosis APOE Genotype


Language, Ethnicity, Education Effects

Episodic Memory


Baseline Change

Effect Estimate

SE Estimate

SE

Intercept .62 .08

Time -.04 .02

Previous Eval

-.06 .04

Language (Spanish)

-.30 .13 -.04 .04

Span x PrevEval

.33 .12

Hispanic -.29 .11 .04 .03

African American

-.06 .08 .01 .02

Education .03 .01 -.00 .00

Language, Ethnicity, Education Effects

Executive Function


Baseline Change

Effect Estimate

SE Estimate

SE

Intercept .32 .07

Time -.03 .02

Previous Eval

.01 .03

Language (Spanish)

-.08 .11 -.01 .03

Span x PrevEval

.08 .07

Hispanic -.32 .09 .02 .03

African American

-.30 .07 .04 .02

Education .05 .01 -.00 .00

Executive Function Trajectories by Education


Average Ethnic Group Performance by Model –

Episodic Memory


Ethnicity, clinical diagnosis, and cognitive change


Episodic Memory by Diagnosis Change


Mungas et al., 2010


African American



Hispanic



Caucasian


Age, education, and relationship to brain



MRI Effect Sizes and Age and Education Adjustment -

Episodic Memory

Mungas et al., 2009


Executive Function, MRI, Age & Education

African Americans

Executive Function

Age BM Education-.55 .00

-.03 .32.36

.39-.26

-.24

Mungas et al., 2009


Executive Function, MRI, Age & EducationHispanics

Executive Function


-.17 .46.30

.44-.40

.01

Mungas et al., 2009


Executive Function, MRI, Age & EducationCaucasians

Executive Function


-.10 .29.29

.28-.36

-.07

Mungas et al., 2009


Age and Education Influences on

MRI - Cognition Relationships• It is possible to separate disease effects from extraneous influences

• MRI effects on cognition can obscured by demographic effects on test performance, especially when

•Demographic relationship with test score is larger than relationship with disease

•Substantial heterogeneity of demographic variable in population of interest

Decomposing demographic and brain effects


Reed et al., 2010


Data

• 288 subjects. 158 normal, 92 MCI, 38 demented

• 96 African Americans, 74 Hispanics (32 tested in English, 42 tested in Spanish), and 118 Caucasians.

• Education M = 12.7 yrs (range 0-25)

• Age M = 74.7 yrs (range 60-93)

• Mean evaluations = 3.5; 74% had 3 or more evaluations. N of evaluations truncated at 5.

Reed et al., 2010


• Mem-D ~ 20% of episodic memory variance

• Mem-B ~ 20% of episodic memory variance

• Mem-R ~ 50% of episodic memory variance

Reed et al., 2010


Relationships of memory components with global cognitive function (CDR sum of

boxes)

DV Memory Component

Standardized Coefficient

p

CDR Sum Mem-D -0.09 ns

Mem-B -0.43 0.001

Mem-R -0.44 0.001

Reed et al., 2010


Relationships of memory components with clinical progression (conversion to MCI

or dementia)

Memory Component

Relative Risk Ratio (confidence interval)

Mem-D 1.58 (0.92 - 2.71)

Mem-B 0.19 (0.11 - 0.33)

Mem-R 0.27 (0.18 - 0.40)

Reed et al., 2010


Relationships of memory components with longitudinal change in cognition

(executive function)

Memory Component

Random Effect

Parameter Standard Error

p

Mem-D baseline 0.364 0.043 0.001

Mem-B baseline 0.180 0.031 0.001

Mem-R baseline 0.329 0.034 0.001

Mem-D change -0.011 0.010 ns

Mem-B change 0.050 0.010 0.001

Mem-R change 0.047 0.011 0.001

Reed et al., 2010


Effects of Mem-D?

• Substantial variance in Episodic Memory was uniquely related to demographic variables (ethnicity and education especially) Greater amount for other cognitive domains

• Mem-D was minimally related to clinical outcomes Especially longitudinal decline and conversion

• Challenge in clinical neuropsychological assessment is to separate demographic from brain influences on test scores

Reed et al., 2010

Intracranial volume and cognition


Farias et al., 2012

Model of joint effects of intracranial volume and brain

structure


Farias et al., 2012

Semantic Memory by ICV


Farias et al., 2012

Cognition and ICV

• ICV related to Semantic Memory and Executive Function

• ICV not related to Episodic Memory

• Early development might have impact on cortical development


Farias et al., 2012

Factorial invariance



Sample

• Community dwelling recruited using cognitive screening protocol and clinic referrals Whites - n=678 Blacks - n=352 Hispanics, English Speaking n=434 Hispanics, Spanish Speaking n=877

• Broad range of cognitive function Normal to demented

Mungas et al., 2011


Word List Learning 2

Spatial Config Learning

Word List Learning 1

Verbal Comprehension

Verbal Expression

Verbal Abstraction

Picture Association

Object Naming

Category Fluency

Phonemic Fluency

Working Memory

Verbal Attention

Visual Attention

NonVerbal Reasoning

Pattern Recognition

Spatial Localization

Best Factor Structure

Fluency

Spatial

Verbal

Memory

Attention

Mungas et al., 2011


Invariance of Dimensional Structure Across Ethnic and

Language Groups• Same number of dimensions• Invariant Factor Loadings

Observed test scores have same relationship to latent dimensions

• Some differences in Intercepts for tests Spanish speaker has to have greater latent attention ability to achieve a given Verbal Attention score

Mungas et al., 2011


Verbal Attention by Attention/Working Memory

Mungas et al., 2011

Final Thoughts


Longitudinal Trajectories


Conclusions

• Cross-sectional results can be misleading Confound lifelong ability and disease effects

• Longitudinal assessment important to separate these effects Ideally beginning in mid adulthood In absence of ideal – evidence that demographic effects on cognitive decline are small and distal

• Don’t assume a low score is impairment


References

• Farias, S. T., D. Mungas, et al. (2012). "Maximal brain size remains an important predictor of cognition in old age, independent of current brain pathology." Neurobiol Aging 33(8): 1758-1768.

• Mungas, D., B. R. Reed, et al. (2004). "Spanish and English Neuropsychological Assessment Scales (SENAS): Further development and psychometric characteristics." Psychological Assessment 16(4): 347-359.

• Mungas, D., B. R. Reed, et al. (2005). "Spanish and English Neuropsychological Assessment Scales: relationship to demographics, language, cognition, and independent function." Neuropsychology 19(4): 466-475.


References

• Mungas, D., B. R. Reed, et al. (2009). "Age and education effects on relationships of cognitive test scores with brain structure in demographically diverse older persons." Psychology and Aging 24(1): 116-128.

• Mungas, D., L. Beckett, et al. (2010). "Heterogeneity of cognitive trajectories in diverse older persons." Psychol Aging 25(3): 606-619.

• Mungas, D., K. F. Widaman, et al. (2011). "Measurement invariance of neuropsychological tests in diverse older persons." Neuropsychology 25(2): 260-269.

• Reed, B. R., D. Mungas, et al. (2010). "Measuring cognitive reserve based on the decomposition of episodic memory variance." Brain 133: 2196-2209.



Ethnic Differences and DIF

• SENAS Picture Association and Verbal Learning

• English Administration - N = 1113 396 Hispanics, 480 Whites, 237 Blacks

• Spanish Administration - N = 801 • Ability adjusted for all-source DIF Education, Ethnicity, Age

• MIMIC approach


Picture AssociationEffects of DIF Adjustment


Word List LearningEffects of DIF Adjustment


Effect of DIF Adjustment on Individual Scores (95% confidence interval)

PictureAssociation

Word ListLearning

White ±.29 s.d. ±.10 s.d.

Black ±.41 s.d. ±.02 s.d.

Hispanic - English

±.23 s.d. ±.00 s.d.

Hispanic - Spanish

±.36 s.d. ±.06 s.d.


SENAS and DIF

• DIF accounted for a very small amount of ethnic differences in Object Naming, Picture Association, Word List Learning

• DIF effects differ by domain Greater for measures of semantic memory

Smaller for episodic memory• DIF effects can potentially bias interpretation of individuals scores Especially in borderzone areas

Cross-sectional ethnic differences can be explained

by confounding variables



Semantic Memory Mean Ethnic Group Differences

friday harbor psychometrics 2012 scientific summary uc davis / senas (spanish and english...

Documents

harbor psychometrics

senas scales

matchedwithin english

matched english

approachconcept of item

purpose of senas project

senas object naming

cognitive changeage