friday harbor psychometrics 2012 scientific summary uc davis / senas (spanish and english...

Friday Harbor Psychometrics 2012

Scientific SummaryUC Davis / SENAS (Spanish and English Neuropsychological

Assessment Scales)

Friday Harbor Psychometrics Workshop 2011

Acknowledgements

• Funded in part by Grant R13AG030995-01A1 from the National Institute on Aging

• The views expressed in written conference materials or publications and by speakers and moderators do not necessarily reflect the official policies of the Department of Health and Human Services; nor does mention by trade names, commercial practices, or organizations imply endorsement by the U.S. Government.

Overview

• SENAS Overview• Demographic variables and longitudinal change

• Ethnicity, clinical diagnosis, and cognitive change

• Age, education, and relationship to brain• Decomposing demographic and brain effects• Intracranial volume and cognition• Factorial invariance• Final thoughts

SENAS Overview

Purpose of SENAS Project (circa 1992)

• Create matched English and Spanish language neuropsychological tests for ages 60+ New scales based upon neuropsychological model of cognitive functioning

Scales psychometrically matched•Within English and Spanish language versions

•Between English and Spanish language versions

Distribution of item difficulty appropriate to elderly population

Approach

• Concept of item and test bias central to project

• Development and validation based on modern psychometric methods Item response theory (IRT) Latent variable modeling

• Empirically based Data based approach to making decisions about item selection and scale construction

Empirical evaluation of reliability and validity

Guiding PrinciplesInitial Scale Development

• Scales targeted to assess neuropsychologically relevant cognitive domains

• Verbal and non-verbal measures• Non-timed• New scales, not translations of existing scales

• Examiner administered• Item generation

•New items•Broad range of difficulty

SENAS Scales and Abilities Measured

Model for Item Selection

Test Information for 3MS and SENAS Object Naming

Mungas et al., 2004

Scientific Applications

Summary Conclusions / Challenge for Conference

• Demographic variables have robust effects on baseline test scores Especially ethnicity and education

• Demographic variables have minimal effects on longitudinal change Brain and disease variables account for change

African AmericanN ≈ 425

Hispanic EnglishN ≈ 450

Hispanic SpanishN ≈ 900

CaucasianN ≈ 700

English

Hispanic

Community

African AmericanN ≈ 140

HispanicN ≈ 130

CaucasianN ≈ 235

Longitudinal

Clinic

N ≈350

N ≈ 175

Cognition

Clinical

LifeExperience

Morphometric

Measures

Ethnicity, education, and cognitive change

Purpose of this study

• Understand how demographic variables relate to cognitive change

• Demographic variables of interest Race/Ethnicity Education Language of test administration

Independent Variables

• Model 1 Verbal Memory Form Previous Evaluation Spanish Administration

Prev Eval by Spanish Interaction

• Model 3 Age & Education Gender

• Model 2 Race/Ethnicity

• Model 4 Recruitment Source Clinical Diagnosis APOE Genotype

Language, Ethnicity, Education Effects

Episodic Memory

Baseline Change

Effect Estimate

SE Estimate

Intercept .62 .08

Time -.04 .02

Previous Eval

-.06 .04

Language (Spanish)

-.30 .13 -.04 .04

Span x PrevEval

.33 .12

Hispanic -.29 .11 .04 .03

African American

-.06 .08 .01 .02

Education .03 .01 -.00 .00

Language, Ethnicity, Education Effects

Executive Function

Baseline Change

Effect Estimate

SE Estimate

Intercept .32 .07

Time -.03 .02

Previous Eval

.01 .03

Language (Spanish)

-.08 .11 -.01 .03

Span x PrevEval

.08 .07

Hispanic -.32 .09 .02 .03

African American

-.30 .07 .04 .02

Education .05 .01 -.00 .00

Executive Function Trajectories by Education

Average Ethnic Group Performance by Model –

Episodic Memory

Ethnicity, clinical diagnosis, and cognitive change

Episodic Memory by Diagnosis Change

Mungas et al., 2010

African American

Hispanic

Caucasian

Age, education, and relationship to brain

MRI Effect Sizes and Age and Education Adjustment -

Episodic Memory

Mungas et al., 2009

Executive Function, MRI, Age & Education

African Americans

Executive Function

Age BM Education-.55 .00

-.03 .32.36

.39-.26

Mungas et al., 2009

Executive Function, MRI, Age & EducationHispanics

Executive Function

-.17 .46.30

.44-.40

Mungas et al., 2009

Executive Function, MRI, Age & EducationCaucasians

Executive Function

-.10 .29.29

.28-.36

Mungas et al., 2009

Age and Education Influences on

MRI - Cognition Relationships• It is possible to separate disease effects from extraneous influences

• MRI effects on cognition can obscured by demographic effects on test performance, especially when

•Demographic relationship with test score is larger than relationship with disease

•Substantial heterogeneity of demographic variable in population of interest

Decomposing demographic and brain effects

Reed et al., 2010

• 288 subjects. 158 normal, 92 MCI, 38 demented

• 96 African Americans, 74 Hispanics (32 tested in English, 42 tested in Spanish), and 118 Caucasians.

• Education M = 12.7 yrs (range 0-25)

• Age M = 74.7 yrs (range 60-93)

• Mean evaluations = 3.5; 74% had 3 or more evaluations. N of evaluations truncated at 5.

Reed et al., 2010

• Mem-D ~ 20% of episodic memory variance

• Mem-B ~ 20% of episodic memory variance

• Mem-R ~ 50% of episodic memory variance

Reed et al., 2010

Relationships of memory components with global cognitive function (CDR sum of

boxes)

DV Memory Component

Standardized Coefficient

CDR Sum Mem-D -0.09 ns

Mem-B -0.43 0.001

Mem-R -0.44 0.001

Reed et al., 2010

Relationships of memory components with clinical progression (conversion to MCI

or dementia)

Memory Component

Relative Risk Ratio (confidence interval)

Mem-D 1.58 (0.92 - 2.71)

Mem-B 0.19 (0.11 - 0.33)

Mem-R 0.27 (0.18 - 0.40)

Reed et al., 2010

Relationships of memory components with longitudinal change in cognition

(executive function)

Memory Component

Random Effect

Parameter Standard Error

Mem-D baseline 0.364 0.043 0.001

Mem-B baseline 0.180 0.031 0.001

Mem-R baseline 0.329 0.034 0.001

Mem-D change -0.011 0.010 ns

Mem-B change 0.050 0.010 0.001

Mem-R change 0.047 0.011 0.001

Reed et al., 2010

Effects of Mem-D?

• Substantial variance in Episodic Memory was uniquely related to demographic variables (ethnicity and education especially) Greater amount for other cognitive domains

• Mem-D was minimally related to clinical outcomes Especially longitudinal decline and conversion

• Challenge in clinical neuropsychological assessment is to separate demographic from brain influences on test scores

Reed et al., 2010

Intracranial volume and cognition

Farias et al., 2012

Model of joint effects of intracranial volume and brain

structure

Farias et al., 2012

Semantic Memory by ICV

Farias et al., 2012

Cognition and ICV

• ICV related to Semantic Memory and Executive Function

• ICV not related to Episodic Memory

• Early development might have impact on cortical development

Farias et al., 2012

Factorial invariance

Sample

• Community dwelling recruited using cognitive screening protocol and clinic referrals Whites - n=678 Blacks - n=352 Hispanics, English Speaking n=434 Hispanics, Spanish Speaking n=877

• Broad range of cognitive function Normal to demented

Mungas et al., 2011

Word List Learning 2

Spatial Config Learning

Word List Learning 1

Verbal Comprehension

Verbal Expression

Verbal Abstraction

Picture Association

Object Naming

Category Fluency

Phonemic Fluency

Working Memory

Verbal Attention

Visual Attention

NonVerbal Reasoning

Pattern Recognition

Spatial Localization

Best Factor Structure

Fluency

Spatial

Verbal

Memory

Attention

Mungas et al., 2011

Invariance of Dimensional Structure Across Ethnic and

Language Groups• Same number of dimensions• Invariant Factor Loadings

Observed test scores have same relationship to latent dimensions

• Some differences in Intercepts for tests Spanish speaker has to have greater latent attention ability to achieve a given Verbal Attention score

Mungas et al., 2011

Verbal Attention by Attention/Working Memory

Mungas et al., 2011

Final Thoughts

Longitudinal Trajectories

Conclusions

• Cross-sectional results can be misleading Confound lifelong ability and disease effects

• Longitudinal assessment important to separate these effects Ideally beginning in mid adulthood In absence of ideal – evidence that demographic effects on cognitive decline are small and distal

• Don’t assume a low score is impairment

References

• Farias, S. T., D. Mungas, et al. (2012). "Maximal brain size remains an important predictor of cognition in old age, independent of current brain pathology." Neurobiol Aging 33(8): 1758-1768.

• Mungas, D., B. R. Reed, et al. (2004). "Spanish and English Neuropsychological Assessment Scales (SENAS): Further development and psychometric characteristics." Psychological Assessment 16(4): 347-359.

• Mungas, D., B. R. Reed, et al. (2005). "Spanish and English Neuropsychological Assessment Scales: relationship to demographics, language, cognition, and independent function." Neuropsychology 19(4): 466-475.

References

• Mungas, D., B. R. Reed, et al. (2009). "Age and education effects on relationships of cognitive test scores with brain structure in demographically diverse older persons." Psychology and Aging 24(1): 116-128.

• Mungas, D., L. Beckett, et al. (2010). "Heterogeneity of cognitive trajectories in diverse older persons." Psychol Aging 25(3): 606-619.

• Mungas, D., K. F. Widaman, et al. (2011). "Measurement invariance of neuropsychological tests in diverse older persons." Neuropsychology 25(2): 260-269.

• Reed, B. R., D. Mungas, et al. (2010). "Measuring cognitive reserve based on the decomposition of episodic memory variance." Brain 133: 2196-2209.

Ethnic Differences and DIF

• SENAS Picture Association and Verbal Learning

• English Administration - N = 1113 396 Hispanics, 480 Whites, 237 Blacks

• Spanish Administration - N = 801 • Ability adjusted for all-source DIF Education, Ethnicity, Age

• MIMIC approach

Picture AssociationEffects of DIF Adjustment

Word List LearningEffects of DIF Adjustment

Effect of DIF Adjustment on Individual Scores (95% confidence interval)

PictureAssociation

Word ListLearning

White ±.29 s.d. ±.10 s.d.

Black ±.41 s.d. ±.02 s.d.

Hispanic - English

±.23 s.d. ±.00 s.d.

Hispanic - Spanish

±.36 s.d. ±.06 s.d.

SENAS and DIF

• DIF accounted for a very small amount of ethnic differences in Object Naming, Picture Association, Word List Learning

• DIF effects differ by domain Greater for measures of semantic memory

Smaller for episodic memory• DIF effects can potentially bias interpretation of individuals scores Especially in borderzone areas

Cross-sectional ethnic differences can be explained

by confounding variables

Semantic Memory Mean Ethnic Group Differences

friday harbor psychometrics 2012 scientific summary uc davis / senas (spanish and english...

harbor psychometrics

senas scales

matchedwithin english

matched english

approachconcept of item

purpose of senas project

senas object naming

cognitive changeage

Documents

handout on psychometrics

mairie de senas

senas ģimenes bildes #01

psychometrics ppt

senas portfolio

giunti eos psychometrics

senas ir naujas vilnius

mi mundo libro lengua de senas chilena

psychometrics slides

134965405 mi mundo libro lengua de senas chilena

psicologia - giunti psychometrics

neuropsychological diagnosis: how clinical...

l 5 psychometrics

ch 31 psychometrics

psychometrics: an introduction psychometrics: an intro

psychometrics final project

sdr-psychometrics & personality

online psychometrics

psychometrics in corporate india

senas ģimenes bildes #03