Approaches to Assessing and Correcting for Bias in Distributions

of Cognitive Ability due to Non-Response

David R. WeirJessica D. Faul

Kenneth M. LangaHealth and Retirement Study

Why focus on cognition?Surveying the cognitively

impaired population is difficult

• In contrast to income or physical health, which may affect participation but for which there is no strong theoretical reason for a particular direction of bias,

• Surveys are complex conversations that require cognitive ability to participate

• The cognitively impaired will be less likely to participate or even excluded from participation

So what?

• Cognitive impairment is the most important reason for long-term care

• Burden on families

• Cost to society

• Measuring it, and its effects, accurately is a crucial aim of “HRS” surveys

Attrition and non-response

• Baseline cross-section may underrepresent impairment

• Could get worse over time as newly impaired drop out

• There are ways to minimize this but they are not always used

Outline

• Compare HRS and ELSA non-response

• Compare HRS and ELSA bias in cognition due to non-response

• Further examine HRS

• Panel data on cognition and non-response

• Linked Medicare claims to test for bias not evident in panel observations (occurring after last interview taken)

Non-response, NOT attrition• Attrition means permanent departure from

sample– Mortality?– No, if our samples did not have mortality they

would be extremely UNrepresentative!– Both HRS and ELSA have mortality similar to

population life tables

• Permanent removal from sample is a somewhat arbitrary definition/decision

• We focus on non-response of all survivors

Cognition and mortality

• Mortality rates are higher for those with low cognition, controlling for other things

• These differentials are similar in HRS and ELSA

Mortality by Cognition, ELSA and HRS (mortality ratio age-

adjusted)

0

0.5

1

1.5

2

0 5 10 15 20

Memory Score

Ad

jus

ted

Mo

rta

lity

Ra

tio

ELSA

HRS

Response Rate of Survivors at Follow-up Waves (Age-eligible

at baseline), ELSA and HRS

2 3 4 5 6 7 8 9HRS 1992 91.8 88.8 86.2 83.6 83.2 81.7 80.1 79.4AHEAD 1993 93.8 91.8 90.1 88.2 86.3 83.8 82.6CODA 1998 93.8 91.5 89.7 88.6 87.4WB 1998 92.3 91.6 88.7 87.3 86.0EBB 2004 89.9 87.4average 92.3 90.2 88.7 86.9 85.7 82.8 81.4ELSA 2002 80.1 72.9

Follow-up WaveEntry Cohort

Baseline Year

Response Rate of Survivors at Follow-up Waves (Age-eligible

at baseline), ELSA and HRS (excluding proxies)

2 3 4 5 6 7 8 9HRS 1992 86.4 83.7 80.1 76.8 76.3 75.8 76.1 75.3AHEAD 1993 81.3 77.6 74.7 71.6 70.5 69.6 66.8CODA 1998 87.1 83.5 82.9 82.6 81.4WB 1998 85.6 83.9 82.7 83.0 82.4EBB 2004 86.2 84.1average 85.3 82.6 80.1 78.5 77.7 72.7 71.5ELSA 2002 80.1 72.9

Entry Cohort

Baseline Year

Non-response

• Non-response much higher in ELSA than in HRS

• More than twice as much

• Smaller difference when proxies are excluded from HRS

• What about bias?

How to assess bias in cognition measurement due to non-

response?• One approach: use baseline measure• Compare baseline cognition level of

respondents at follow-up wave to baseline level of all survivors to that follow-up wave

• If non-response in unrelated to baseline cognition, there would be no difference

• Difference between them measures the bias• Given the correlation of cognition and survival,

must exclude decedents at each follow-up wave because that is not bias

Bias in Cognition at Follow-up (Unweighted), ELSA and HRS

1 2 3 4 5 6 7 8HRS 0.06 0.08 0.08 0.08 0.06 0.07 0.07 0.09AHEAD 0.03 0.02 0.01 0.00 0.04 0.07 0.11CODA 0.00 0.05 0.02 0.03 0.04WB 0.08 0.10 0.06 0.06 0.06EBB 0.07 0.09average 0.05 0.07 0.04 0.04 0.05ELSA 0.20 0.21

Entry Cohort

Follow-up Wave

Non-response and bias

• ELSA had about twice the rate of non-response as HRS

• Nearly four times the bias in cognition

• Why?

• ELSA also had stronger correlation of cognition and non-response

Non-response by Cognitive Score at Prior Wave, ELSA and

HRS

0

0.1

0.2

0.3

0.4

0.5

0.6

0 2 4 6 8 10 12 14 16 18 20

Total Recall at Starting Wave

Prob

. Non

Resp

onse

Ne

xt W

ave ELSA

HRS

Non-response by Cognitive Score at Prior Wave, ELSA and HRS with and without proxy

interviews

0

0.1

0.2

0.3

0.4

0.5

0.6

0 5 10 15 20

Total Recall at Starting Wave

Prob

. Non

Resp

onse

Ne

xt W

ave ELSA

HRS

HRS self IWs

Proxy interviews

• Without proxy interviews, HRS bias in cognition from non-response would look very similar to ELSA

Bias in Cognition at Follow-up (Uneighted), ELSA and HRS

(excluding proxies)

1 2 3 4 5 6 7 8HRS 0.12 0.15 0.18 0.19 0.17 0.15 0.14 0.15AHEAD 0.27 0.37 0.42 0.44 0.43 0.38 0.46CODA 0.10 0.20 0.20 0.17 0.15WB 0.13 0.17 0.11 0.10 0.11EBB 0.08 0.11average 0.14 0.20 0.23 0.23 0.21ELSA 0.20 0.21

Entry Cohort

Follow-up Wave

Proxy Interviewing Eliminates Most of the Bias in Cognition

from Non-response

What about sample weights?

• Weights are the primary option available to surveys to correct for bias in non-response

• Do the current sample weights in HRS and ELSA correct the non-response bias in cognition?

Bias in Cognition at Follow-up (Weighted), ELSA and HRS

1 2 3 4 5 6 7 8HRS 0.05 0.05 0.05 0.04 0.03 0.08 0.06 0.08AHEAD 0.00 0.21 0.18 0.21 0.25 0.31 0.30CODA -0.01 0.10 0.06 0.12 0.09WB 0.04 0.11 0.04 0.04 0.04EBB 0.08 0.10average 0.03 0.11 0.08 0.10 0.10ELSA 0.10 0.15

Entry Cohort

Follow-up Wave

Effect of weights

• For ELSA, sample weights eliminate about half the bias in cognition due to non-response

Why are weighted numbers worse for HRS (especially

AHEAD)?• HRS weights give zero weight to nursing

home residents

• Undoes some of the advantage of proxy interviewing

• HRS now has capacity to produce weights for nursing home residents

Percent in Nursing Homes, by Age, 2006: American

Community Survey and HRS

0

5

10

15

20

65-74 75-84 85+

Age

Pe

rce

nt

in N

urs

ing

Ho

me

s

ACS

HRS

HRS represents well the nursing home population

• So unweighted numbers (including nursing home residents) should be close to weighted numbers when weights for nursing home residents are included

Comparison of bias in AHEAD cohort in 2000 and 2002 with

and without nursing home weights

UnweightedCommunity weights

Weights including nursing homes

2000 0.01 0.18 -0.032002 0 0.21 -0.01

Proxy Reporting Crucial to Representing the Cognitively

Impaired in Surveys• From the proxy we can capture their costs,

impact on family,…• What about their cognition?• Proxy/informant reports can be useful for

ascertaining dementia• But not easily comparable to scores on

cognitive testing Need a crosswalk between observed

cognition and proxy reporting

HRS and ADAMS

• HRS uses Jorm IQCODE with proxies

• Ideally, we’d want HRS cognitive scores and Jorm scores on the same people

• In HRS, get one or the other

• Next-best: have some good third measure available for both

ADAMS has many such measures

ADAMS Clinical Dementia Rating Scale, by HRS Jorm

IQcode0

12

34

cjm

ea

n

2.5 3 3.5 4 4.5 5prejint

ADAMS Clinical Dementia Rating Scale, by HRS Cognition

Score0

.51

1.5

2cm

ea

n

0 10 20 30 40precog

Crosswalk can be established

• Combine proxy and self-reports in HRS

• Track overall cognitive health

• Robust to whether interview done by proxy or self

• Can use combined data in longitudinal model of non-response by cognition

Logistic regression of non-response at wave t, by age and

cognition at wave t-1 (combining self-IWs and

proxies)

Coefficient S.E. z-statAge -0.0341 0.0017 -20.27Cognition -0.0335 0.0026 -12.87Intercept 0.0622 0.1418 0.44

So, why worry?

• Selection bias at survey baseline

• Cognitive change between waves may be related to rates of non-response

Use information external to the survey

THANK YOU

http://hrsonline.isr.umich.edu/