Approaches to Assessing and Correcting for Bias in Distributions
of Cognitive Ability due to Non-Response
David R. WeirJessica D. Faul
Kenneth M. LangaHealth and Retirement Study
Why focus on cognition?Surveying the cognitively
impaired population is difficult
• In contrast to income or physical health, which may affect participation but for which there is no strong theoretical reason for a particular direction of bias,
• Surveys are complex conversations that require cognitive ability to participate
• The cognitively impaired will be less likely to participate or even excluded from participation
So what?
• Cognitive impairment is the most important reason for long-term care
• Burden on families
• Cost to society
• Measuring it, and its effects, accurately is a crucial aim of “HRS” surveys
Attrition and non-response
• Baseline cross-section may underrepresent impairment
• Could get worse over time as newly impaired drop out
• There are ways to minimize this but they are not always used
Outline
• Compare HRS and ELSA non-response
• Compare HRS and ELSA bias in cognition due to non-response
• Further examine HRS
• Panel data on cognition and non-response
• Linked Medicare claims to test for bias not evident in panel observations (occurring after last interview taken)
Non-response, NOT attrition• Attrition means permanent departure from
sample– Mortality?– No, if our samples did not have mortality they
would be extremely UNrepresentative!– Both HRS and ELSA have mortality similar to
population life tables
• Permanent removal from sample is a somewhat arbitrary definition/decision
• We focus on non-response of all survivors
Cognition and mortality
• Mortality rates are higher for those with low cognition, controlling for other things
• These differentials are similar in HRS and ELSA
Mortality by Cognition, ELSA and HRS (mortality ratio age-
adjusted)
0
0.5
1
1.5
2
0 5 10 15 20
Memory Score
Ad
jus
ted
Mo
rta
lity
Ra
tio
ELSA
HRS
Response Rate of Survivors at Follow-up Waves (Age-eligible
at baseline), ELSA and HRS
2 3 4 5 6 7 8 9HRS 1992 91.8 88.8 86.2 83.6 83.2 81.7 80.1 79.4AHEAD 1993 93.8 91.8 90.1 88.2 86.3 83.8 82.6CODA 1998 93.8 91.5 89.7 88.6 87.4WB 1998 92.3 91.6 88.7 87.3 86.0EBB 2004 89.9 87.4average 92.3 90.2 88.7 86.9 85.7 82.8 81.4ELSA 2002 80.1 72.9
Follow-up WaveEntry Cohort
Baseline Year
Response Rate of Survivors at Follow-up Waves (Age-eligible
at baseline), ELSA and HRS (excluding proxies)
2 3 4 5 6 7 8 9HRS 1992 86.4 83.7 80.1 76.8 76.3 75.8 76.1 75.3AHEAD 1993 81.3 77.6 74.7 71.6 70.5 69.6 66.8CODA 1998 87.1 83.5 82.9 82.6 81.4WB 1998 85.6 83.9 82.7 83.0 82.4EBB 2004 86.2 84.1average 85.3 82.6 80.1 78.5 77.7 72.7 71.5ELSA 2002 80.1 72.9
Entry Cohort
Baseline Year
Non-response
• Non-response much higher in ELSA than in HRS
• More than twice as much
• Smaller difference when proxies are excluded from HRS
• What about bias?
How to assess bias in cognition measurement due to non-
response?• One approach: use baseline measure• Compare baseline cognition level of
respondents at follow-up wave to baseline level of all survivors to that follow-up wave
• If non-response in unrelated to baseline cognition, there would be no difference
• Difference between them measures the bias• Given the correlation of cognition and survival,
must exclude decedents at each follow-up wave because that is not bias
Bias in Cognition at Follow-up (Unweighted), ELSA and HRS
1 2 3 4 5 6 7 8HRS 0.06 0.08 0.08 0.08 0.06 0.07 0.07 0.09AHEAD 0.03 0.02 0.01 0.00 0.04 0.07 0.11CODA 0.00 0.05 0.02 0.03 0.04WB 0.08 0.10 0.06 0.06 0.06EBB 0.07 0.09average 0.05 0.07 0.04 0.04 0.05ELSA 0.20 0.21
Entry Cohort
Follow-up Wave
Non-response and bias
• ELSA had about twice the rate of non-response as HRS
• Nearly four times the bias in cognition
• Why?
• ELSA also had stronger correlation of cognition and non-response
Non-response by Cognitive Score at Prior Wave, ELSA and
HRS
0
0.1
0.2
0.3
0.4
0.5
0.6
0 2 4 6 8 10 12 14 16 18 20
Total Recall at Starting Wave
Prob
. Non
Resp
onse
Ne
xt W
ave ELSA
HRS
Non-response by Cognitive Score at Prior Wave, ELSA and HRS with and without proxy
interviews
0
0.1
0.2
0.3
0.4
0.5
0.6
0 5 10 15 20
Total Recall at Starting Wave
Prob
. Non
Resp
onse
Ne
xt W
ave ELSA
HRS
HRS self IWs
Proxy interviews
• Without proxy interviews, HRS bias in cognition from non-response would look very similar to ELSA
Bias in Cognition at Follow-up (Uneighted), ELSA and HRS
(excluding proxies)
1 2 3 4 5 6 7 8HRS 0.12 0.15 0.18 0.19 0.17 0.15 0.14 0.15AHEAD 0.27 0.37 0.42 0.44 0.43 0.38 0.46CODA 0.10 0.20 0.20 0.17 0.15WB 0.13 0.17 0.11 0.10 0.11EBB 0.08 0.11average 0.14 0.20 0.23 0.23 0.21ELSA 0.20 0.21
Entry Cohort
Follow-up Wave
Proxy Interviewing Eliminates Most of the Bias in Cognition
from Non-response
What about sample weights?
• Weights are the primary option available to surveys to correct for bias in non-response
• Do the current sample weights in HRS and ELSA correct the non-response bias in cognition?
Bias in Cognition at Follow-up (Weighted), ELSA and HRS
1 2 3 4 5 6 7 8HRS 0.05 0.05 0.05 0.04 0.03 0.08 0.06 0.08AHEAD 0.00 0.21 0.18 0.21 0.25 0.31 0.30CODA -0.01 0.10 0.06 0.12 0.09WB 0.04 0.11 0.04 0.04 0.04EBB 0.08 0.10average 0.03 0.11 0.08 0.10 0.10ELSA 0.10 0.15
Entry Cohort
Follow-up Wave
Effect of weights
• For ELSA, sample weights eliminate about half the bias in cognition due to non-response
Why are weighted numbers worse for HRS (especially
AHEAD)?• HRS weights give zero weight to nursing
home residents
• Undoes some of the advantage of proxy interviewing
• HRS now has capacity to produce weights for nursing home residents
Percent in Nursing Homes, by Age, 2006: American
Community Survey and HRS
0
5
10
15
20
65-74 75-84 85+
Age
Pe
rce
nt
in N
urs
ing
Ho
me
s
ACS
HRS
HRS represents well the nursing home population
• So unweighted numbers (including nursing home residents) should be close to weighted numbers when weights for nursing home residents are included
Comparison of bias in AHEAD cohort in 2000 and 2002 with
and without nursing home weights
UnweightedCommunity weights
Weights including nursing homes
2000 0.01 0.18 -0.032002 0 0.21 -0.01
Proxy Reporting Crucial to Representing the Cognitively
Impaired in Surveys• From the proxy we can capture their costs,
impact on family,…• What about their cognition?• Proxy/informant reports can be useful for
ascertaining dementia• But not easily comparable to scores on
cognitive testing Need a crosswalk between observed
cognition and proxy reporting
HRS and ADAMS
• HRS uses Jorm IQCODE with proxies
• Ideally, we’d want HRS cognitive scores and Jorm scores on the same people
• In HRS, get one or the other
• Next-best: have some good third measure available for both
ADAMS has many such measures
ADAMS Clinical Dementia Rating Scale, by HRS Jorm
IQcode0
12
34
cjm
ea
n
2.5 3 3.5 4 4.5 5prejint
ADAMS Clinical Dementia Rating Scale, by HRS Cognition
Score0
.51
1.5
2cm
ea
n
0 10 20 30 40precog
Crosswalk can be established
• Combine proxy and self-reports in HRS
• Track overall cognitive health
• Robust to whether interview done by proxy or self
• Can use combined data in longitudinal model of non-response by cognition
Logistic regression of non-response at wave t, by age and
cognition at wave t-1 (combining self-IWs and
proxies)
Coefficient S.E. z-statAge -0.0341 0.0017 -20.27Cognition -0.0335 0.0026 -12.87Intercept 0.0622 0.1418 0.44
So, why worry?
• Selection bias at survey baseline
• Cognitive change between waves may be related to rates of non-response
Use information external to the survey
THANK YOU
http://hrsonline.isr.umich.edu/