SUB GROUP BIAS IN PUBLIC HEALTH RESEARCHApplying survey over coverage methodology to health disparities research

Naomi Zewde, MPH and Rhonda Belue, PhD

Presenter Disclosures

The following personal financial relationships with commercial interests relevant to this presentation existed during the past 12 months:

No relevant relationships to disclose

Increasing immigration challenges racial classification

• EG: Classifying Hispanic ethnicity when the individual is also either black or white

• ACA charged the DHHS with revising standards of race/ ethnicity data collection1

“While data alone will not reduce disparities, it can be foundational in our efforts to understand the causes, design effect responses and evaluate our progress.”

Obesity Rates among Hispanic and Non-Hispanic White American Adults 2012

Non- Hispanic White Hispanic0.27

0.28

0.29

0.3

0.31

0.32

0.33

Obesity Rate

Sub group bias

• Documented health outcome variation by:2-4

• Socio-economic status• Country of origin• Immigrant status• Combined effects

• African immigrants report higher health than US born whites, while US, West Indian and European born blacks do not

Obesity Rates among Hispanic and Non-Hispanic White American Adults 2012

Non-H

ispan

ic W

hite

Puerto

Rica

n

Mex

ican

Oth

er C

entra

l/ S. A

mer

ican

Oth

er H

isp/ L

atino

Cuban

Domini

can

0.15

0.2

0.25

0.3

0.35

0.4

Obesity Rate

Obesity Rate

Sub group inefficiency• Large sample sizes enable precise estimates

• Racial groups are necessarily larger than sub-groups • There are necessarily more Black Americans than there

are middle-income or African Americans.

Precision - bias tradeoff

• Sub-group analysis is statistically inefficient if the original results are unbiased• No practical difference in outcomes; or• Sub-group is a majority, thus driving the results

• Our paper suggests a method of quantifying the tradeoff between precision and bias.

Objectives

This paper draws a conceptual and methodological parallel between survey over coverage bias and sub- group bias in health disparities research to:

1. Demonstrate a method of quantifying sub-group bias

2. Demonstrate a method of identifying the relative statistical efficiency of using sub-group data

Over coverage bias: sampled persons are not part of the target population

One to one correspondence

F------------T...

F------------T

Over coverage

F-------------...

F------------T

Sub-group : over coverage parallel• Sub group members serve as the target population

• Example: Puerto Rican Americans

• Non sub group members are overrepresented in the data• Non- Puerto Rican Hispanic Americans

Sub-group : over coverage parallel

• Over coverage and sub-group bias each occur when unintended observations contribute to sample statistics

• Survey methodology identifies two drivers of over coverage bias5

• Difference in outcome between foreign and targeted units• Proportion of foreign vs. targeted elements

Applied example• Obesity prevalence among Hispanic Americans

• Obesity is a growing public health concern• Risk factors are correlated with cultural variation

• Data source• 2012 Medical Expenditure Panel Survey (MEPS)• Identifies Hispanic ethnicity across six countries of origin

Mean bias5

Full sample mean

Number of foreign elements

Full sample size Mean of foreign elements

Mean of target population

Ethnic Group Sample Size

Obesity Rate Obesity Ratio to N.H. W.

Bias* Relative Bias

Hispanic 7,446 32.58 1.10 -- --

N.H. White 11,319 29.521.00

-- --

Reported obesity Statistics from MEPS 2012 represent non-institutionalized American adults.* Demonstration of Bias from using obesity statistic calculated on Hispanic ethnicity (Szameitat and Schafer, 1963)

Ethnic Group Sample Size

Obesity Rate Obesity Ratio to N.H. W.

Bias* Relative Bias

Hispanic 7,446 32.58 1.10 -- --

N.H. White 11,319 29.521.00

-- --

Central/S. American1,223

25.74 0.87 6.77 26.32

Dominican

307 27.49 0.93 5.10 18.56

Reported obesity Statistics from MEPS 2012 represent non-institutionalized American adults.* Demonstration of Bias from using obesity statistic calculated on Hispanic ethnicity (Szameitat and Schafer, 1963)

Ethnic Group Sample Size

Obesity Rate Obesity Ratio to N.H. W.

Bias* Relative Bias

Hispanic 7,446 32.58 1.10 -- --

N.H. White 11,319 29.521.00

-- --

Central/S. American1,223

25.74 0.87 6.77 26.32

Dominican

307 27.49 0.93 5.10 18.56

Puerto Rican

687 36.99 1.25 -4.46 -12.07

Other Hisp/ Latino

303 28.59 0.97 4.06 14.22

Cuban

336 35.15 1.19 -2.56 -7.27

Mexican

4,590 34.10 1.16 -1.45 -4.25

Reported obesity Statistics from MEPS 2012 represent non-institutionalized American adults.* Demonstration of Bias from using obesity statistic calculated on Hispanic ethnicity (Szameitat and Schafer, 1963)

Mean bias in Hispanic ethnic sub-groups

Statistical Efficiency• Efficiency can be measured by relative mean squared

error• Rewards sample size• Penalizes unexplained variation and bias

• Relative efficiency of sub-group analysis is ambiguous apriori

Relative Efficiency

Relative Mean Squared Error=

Relative efficiency of Hispanic ethnic sub- groups

Ethnic Group Sample Size Relative Bias MSE Relative MSE*

Hispanic 7,446 -- 1.05E-04 --

Central/S. American 1,223 26.32 4.07E-02 11.77

Puerto Rican 687 -12.07 9.18E-02 2.23

Mexican 4,590 -4.25 1.64E-02 2.05

Reported obesity statistics from MEPS 2012 represents non-institutionalized American adults. *Ratio of sub- group MSE to full sampling frame MSE (all Hispanic)

Ethnic Group Sample Size Relative Bias MSE Relative MSE*

Hispanic 7,446 -- 1.05E-04 --

Central/S. American 1,223 26.32 4.07E-02 11.77

Puerto Rican 687 -12.07 9.18E-02 2.23

Mexican 4,590 -4.25 1.64E-02 2.05

Dominican 307 18.56 2.74E-01 0.99

Other Hisp/ Latino 303 14.22 1.82E-01 0.93

Cuban 336 -7.27 2.03E-01 0.38Reported obesity statistics from MEPS 2012 represents non-institutionalized American adults. *Ratio of sub- group MSE to full sampling frame MSE (all Hispanic)

Non H

ispan

ic W

hite

Puerto

Rica

n

Mex

ican

Domini

can/

Cub

an/O

ther

Hisp

. Lat

ino

CS Am

erica

n0.00

0.10

0.20

0.30

0.40

0.50

Obesity Among Hispanic and Non-Hispanic Amer-ican Adults, 2012

Obesity Rate

Discussion• Over coverage methodology provides a concrete tool to

assess the tradeoff between precision and bias to present racial ethnic minority findings

• Mean bias has been demonstrated in survey over coverage methodology, future research is needed to identify bias in other statistics, including regression coefficients.

References

1. U.S. Department of Health and Human Services. (2011, October). Implementation

guidance on data collection standards for race, ethnicity, sex, primary language and disability

status. Retrieved from: http://aspe.hhs.gov/datancl/standards/ACA/4302

2. Read, J. G., Emerson, M. O., & Tarlov, A. (2005). Implications of black immigrant health

for U.S. racial disparities in health. Journal of Immigrant Health , 205-212.

3. National Research Council. (2004). Eliminating health disparities: Measurement and data

needs. Panel on DHHS Collection of Race and Ethnicity Data, Committee on National

Statistics.Washington, DC: National Academies Press.

4. Liang, J., Van Tran, T., Krause, N., and Markides, K. S.Generational differences in the

structure of the CES-D Scale in Mexican Americans.Journal of Gerontology: Social

Sciences44(1989).5110–5120.

5. Szameitat, K., & Schaffer, K. A. (1963). Imperfect Frames in Statistics and the

consequences for their use in sampling. Bulletin of the International Statistical Institute ,

40, pp. 517- 544.