jsm 2012, san diego1 caution should be used in applying propensity scores estimated in a full cohort...

JSM 2012, San Diego 1

Caution should be used in applying propensity scores estimated in a full cohort to adjust for confounding in subgroup analyses

Sue M. Marcus, Columbia University

Robert D. Gibbons, University of Chicago


Testimony of Andrew Leon:Medication and Veteran Suicide

• ‘All of us here today share a common goal: to do the very best for our veterans’

• ‘doing the best requires the discipline to use empirical methods to understand optimal mental health care and prevention of suicide.’


Outline: Caution should be used…

• Context: automated propensity score analyses of large observational databases for drug safety surveillence

• When to use caution (Rosenbaum and Rubin 1983; Marcus and Gibbons 2012)

• Illustration: Do antiepileptic drugs cause suicide?


Drug Safety

• Spontaneous reports collected through FDA’s Adverse Event Reporting System

• Analysis of large-scale integrated medical claims data

• Large potential for bias


Propensity scores estimated in full cohort for subgroup?

• If so, one step closer to automated drug safety system for which separate analysis for each subgroup is unnecessary

• A correctly specified propensity score should (at least in expectation) remain valid in a subgroup population (Rosenbaum and Rubin 1983)

• When can this go wrong?


Illustration: Do AEDs cause suicide?

• 1/2008 FDA alert: AEDs can increase suicidal thoughts and behaviors

• 7/2008 FDA scientific advisory committee: association between AEDs and suicidality

• American Epilepsy Society: unintended dire consequences, do not want to discontinue effective seizure medication if it does not cause suicide


Causal question?

• AEDs given for bipolar disorder, major depression, epilepsy, pain disorders, migraines, alcohol craving, others

• Do AEDs cause suicide or do people with higher propensity for suicide tend to have higher propensity to take AEDs?

• Goal: disentangle who takes AEDs from the biological effect of the drugs


Conflicting conclusions following FDA alert for two propensity–score adjusted analyses

Paper Gibbons et al 2009

Patorno et al

2010

Population Bipolar Disorder BD, epilepsy, migraine, pain

Comparison AED vs no AED Each AED vs topiramate

Conclusion AEDs do not increase SA

Some AEDs may have increased risk


AED A (↑BP) vs AED B (↑epilepsy)

• Answers public health question: more suicide among those who take A vs B?

• Does not address whether cause of suicide is biological effect of drug or reflects who is taking drug

• Higher suicide rate for A reflects higher suicide rate for BP compared to epilepsy


Correct specification for full vs subgroup

• Propensity to use drug depends on different characteristics for different disorders (eg bipolar disorder vs epilepsy)

• Can we correctly specify propensity for each subgroup using full cohort?

• Propensity to use AED vs Topiramate does not balance comparison of AED vs no treatment for particular disorder


Potential Outcomes Framework

• r1= response if AED, r0 = response if no AEDZ = 1 for AED, = 0 for no AED

• in general, E (r1 - r0 ) is not equal to E (r1| Z = 1) – E (r0 | Z = 0)

• E (r1 - r0 ) may be equal to E (r1| Z = 1, x) – E (r0 | Z = 0, x)


What is being estimated?

• Gibbons et alE (r1| Z = 1, x, BP) – E (r0 | Z = 0, x, BP)

• Patorno et alE (r1| Z = particular AED, x, BP or epilepsy or pain) – E (r0 | Z = Topiramate, x, BP or epilepsy or pain )

• Patorno et al estimate reflects who takes each AED, rather than biologic effect of each AED


Correctly specified PS?

• Generally more difficult to correctly specify PS for full cohort when many subgroups have different processes related to confounding by indication

• Those with epilepsy have different reason for choosing particular AED compared to those with BP and also have different underlying suicide rates

• Better to analyze each subgroup separately?


Covariance adjustment on PS

• Known to perform poorly when PS is poorly estimated (Rosenbaum and Rubin, 1983; Marcus and Gibbons 2011)

• Can happen when the variance in the PS for the treatment group is smaller than for control (those who receive new treatment more homogeneous)

• Univariate covariance adjustment can greatly increase bias (Rubin, 1973)


Conclusions

• Potential outcome framework can help to clarify whether what is being estimated makes sense

• AED vs no AED for single disorder better than AED 1 vs AED 2 for many disorders

• Goal is to ‘add efficiency to studies with many subgroups’ which could greatly facilitate automatic large-scale drug safety screening

• Is this worth the cost of increased bias: ‘stopping or refusing to start AEDs in epilepsy may result in serious harm, including death’ Fountoulakis et al 2012

jsm 2012, san diego1 caution should be used in applying propensity scores estimated in a full cohort...

Documents