basel epidemiology seminar, bes julia spoendlin,...

Introduction to Propensity Score Methods

Basel Epidemiology Seminar, BES

Julia Spoendlin, 17.08.18

Outline

• Goal of propensity score (PS) methods

• How to calculate a PS

• example from BPU (statins and hand osteoarthritis)

• Definition of the PS

• Implementation of PS methods

• Strengths and limitations of PS methods

Universität Basel 2PS Intro, Julia Spoendlin, 17.08.18

0

500

1000

1500

2000

2500

(Pubmed, 25.4.2017)

Studies using Propensity Score Methods in PubMed

1st Study

University of Basel 3PS Intro, Julia Spoendlin, 17.08.18

Confounding in Pharmacoepidemiology

PS Intro, Julia Spoendlin, 17.08.18 Universität Basel 4

• Randomized controlled trials (RCT) = gold standard

• Randomization eliminates all confounding

• Observational studies

• Confounding by indication

• Confounding by frailty / comorbidities

• Differences in health-care utilization

• 99% of our work: control confounding

Randomisation

Indication

Yes

No

Propensity scores – the lay of the land

Propensity score (PS) methods

• control confounding (by indication)

• mimic the design of an RCT

• are (only) used in cohort studies (new user design)

• control for likelihood of being exposed at treatment decision (RCT)

• are great but are not a magic bullet

• do NOT adjust for unmeasured confounding


Propensity score (PS) methods

Universität Basel 6

Exposure Outcome

Confounders?

Outcome regressionPS methods

PS eliminates arrow from confounder to exposure

PS Intro, Julia Spoendlin, 17.08.18

Association of interest

Example The risk of hand osteoarthritis (HOA) in new statin users

Statins

CholesterolInflammation

University of Basel

Study question: do statins (compared to no statin) reduce the risk of

HOA in the Clinical Practice Research Datalink (CPRD)?

Systemic inflammation in osteoarthritispathophysiology

PS Intro, Julia Spoendlin, 17.08.18 Burkard et al. Arthritis Care Res. 2018 Jun 8. [Epub ahead of print]

*Outcome of interest=HOA and not all OA due to increased risk of confounding by mechanical factors in weight-bearing joints

Background:

Confounders of statin use and HOA?

Which confounders do we need to worry about?

?

• Statin users are older and more male

• HOA is most frequent in postmenopausal women

• Statin users may be more/less frail than non-users….

• Differential health care utilization…..


Included confounders

44 confounders (identified based on clinical knowledge):

• Demographics (age, sex)

• Comorbidities and comedication

• cardiovascular and others

• Frailty indicators

• COPD, pneumonia, fractures, No of comedication classes, CKD, pressure ulcer,

incontinence, volume depletion etc.

• Health care utilisation indicator

• No. of GP visits, hospitalizations

Titel Vortrag, Autor, DD.MM.YY Universität Basel 9

Study design: Cohort study (1:1 propensity score matching)

Exposure: New statin prescriptionExcluded: • Age <45 and >84 years• A few more exclusion criteria

Comparator Non-users of statins (at the time of cohort entry)

Our study design


Follow up: Outcome diagnosed hand OAAssessment 44 confounders

Treatment decision

Calculate PS

Jan 1996 Dec 2015

Burkard et al. Arthritis Care Res. 2018 Jun 8. [Epub ahead of print]

How to calculate a PS


Logistic Regression• statin use = dependent variable• 44 confounders = independent variablesWe don’t look at outcome (HOA) at this stage


PS = probability of statin use = 0 (no chance of statin use) – 1 (certain statin use)

• Logistic regression

• Sas command:

proc logistic desc data=statin_hoa; model statin=var1 var2 var3…….var44;output out=conc_ps p=ps;run;

• From now on we work with PS

• instead of our 44 covariates

• Combines 44 confounders in summary

score:

• PS=P(E | 44 covariates)

How to calculate a PS


Frequency distribution of PS

University of Basel 13

• Higher PS in statin users, lower PS in non-users

• Need sufficient overlap - measure of exchangeability

• Randomization: PS = 0.5 (50% chance to get exposed)

PS Intro, Julia Spoendlin, 17.08.18 Burkard et al. Arthritis Care Res. 2018 Jun 8. [Epub ahead of print]

Ideal situation That’s how it usually looks

RCT

Who came up with the PS


Rosenbaum & Rubin 1983

They showed:

• Treatment unrelated to confounders

• in patients with same PS

• Treated and untreated have same

distribution of measured confounders

• similar to randomizationBiometrika. 1983:70 (1); 41-55


Implementation of PS methods

• Covariate adjusting – PS as covariate in outcome regression

• Stratification – outcome regression within strata of PS

• Matching – match exposed and unexposed on PS (1:n)


PS Publications in Pubmed(2000-2009)

32% 24% 22% 18%

Stat. Med. 33, 74–87 (2014)

Matching StratificationAdjusting IPTW

Implementation

Covariate adjusting (modelling)

Include PS as continuous or categorical covariate into regression model

• Limitations:

• Model mis-specification (relationship between PS and outcome: linear, non-linear, cubic,

splines…?)

Stratification

Rank subjects by PS and stratify into quintiles / deciles of PS

• Effect estimated in each stratum weighted average

• Limitations:

• Residual confounding within strata (distribution of PS of treated and untreated differ)

• Some strata may have mainly exposed or unexposed if overlap not sufficient


Matching (most commonly done)

Match n unexposed to each exposed on the PS exclude those that cannot match

•Decisions: if/which caliper (closeness of matching), with/without replacement, greedy/optimal matching…..

Advantages

• Excludes non-comparable patients

• High internal validity

• Analysis mimics RCT: PS matching at moment of treatment decision

Limitations

•Reduction in sample size

•Transportabilty of results

•Non-intuitive interpretation of effect estimate: who is a person with PS 0.5639?


Balance diagnostics

Table 1: covariate balance

• after matching / within strata

• means & proportions

• standardized difference

• Do not rely on significance testing only

only proceed if sufficient balance

• Frequently reported: C-statistic

• Measures ability of PS to predict treatment

• C-statistic is not relevant

• PS aims to balance confounding not to

predict treatment

Titel Vortrag, Autor, DD.MM.YY Universität Basel 18Arthritis Care Res. 2018 Jun 8. [Epub ahead of print]


Intensive Care Medicine. 2010; 36(12):1993-2003

Frequently used:

• Standarized differences (difference in means in units of pooled SD)

Balance diagnostics

Balance diagnostics in a PS stratified model


Quintile1 Q 2 Q 3 Q 4 Quintile5

ExposureStatin(n=25)

No stat(n=3 mio)

Statin(n=20

0)No stat(n=200)

Statin(n=200)

No stat(n=200)

Statin(n=200)

No stat(n=200)

Statin(n=200’000)

No stat(n=25)

Obesity (%) 20% 20% 20% 20% 20% 20% 20% 20% 20% 20%

COPD (%) 18% 18% 18% 18% 18% 18% 18% 18% 18% 27%

No of GPvisits (No)

40 20 20 20 20 20 20 20 20 20

These are made up numbers

COPD is a proxy for patient frailty

Why use PS?

• PS controls for confounding (like multivariable outcome regression)

• Little difference between PS and multivariable results in confounding control (J. Clin. Epidemiol. 2006;59: 437–447)

Advantages of PS methods

Timing

• Separate design of study from outcome analysis

• Mimic RCT - make people as similar as possible at treatment start

• PS includes covariates prior to treatment start (new user design)

• No adjustment for causal intermediates


Treatment decision

Step 3: follow up & outcome analysisSTEP 1: covariate assessment

Step 2: Calculate PS


Why use PS

Control of many covariates in settings with few outcomes

• Rule of thumb: 1 covariate per 6-10 outcomes in outcome regression model

• Large databases: large number of weak confounders

• In PS no need to be parsimonious (usually many exposed)

• Can even do high-dimensional PS (Epidemiology. 2009; 20, 512-522)

Check exchangeability of comparison groups before outcome analysis

Check covariate balance before outcome regression


• Modify PS until balance achieved

• Covariate balance cannot be checked in multivariable outcome (or PS) regression

• No point doing a study if your comparison groups cannot be compared

Risk of HOA in new statin users matched on PS to non-users


Summary – how to proceed with PS methods

1. Define study design and base population

2. Identify relevant confounders

• during specified time period or at any time PRIOR to cohort entry (PDS. 2013; 22:542-550)

3. Calculate the PS (logisitc regression with exposure as dependent variable)

4. Check exchangeability between groups - feasibility of study

5. Match, stratify on or adjust for PS

6. Check balance of covariates (not if adjusted)

• Modify and rerun PS if imbalances remain until dataset is balanced

• Or apply additional restriction criteria (age, sex etc.) if too unbalanced

7. Run outcome model once measured confounding is under control


PS methods are no magic bullet….


• No control for unmeasured confounding!

• PS is only as good as you make it (include right covariates that are measured correctly)

• Model-misspecification can occur

• Possible bias when assessing time-varying treatment (limit follow-up time)

• No control for flaws in study design

• Selection bias, invalid exposure or outcome measures (sensitivity, specificity)

Sensitivity analyses to explore sensitivity of results to changes in design


Thank you for your attention

Questions?

Comments?

Complaints

References

• Hernan MA & Robbins J. Causal Inference Part II: Chapter 15.4-5. Accessed from https://www.hsph.harvard.edu/miguel-hernan/causal-inference-book/ (July 2018)

• Brookhart AB, Wyss R, Layton BL, Stürmer T. Circulation Cardiovasculat Quality and Outcomes. 2013:6;604-611.

• Rosenbaum PR and Rubin DB. Biometrika. 1983:70 (1); 41-55.

• Austin, P. C. & Stuart, E. A. Moving towards best practice when using inverse probability of treatment weighting (IPTW) using the propensityscore to estimate causal treatment effects in observational studies. Stat. Med. 34, 3661–3679 (2015)

• Austin, P. C. An Introduction to Propensity Score Methods for Reducing the Effects of Confounding in Observational Studies. Multivariate Behav. Res. 46, 399–424 (2011)

• Stürmer, T. et al. A review of the application of propensity score methods yielded increasing use, advantages in specific settings, but not substantially different estimates compared with conventional multivariable methods. J. Clin. Epidemiol. 59, 437–447 (2006)

• Williamson, E., Morley, R., Lucas, A. & Carpenter, J. Propensity scores: from naive enthusiasm to intuitive understanding. Stat. Methods Med. Res. 21, 273–93 (2012)

• D’Agostino, R. B. Tutorial in Biostatistics Propensity Score Methods for Bias Reduction in the Comparison of a Treatment To a Non-Randomized Control Group. Stat. Med 17, 2265–2281 (1998)

• Desai JD, Rothman JR, Bateman BT et al. A Propensity-Score based fine stratification approach for confounding adjusting when exposure isinfrequent. Epidemiology 28, 249-257 (2017)

• Schneeweiss S, Rassen, JA, Glynn RJ et al. High-Dimensional Propensity Score Adjustment in Studies of Treatment Effects using Health Care Claims Data. Epidemiology 20, 512-522 (2009)


https://www.hsph.harvard.edu/miguel-hernan/causal-inference-book/

What method to choose

University of Basel 28

Matching Adjusting Stratification IPTW

Balance of covariates very strong strong strong** very strong

Diagnostics of covariatebalance

strong not transparent strong strong

Mimics RCT yes no no yes

Effect on result if PS model misspecified

weak strong weak intermediate

Generalizability (weak*) strong strong strong

Stat. Med. 33, 1242–1258 (2014)


*Depending on how many excluded during matching

**Fine-stratification on PS in the exposed achieves better control of bias

basel epidemiology seminar, bes julia spoendlin,...

Documents