analysis choices in longitudinal volume-outcome studies...b french (university of pennsylvania)...

Analysis choices in longitudinal volume-outcome studies

Benjamin French, PhDDivision of BiostatisticsUniversity of [email protected]

Division of Biostatistics and BioinformaticsPenn State Hershey17 May 2012

Overview

Scientific and statistical background

Longitudinal data analysis methods

Volume specifications

Application to SEER-Medicare data

Summary and discussion

B French (University of Pennsylvania) Volume-outcome studies Penn State Hershey 2 / 40

Research Article

Received 6 July 2010, Accepted 24 August 2011 Published online 15 November 2011 in Wiley Online Library

(wileyonlinelibrary.com) DOI: 10.1002/sim.4410

A general framework for estimatingvolume-outcome associations fromlongitudinal dataBenjamin French,a*† Farhood Farjah,b David R. Flumb andPatrick J. Heagertyc

Recently, there has been much interest in using volume-outcome data to establish causal associations betweenmeasures of surgical experience or quality and patient outcomes following a surgical procedure, such as coro-nary artery bypass graft, total hip replacement, and radical prostatectomy. However, there does not appear tobe a standard approach to a volume-outcome analysis with respect to specifying a volume measure and selectingan estimation method. We establish the recurrent marked point process as a general framework from which toapproach a longitudinal volume-outcome analysis and examine the statistical issues associated with using lon-gitudinal data analysis methods to model aggregate volume-outcome data. We review assumptions to ensurethat linear or generalized linear mixed models and generalized estimating equations provide valid estimates ofthe volume-outcome association. In addition, we provide theoretical and empirical evidence that bias may beintroduced when an aggregate volume measure is used to address a scientific question regarding the effect ofcumulative experience. We conclude with the recommendation that analysts carefully specify a volume mea-sure that most accurately reflects their scientific question of interest and select an estimation method that isappropriate for their scientific context. Copyright © 2011 John Wiley & Sons, Ltd.

Keywords: Estimating equations; health services research; informative cluster size; mixed models; surgeonexperience

1. Introduction

Volume-outcome studies are typically used to evaluate whether patients treated by high-volume health-care providers (e.g., surgeons or hospitals) experience better post-treatment outcomes than those treatedby low-volume providers. Examples include evaluating the association between surgeon volume andpatient mortality following coronary artery bypass graft [1] and estimating the effect of hospital volumeon patient mortality following treatment with mechanical ventilation [2]. Volume-outcome studies areimportant among health services researchers because the results may have direct policy implications [3],such as regionalization of health care into large healthcare centers [4] or selective referral of patients tohigh-volume providers [5]. In our motivating example, interest lies in estimating the effect of surgeonvolume, as a measure of surgeon experience, on patient mortality following lung resection, in whichcancerous regions are removed.

1.1. Estimation methods for longitudinal outcomes

Even though volume-outcome analyses have become common in the applied literature, there does notappear to be definitive guidance on appropriate estimation methods in the methodological literature.

aDepartment of Biostatistics and Epidemiology, University of Pennsylvania, 625 Blockley Hall, 423 Guardian Drive,Philadelphia, PA 19104-6021, U.S.A.

bDepartment of Surgery, University of Washington, BB-400 Health Sciences Building, Campus Mail Stop 356410, Seattle,WA 98195-6410, U.S.A.

cDepartment of Biostatistics, University of Washington, F-600 Health Sciences Building, Campus Mail Stop 357232, Seattle,WA 98195-7232, U.S.A.

*Correspondence to: Benjamin French, Department of Biostatistics and Epidemiology, University of Pennsylvania,625 Blockley Hall, 423 Guardian Drive, Philadelphia, PA 19104-6021, U.S.A.

†E-mail: [email protected]

366

Copyright © 2011 John Wiley & Sons, Ltd. Statist. Med. 2012, 31 366–382

Research Article

Received 6 July 2010, Accepted 24 August 2011 Published online 15 November 2011 in Wiley Online Library

(wileyonlinelibrary.com) DOI: 10.1002/sim.4410

A general framework for estimatingvolume-outcome associations fromlongitudinal dataBenjamin French,a*† Farhood Farjah,b David R. Flumb andPatrick J. Heagertyc

Recently, there has been much interest in using volume-outcome data to establish causal associations betweenmeasures of surgical experience or quality and patient outcomes following a surgical procedure, such as coro-nary artery bypass graft, total hip replacement, and radical prostatectomy. However, there does not appear tobe a standard approach to a volume-outcome analysis with respect to specifying a volume measure and selectingan estimation method. We establish the recurrent marked point process as a general framework from which toapproach a longitudinal volume-outcome analysis and examine the statistical issues associated with using lon-gitudinal data analysis methods to model aggregate volume-outcome data. We review assumptions to ensurethat linear or generalized linear mixed models and generalized estimating equations provide valid estimates ofthe volume-outcome association. In addition, we provide theoretical and empirical evidence that bias may beintroduced when an aggregate volume measure is used to address a scientific question regarding the effect ofcumulative experience. We conclude with the recommendation that analysts carefully specify a volume mea-sure that most accurately reflects their scientific question of interest and select an estimation method that isappropriate for their scientific context. Copyright © 2011 John Wiley & Sons, Ltd.

Keywords: Estimating equations; health services research; informative cluster size; mixed models; surgeonexperience

1. Introduction

Volume-outcome studies are typically used to evaluate whether patients treated by high-volume health-care providers (e.g., surgeons or hospitals) experience better post-treatment outcomes than those treatedby low-volume providers. Examples include evaluating the association between surgeon volume andpatient mortality following coronary artery bypass graft [1] and estimating the effect of hospital volumeon patient mortality following treatment with mechanical ventilation [2]. Volume-outcome studies areimportant among health services researchers because the results may have direct policy implications [3],such as regionalization of health care into large healthcare centers [4] or selective referral of patients tohigh-volume providers [5]. In our motivating example, interest lies in estimating the effect of surgeonvolume, as a measure of surgeon experience, on patient mortality following lung resection, in whichcancerous regions are removed.

1.1. Estimation methods for longitudinal outcomes

Even though volume-outcome analyses have become common in the applied literature, there does notappear to be definitive guidance on appropriate estimation methods in the methodological literature.

aDepartment of Biostatistics and Epidemiology, University of Pennsylvania, 625 Blockley Hall, 423 Guardian Drive,Philadelphia, PA 19104-6021, U.S.A.

bDepartment of Surgery, University of Washington, BB-400 Health Sciences Building, Campus Mail Stop 356410, Seattle,WA 98195-6410, U.S.A.

cDepartment of Biostatistics, University of Washington, F-600 Health Sciences Building, Campus Mail Stop 357232, Seattle,WA 98195-7232, U.S.A.

*Correspondence to: Benjamin French, Department of Biostatistics and Epidemiology, University of Pennsylvania,625 Blockley Hall, 423 Guardian Drive, Philadelphia, PA 19104-6021, U.S.A.

†E-mail: [email protected]

366

Copyright © 2011 John Wiley & Sons, Ltd. Statist. Med. 2012, 31 366–382


Overview







Volume-outcome studies

• Establish associations between measures of surgical experienceor quality and patient outcomes following a surgical procedure

• Results may have direct policy implications (Livingston et al., 2007)I Regionalization of health care (Birkmeyer, 2000)

I Selective referral of patients (Dudley et al., 2000)

n engl j med

349;22

www.nejm.org november

27, 2003

The

new england journal

of

medicine

2117

special article

Surgeon Volume and Operative Mortality in the United States

John D. Birkmeyer, M.D., Therese A. Stukel, Ph.D., Andrea E. Siewers, M.P.H., Philip P. Goodney, M.D., David E. Wennberg, M.D., M.P.H.,

and F. Lee Lucas, Ph.D.

From the Department of Surgery, Dart-mouth–Hitchcock Medical Center, Leba-non, N.H. (J.D.B., P.P.G.); the VeteransAffairs Outcomes Group, Veterans AffairsMedical Center, White River Junction, Vt.(J.D.B., P.P.G.); the Institute for ClinicalEvaluative Sciences, Toronto (T.A.S.); andthe Center for Outcomes Research andEvaluation, Maine Medical Center, Portland(A.E.S., D.E.W., F.L.L.). Address reprint re-quests to Dr. Birkmeyer at the Section ofGeneral Surgery, Dartmouth–HitchcockMedical Center, Lebanon, NH 03756, or [email protected].

N Engl J Med 2003;349:2117-27.

Copyright © 2003 Massachusetts Medical Society.

background

Although the relation between hospital volume and surgical mortality is well estab-lished, for most procedures, the relative importance of the experience of the operatingsurgeon is uncertain.

methods

Using information from the national Medicare claims data base for 1998 through1999, we examined mortality among all 474,108 patients who underwent one of eightcardiovascular procedures or cancer resections. Using nested regression models, weexamined the relations between operative mortality and surgeon volume and hospitalvolume (each in terms of total procedures performed per year), with adjustment forcharacteristics of the patients and other characteristics of the providers.

results

Surgeon volume was inversely related to operative mortality for all eight procedures(P=0.003 for lung resection, P

Advertising

“An unbeatable cardiac-care campaign for East Florida hospitals”

• Based on the high volume of cardiac procedures performed

• Volume correlates to hospitals’ high number of successful outcomes

• Focused on the statistics at the heart of East Florida’s success

www.youtube.com/watch?v=qpq8MDqJrGw


http://www.youtube.com/watch?v=qpq8MDqJrGw

Volume-outcome data

• Observational; a controlled experiment would be impractical

• Typically collected using an administrative database, withI Unique surgeon identification code

I Date of surgical procedure

I Post-surgery patient outcome

I Patient case-mix

I Surgeon and/or hospital characteristics

• Allows several specifications for a volume exposure variableI Number of surgeries at each surgery time (non-aggregate)

I Number of surgeries during each calendar year (aggregate)


Statistical challenges

• Analysis of longitudinal volume-outcome data with potentiallyinformative cluster sizes require appropriate estimation methods

I Multi-level marginal methods (GEE; Liang and Zeger, 1986)

I Mixed model methods (GLMM; Breslow and Clayton, 1993)

I Specialized methods (Cluster-weighted GEE; Williamson et al., 2003)

• Volume specified using an aggregate measure on a coarse time scaleI Treats a time-dependent exposure as time-independent

I May not reflect the scientific question of interest

n engl j med

349;22

www.nejm.org november

27, 2003

The

new england journal

of

medicine

2117

special article

Surgeon Volume and Operative Mortality in the United States

John D. Birkmeyer, M.D., Therese A. Stukel, Ph.D., Andrea E. Siewers, M.P.H., Philip P. Goodney, M.D., David E. Wennberg, M.D., M.P.H.,

and F. Lee Lucas, Ph.D.

From the Department of Surgery, Dart-mouth–Hitchcock Medical Center, Leba-non, N.H. (J.D.B., P.P.G.); the VeteransAffairs Outcomes Group, Veterans AffairsMedical Center, White River Junction, Vt.(J.D.B., P.P.G.); the Institute for ClinicalEvaluative Sciences, Toronto (T.A.S.); andthe Center for Outcomes Research andEvaluation, Maine Medical Center, Portland(A.E.S., D.E.W., F.L.L.). Address reprint re-quests to Dr. Birkmeyer at the Section ofGeneral Surgery, Dartmouth–HitchcockMedical Center, Lebanon, NH 03756, or [email protected].

N Engl J Med 2003;349:2117-27.

Copyright © 2003 Massachusetts Medical Society.

background

Although the relation between hospital volume and surgical mortality is well estab-lished, for most procedures, the relative importance of the experience of the operatingsurgeon is uncertain.

methods

Using information from the national Medicare claims data base for 1998 through1999, we examined mortality among all 474,108 patients who underwent one of eightcardiovascular procedures or cancer resections. Using nested regression models, weexamined the relations between operative mortality and surgeon volume and hospitalvolume (each in terms of total procedures performed per year), with adjustment forcharacteristics of the patients and other characteristics of the providers.

results

Surgeon volume was inversely related to operative mortality for all eight procedures(P=0.003 for lung resection, P

Illustration

Alternative specifications for time-dependent volume at t = 10, 11, 12

Time t 1 2 3 4 5 6 7 8 9 10 11 12

Surgery × × − × × × − − − × × ×Year j 1 2 3

Volume Non-aggregate Total 6 7 8

Recent 2 2 3

Aggregate Total 8 8 8

Recent 3 3 3

× denotes a surgery; − denotes no surgery

? Which specification best reflects the question of interest?

? What factors influence the occurrence of a surgery?


Motivating example

Interested in the association between surgeon volume and patient mortalityfollowing surgical treatment for lung cancer (SEER-Medicare data)

• Patient outcomes comprise repeated measures on surgeon i

• Data consist of outcome (mark), exposure, and event time processesmeasured at discrete calendar times t

I Yi (t): 30-day patient mortality

I Xi (t): Patient or provider characteristic

I Ni (t): Number of surgeries performed

• A patient outcome exists if and only if a surgery occursI dNi (t) = 1 indicates a surgery for surgeon i at time t

I Adapted recurrent marked point process (French and Heagerty, 2009)


Framework

0 2 4 6 8 10

Calendar time

| | | | |

Yi(3)Yi(8)

Xi(3) Xi(8)

Ni(8)Ni(3)

Xi(t)

Ni(t)


Endogeneity

• Current outcome may affect the occurrence of subsequent event

Yi (t)→ Ni (t ′), t < t ′

Example: If a patient experiences a poor outcome, then their surgeonmay obtain fewer referrals due to their past surgical performance

• Current outcome may affect future exposure

Yi (t)→ Xi (t ′), t < t ′

Example: If a patient experiences a poor outcome, then their surgeonmay be required to operate on a less risky patients in the future


Target of inference

Interested in a regression model to quantify the association betweensurgeon volume and patient outcomes among those who undergo surgery

µi (t) = E[Yi (t) | dNi (t) = 1, Xi (t), Ni (t)]= g−1(xitβ)

• dNi (t) = 1 is required, otherwise Yi (t) does not exist• Include history of exposure and event time processes

I Xi (t) = {Xi (s) | s ≤ t}I Ni (t) = {Ni (s) | s ≤ t}

• Identifies a partly conditional model (Pepe and Couper, 1997)I Do not include unobserved future exposures or events

I Do not include potentially intermediate outcomes

• May be reduced to a cross-sectional model


Overview







Assumptions

To ensure consistency of a generalized estimating equation or likelihood-based mixed model estimator for β, it is sufficient to assume for t ′ > t

(1) Yi (t) ⊥ Ni (t ′) | Xi (t), Ni (t), dNi (t) = 1

(2) Yi (t) ⊥ Xi (t ′) | Xi (t), Ni (t ′), dNi (t) = 1

Otherwise an independence estimating equation should be usedfor consistent estimation of β

• (2) is similar to the full covariate conditional mean assumption(Pepe and Anderson, 1994)


Assumptions: Implications

Factor the joint distribution of the exposure and event time processesgiven the mark process

[Ni (t′), Xi (t

′) | Yi (t)] = [Ni (t ′) | Yi (t)]︸︷︷︸Focus of (1)

× [Xi (t ′) | Ni (t ′), Yi (t)]︸︷︷︸Focus of (2)

(1) Current outcome does not affect occurrence of a subsequent event

Yi (t) 6→ Ni (t ′), t < t ′

(2) Current outcome does not affect future exposure

Yi (t) 6→ Xi (t ′), t < t ′


Proof

A covariance-weighted estimating function for β is of the form

Uβ(β,α) =n∑

i=1

T∑t=1

T∑t′=1

xTit′witt′ [Yi (t)− µi (t)]dNi (t)dNi (t ′)

where witt′ is the (t, t′) element of a covariance weight matrix Wi = V

−1i

• Recall: Consistency of β̂ relies on the assumption that E[Uβ(·)] = 0• witt′ 6= 0 and xit′ may contain future exposures and events• May require conditioning on future exposures and events• If assumptions (1) and (2) are satisfied, then

µi (t) = E[Yi (t) | dNi (t) = 1, Xi (t), Ni (t)]= E[Yi (t) | dNi (t) = 1, Xi (T ), Ni (T )]

so that the estimating function is an unbiased estimator for 0


Proof

An independence estimating function for β is of the form

Uβ(β,α) =n∑

i=1

T∑t=1

xTitwitt [Yi (t)− µi (t)]dNi (t)

• Vi is a diagonal matrix, i.e. witt′ = 0 (t 6= t ′)

• No need to condition on future exposures or events

• Assumptions (1) and (2) are not required


Evaluating assumptions

Evaluating assumptions is generally possible using the observed data

(1) Model λi (t′) | Yi (t) using a recurrent events model

log λi (t′) = log λ0(t

′) + η1Yi (t) + η2Xi (t) + η3Ni (t)

and use a model-based test of the null hypothesis H: η1 = 0

(2) Model Xi (t′) | Yi (t) using a generalized linear model

g(E[Xi (t′)]) = θ0 + θ1Yi (t) + θ2Xi (t) + θ3Ni (t ′)

and use a model-based test of the null hypothesis H: θ1 = 0

Violation of assumption (2) motivates methods for causal inference

• Marginal structural models (Robins et al., 2000)

• Structural nested mean models (Brumback et al., 2003)


Simulation study

Evaluate the impact of endogeneity [assumptions (1) and (2)]

t = 1, . . . , 100

P[dNi (t) = 1] = expit[η0 + η1Yi (t − 1) + η2Xi (t)]

E[Xi (t)] = θ0Xi (t − 1) + θ1Yi (t − 1)

Yi (t) = β0 + β1Xi (t) + β2Ni (t)

+ γ0i + γ1iXi (t) + γ2iNi (t) + Wi (t) + �i (t)

β2 = 0.05


Simulation results

Assumption (2) satisfied: θ1 = 0

Assumption (1) Method Mean β̂2 Coverage

Satisfied GEE Independence 0.050 95%

η1 = log 1 Exchangeable 0.050 95%

GLMM Intercepts 0.050 52%

Intercepts/slopes 0.050 95%

Violated GEE Independence 0.050 95%





Simulation results

Assumption (2) violated: θ1 = 0.1

Assumption (1) Method Mean β̂2 Coverage

Satisfied GEE Independence 0.050 95%




Violated GEE Independence 0.050 95%





Overview







Focus of analysis

• Hospital volume (Kahn et al., 2006)I Typically used as a measure of hospital size

I Hospital size may be roughly constant over time

I An aggregate measure may be appropriate

• Surgeon volume (Flum et al., 2001)I Typically used as a measure of surgeon experience

I Surgeon experience is an evolving process

I An aggregate volume measure may not be appropriate

I Inconsistent focus on cumulative and contemporaneous volume


Cumulative volume

index

volu

me

0 1 2 3 4

016

3248

64

Calendar year

Sur

geon

vol

ume

Non−aggregateRunning averageTotal average


Contemporaneous volume

index

volu

me

0 1 2 3 4

016

3248

64

Calendar year

Sur

geon

vol

ume

Non−aggregateYearly totalYearly average


Simulation study

Evaluate the impact of specifying volume using an aggregate measure

t = 1, . . . , 100

P[dNi (t) = 1] = expit[η0 + η1Yi (t − 1) + η2Xi (t)]

E[Xi (t)] = θ0Xi (t − 1) + θ1Yi (t − 1)

Yi (t) = β0 + β1Xi (t) + β2Ni (t)

+ γ0i + γ1iXi (t) + γ2iNi (t) + Wi (t) + �i (t)

β2 = 0.05


Simulation results

Specification Method Mean β̂2 Coverage

Running average GEE Independence 0.050 96%

Exchangeable 0.050 96%



Total average GEE Independence 0.050 94%




Simulation results

Specification Method Mean β̂2 Coverage

Yearly total GEE Independence 0.025 63%




Yearly average GEE Independence 0.099 72%




Overview







Background

Interested in the association between surgeon volume and 30-day patientmortality following surgical treatment for lung cancer

• SEER-Medicare database, 1992–2002

• Resection to remove cancerous region of the lung

• n = 1334 with up to 398 surgeries per surgeon


Three surgeons

Alive

Dead

Alive

Dead

1992 1997 2002

Alive

Dead

Calendar time

Pat

ient

sta

tus


Non-aggregate cumulative volume

Results: OR corresponds to 10-patient increase in SEER-Medicare volume

Method OR 95% CI

GEE Independence 0.980 (0.959, 1.002)

Exchangeable 0.975 (0.952, 0.999)

GLMM Intercepts 0.974 (0.950, 0.998)

Intercepts/slopes 0.973 (0.949, 0.998)

Assumption (1): Patient mortality does not affect future surgery times

exp η̂1 = 1.014

p = 0.71


Aggregate cumulative volume

Specification Method OR 95% CI

Running average GEE Independence 0.980 (0.959, 1.001)

Exchangeable 0.974 (0.951, 0.998)

GLMM Intercepts 0.974 (0.950, 0.998)


Total average GEE Independence 0.980 (0.953, 1.008)

Exchangeable 0.970 (0.939, 1.002)

GLMM Intercepts 0.967 (0.934, 1.002)


Aggregate contemporaneous volume

Specification Method OR 95% CI

Yearly total GEE Independence 0.935 (0.835, 1.046)

Exchangeable 0.916 (0.816, 1.028)

GLMM Intercepts 0.917 (0.798, 1.053)


Yearly average GEE Independence 0.897 (0.769, 1.047)

Exchangeable 0.845 (0.708, 1.009)

GLMM Intercepts 0.834 (0.689, 1.010)


Overview







Summary

• Recurrent marked point process framework motivates specificassumptions that determine which estimation methods are appropriateto generate inference from longitudinal volume-outcome data

• Motivates consideration of a bias/efficiency trade-off becausean independence estimating equation may be inefficient relativeto covariance-weighting methods under non-independence structures(Mancl and Leroux, 1996)

• Careful thought is required to select a volume specification that isappropriate for the scientific question of interest

• Spurious results may be obtained when surgeon volume is specifiedusing an aggregate measure and the effect of non-aggregate surgeonvolume is of primary scientific interest


Re: Farjoodi et al., 2011

Effects of hospital and surgeon volume on post-operative complicationsafter lumbar spine surgery

• Volume quartile based on annual cross-sectional aggregation

• May result in a biased approach when attempting to answerthe causal association of surgical experience on patient outcomes

• Fails to account for changes in individual surgeon volume over time;may misclassify recent volume at the time of some index cases

• “In a rapidly changing technology field such as spine surgery,volume measures that reflect more recent experience of individualsurgeons have greater face validity”

(Martin and Lurie, 2012)


Other modeling challenges

• May also be of interest to simultaneously estimate the impactof cumulative and contemporaneous volume

I Main effects

I Interactions

but collinearly may result in wide confidence intervals

• Limitation: Lack of information regarding surgeon volumeprior to the start of follow-up (previous experience)


References

1. Birkmeyer JD. Journal of the American College of Surgeons 2000; 190:341–349.

2. Birkmeyer JD, et al. New England Journal of Medicine 2003; 349:2117–2127.

3. Breslow NE, Clayton DG. Journal of the American Statistical Association 1993; 88:9–25.

4. Brumback B, et al. Biometrics 2003; 59:274–285.

5. Dudley RA, et al. Journal of the American Medical Association 2000; 283:1159–1166.

6. Farjoodi P, et al. Spine 2011; 36:2069–2075.

7. Flum DR, et al. Archives of Surgery 2001; 136:1287–1292.

8. French B, Heagerty PJ. Biometrics 2009; 65:415–422.

9. French B, et al. Statistics in Medicine 2012; 31:366–382.

10. Kahn JM, et al. New England Journal of Medicine 2006; 355:41–50.

11. Liang K-Y, Zeger SL. Biometrika 1986; 73:13–22.

12. Livingston HE, et al. Archives of Surgery 2007; 142, 979–987.

13. Mancl LA, Leroux BG. Biometrics 1996; 52:500-511.

14. Martin BI, et al. Spine 2012; 37:527–528.

15. Pepe MS, Anderson GL. Communications in Statistics B 1994; 23:939–951.

16. Pepe MS, Couper D. Journal of the American Statistical Association 1997; 92:991–998.

17. Robins JM, et al. Epidemiology 2000; 11:550–560.

18. Williamson JM, et al. Biometrics 2003; 59:36–42.


Scientific and statistical backgroundLongitudinal data analysis methodsVolume specificationsApplication to SEER-Medicare dataSummary and discussion

analysis choices in longitudinal volume-outcome studies...b french (university of pennsylvania)...

Documents