modelling differential clustering and treatment effect heterogeneity in parallel and stepped wedge...

Modeling differential clustering and treatment effectheterogeneity in parallel and stepped wedge cluster trials

Karla Hemming

University of Birmingham

Monica Taljaard (Ottawa) and Andrew Forbes (Monash)

March 4, 2016

Karla Hemming (University of Birmingham) Treatment effect heterogeneity in cluster trials March 4, 2016 1 / 26

Treatment effect heterogeneity - in iRCTs

Sub-group analysis common - desire to identify in whom, for whomand where the intervention works.

Whilst it is recommended that this be pre-specified and by the use ofinteraction tests, this guidance is poorly adhered to. 1

Treatment effect heterogeneity can be incorporated using either fixedor random interaction terms.

1Dealing with heterogeneity of treatment effects: is the literature up to thechallenge? Nicole B Gabler, Naihua Duan, Diana Liao, Joann G Elmore, Theodore GGaniats, Richard L Kravitz Trials. 2009; 10: 43.


Treatment effect heterogeneity - random or fixed effects?

Fixed effect interaction terms straightforward - preferable whennumber of subgroups small.

Random effects approaches preferable when number of groups large(i.e. site).

Different ways of parameterizing these models. 1 2

These different ways have gone largely unnoticed.

1Applied Mixed Models in Medicine by Helen Brown and Robin Prescott2Modeling site effects in the design and analysis of multisite trials Daniel J. Feaster,

Susan Mikulich-Gilbertson, Ahnalee M. Brincks Am J Drug Alcohol Abuse.Karla Hemming (University of Birmingham) Treatment effect heterogeneity in cluster trials March 4, 2016 3 / 26

Cluster randomized trials (CRTs) - basic set up

Cluster randomized trials are frequently used in health serviceevaluation.

It is common practice to use an analysis model with a random effectto combine between cluster information about treatment effects.

This is typically done by using a linear or generalized linear mixedmodel with a random effect for cluster. 1

1Donner, Allan and Klar, Neil; Design and analysis of cluster randomization trials inhealth research; 2000


Differential variation between arms - in parallel CRTs

Can’t estimate treatment by cluster heterogeneity in CRTs

In parallel cluster trials - whilst it may be anticipated that the treatmenteffect might vary across clusters, because clusters are either fully exposedor unexposed to the intervention, the trial design does not allowestimation of this.

Different treatments might be expected to induce homogeneity - oreven heterogeneity. 1

This will manifest itself as different ICCs in intervention and controlarms.

Also important in trials with differential clustering between arms -perhaps where a group therapy is compared with an individualtherapy. 2

1Ukoumunne OC, Thompson SG. Analysis of cluster randomized trials with repeatedcross-sectional binary measurements. Stat Med. 2001

2Roberts, Chris and Roberts, Stephen A; Design and analysis of clinical trials withclustering effects due to treatment; 2005; Clinical Trials


Treatment effect heterogeneity - in cluster cross-over trials

Can estimate treatment by cluster heterogeneity in cross-over CRTs

In cross-over cluster trials (i.e. stepped wedge) - because clusters arecrossed with treatment - the design does allow estimation of cluster bytreatment effects.

In cross-over designs and stepped wedge designs, because each clusterreceives the intervention and control, possible to estimate cluster bytreatment interactions.

Again different ways of parameterizing these models. 1 2

These different ways AGAIN have gone largely unnoticed.

1Hughes JP, Granston TS, Heagerty PJ. Current issues in the design and analysis ofstepped wedge trials. Contemp Clin Trials. 2015

2Baio G, Copas A, Ambler G, Hargreaves J, Beard E, Omar RZ. Sample sizecalculation for a stepped wedge trial. Trials. 2015


When should such extra heterogeneity be allowed for?

At the design stage.

At the analysis stage.

Model choice is therefore important from the outset.


Objectives

Here we show that careful choice is needed over modelparameterization of heterogeneity.

Outline different parameterizations.

Demonstrate that some parameterizations induce unnecessaryassumptions.

Show early results from a simulation study investigating statisticaloptimality of the different methods.

Preliminary recommendations for appropriate modelparameterizations.


Assumptions

Two arm trial:

Continuous outcome.

Testing for superiority.

Cluster randomization:

Assume reasonable number clusters (more than 40).

Analysis using individual level data, using mixed models with a randomeffect for cluster.


CRT: Basic model (single random effect)

Let us consider a two arm parallel CRT. The conventional analysis modelfor this simple setup is 1:

yij = µ+ xijθ + αj + eij (1)

αj ∼ N[0, τ2]

eij ∼ N[0, σ2w ]

Under this model the correlation between two observations in the samecluster, the intra-cluster correlation (ICC), will be:

ρ =τ2

τ2 + σ2w

i : individual ; j : cluster

1Donner, Allan and Klar, Neil; Design and analysis of cluster randomization trials inhealth research; 2000


Model extension A: two independent random effects

Two separate random effects, one for treatment and one for control, areincorporated 1:

yij = µ+ xijθ + xijα(T )j + (1 − xij)α(C )j + eij (2)

α(T )j ∼ N[0, τ2T ]

α(C )j ∼ N[0, τ2C ]

eij ∼ N[0, σ2w ]

The ICC in the control and intervention clusters will be:

ρC =τ2C

τ2C + σ2

w

ρT =τ2T

τ2T + σ2

w

Assumptions

No restrictions made on relative magnitude of two ICCs1Ukoumunne OC, Thompson SG. Analysis of cluster randomized trials with repeated

cross-sectional binary measurements. Stat Med. 2001Karla Hemming (University of Birmingham) Treatment effect heterogeneity in cluster trials March 4, 2016 11 / 26

Model extension B: a (SIMPLE) random interaction

Parametrization that includes a random interaction between the treatmentcovariate and cluster 1:

yij = µ+ xijθ + α(M)j + xijα(I )j + eij (3)

α(M)j ∼ N[0, τ2M ]

α(I )j ∼ N[0, τ2I ]

eij ∼ N[0, σ2w ]

The ICC in the control and treatment clusters will be:

ρC =τ2M

τ2M + σ2

w

ρT =τ2M + τ2

I

τ2I + τ2

M + σ2w

Assumptions

ICC in treatment clusters greater than or equal to ICC in control clusters1Roberts, Chris and Roberts, Stephen A; Design and analysis of clinical trials with

clustering effects due to treatment; 2005; Clinical TrialsKarla Hemming (University of Birmingham) Treatment effect heterogeneity in cluster trials March 4, 2016 12 / 26

The stepped wedge cluster randomized trial (SW-CRT) -basic model

We extend the basic model (1) for parallel CRTs to the SW-CRT, byincorporating fixed effects for each step 1:

yijs = µ+ xijsθ + αj + ts + eijs (4)

αj ∼ N[0, τ2]

eijs ∼ N[0, σ2w ]

Note

This notation also applies without any loss of generality to cross-over trialsand before and after parallel studies, with repeated cross-sectionalmeasurements.

i : individual ; j : cluster ; s : step

1Hussey and Hughes; Design and analysis of stepped wedge cluster randomized trials;2007; Contemporary clinical trials


Model extension A: two independent random effects

Two separate random effects, one for treatment and one for control 1:

yijs = µ+ xijsθ + xijsα(T )j + (1 − xijs)α(C )j + ts + eijs (5)

α(T )j ∼ N[0, τ2T ]

α(C )j ∼ N[0, τ2C ]


The ICC in the control and intervention periods will be:

ρCC =τ2C

τ2C + σ2

w

ρTT =τ2T

τ2T + σ2

w

ρCT = 0

Assumptions

No restrictions made on relative magnitude of two ICCs. But, ρCTassumed zero.

1Ukoumunne OC, Thompson SG. Analysis of cluster randomized trials with repeatedcross-sectional binary measurements. Stat Med. 2001 - FOR REPEATEDCROSS-SECTION DESIGN


Model extension B: an independent random interaction

Model treatment heterogeneity using a SIMPLE interaction term 1 2:

yijs = µ+ xijsθ + α(M)j + xijsα(I )j + ts + eijs (6)


and that (α(M)α(I )

)∼ N

((00

),

(τ2M 00 τ2

I

))The ICC in the control and treatment clusters will be:

ρCC =τ2M

τ2M + σ2

w

ρTT =τ2M + τ2

I

τ2I + τ2

M + σ2w

Assumptions

ICC in treatment clusters greater than or equal to ICC in control clusters;ρCT is non-zero.

1Baio G, Copas A, Ambler G, Hargreaves J, Beard E, Omar RZ. Trials. 20152Hughes JP, Granston TS, Heagerty PJ. Contemp Clin Trials. 2015


Model extension C: a non-independent random interaction

Model treatment effect heterogeneity using an interaction term andallowing for a covariance term (like in RCT literature):

yijs = µ+ xijsθ + α(M)j + xijsα(I )j + ts + eijs (7)


and that (αM

αI

)∼ N

((00

),

(τ2M σMI

σMI τ2I

))The ICC in the control and treatment clusters will be:

ρCC =τ2M

τ2M + σ2

w

ρTT =τ2M + τ2

I + 2σMI

τ2I + τ2

M + σ2w + 2σMI

Assumptions

No restrictions made on relative magnitude of two ICCs; ρCT is non-zero.1Modeling site effects in the design and analysis of multisite trials Daniel J. Feaster,

Susan Mikulich-Gilbertson, Ahnalee M. Brincks Am J Drug Alcohol Abuse.Karla Hemming (University of Birmingham) Treatment effect heterogeneity in cluster trials March 4, 2016 16 / 26

Model fitting

Stata

We have fitted these models in Stata 14 using the xtmixed function, usingREML methods. ICCs were calculated using variance ratios.

Care needed

Some care is needed to make sure you are fitting the model you think youare fitting.


Example: A parallel cluster randomised trial - with morevariability in outcomes in intervention clusters

Parallel cluster trial conducted in 53 schools (clusters).

Behavioral intervention to prevent obesity.

Outcome is BMI measured at the end of the trial.

Total of 689 observations in the intervention arm and 778observations in the control arm.

Average cluster size of 24.


Example: usefully advocates careful choice needed overparameterizations


Simulation study for parallel CRT

Simulation study parameters

Continuous outcome; two arms; large trial (100 centers each of size 50);SES=0.1; Cluster randomization; Compare model A and model B only;Generate data from model A; 1000 simulations

Scenarios

Scenario 1: No treatment by site heterogeneity.Scenario 2: Variation between control clusters smaller than that ofintervention clusters (concordant with model B).Scenario 3: Variation between control clusters greater than that ofintervention clusters (discordant with model B).

Findings:

No model comes out as preferable for bias of treatment effectsNo model comes out as preferable for coverage of treatment effectsModel B gives biased estimate of ICCs under scenario 3


Simulation study for cross-over CRT

Simulation study - greater heterogeneity in control clusters.

Continuous outcome; two arms; large trial (100 centers each of size 50);SES=0.1; Cluster cross-over randomization; Compare model A, B and C;Generate data from model C; 1000 simulations

Model A:

No evidence of bias.Coverage slightly too high.

Model B:

No evidence of bias.Coverage too low.

Model C:

No evidence of biasCoverage slightly too low.


Simulation study - with greater heterogeneity betweencontrol clusters than intervention clusters (scenario 3)

ICC for control clusters 0.01; for intervention clusters 0.001

Model C coverage and Model A coverage > Model B coverage (0.953 vs0.936)






Conclusion and discussion

Need for appreciation of assumptions implicit when fitting randomeffects treatment heterogeneity models.

Choice of parameterization important - some parameterizations makefewer assumptions and this would seem the sensible model choice.

Early simulations suggest model paramaterisation might have moreimportant implications for cross-over trials but not be important forparallel trials.

Simulations suggest little evidence for bias for treatment effect; butpotential for under coverage (standard error too small) from somemodels.

Some models give poor estimate of ICC.


Thank you!


Model extension E: alternative

Treatment heterogeneity modeled by what we call a true interaction term1:

yijs = µ+ xijsθ + αj + xijsα(T )j + (1 − xijs)α(C )j + ts + eijs (8)


and that: αj

α(C )jα(T )j

∼ N

000

,

τ2α 0 00 τ2

αT 00 0 τ2

αc

Assumed that τ2

αT = τ2αC in 2.

1Turner RM, White IR, Croudace T; PIP Study Group. Analysis of clusterrandomized cross-over trial data: a comparison of methods. Stat Med. 2007 Jan30;26(2):274-89. PubMed PMID: 16538700.

2Applied Mixed Models in Medicine by Helen Brown and Robin PrescottKarla Hemming (University of Birmingham) Treatment effect heterogeneity in cluster trials March 4, 2016 25 / 26

Example: A SW cluster randomised trial - with morevariability in outcomes in intervention clusters

SWl cluster trial conducted in X clusters

Across X steps

Outcome is XX

Total of XXX observations

Average cluster size of XX per cluster per period


modelling differential clustering and treatment effect heterogeneity in parallel and stepped wedge...

Data & Analytics