Background to Adaptive Design Background to Adaptive Design

Nigel StallardNigel StallardProfessor of Medical StatisticsProfessor of Medical Statistics

Director of Health Sciences Research InstituteDirector of Health Sciences Research InstituteWarwick Medical SchoolWarwick Medical School

n.stallard@warwick.ac.ukn.stallard@warwick.ac.uk

1. What are adaptive designs?

Types of adaptive designs

2. Advantages and challenges

Advantages

Statistical challenges

Logistical challenges

3. Example – adaptive seamless design in MS

Adaptive seamless phase II/III clinical trial

Evaluation of design options

4. Implications for research funders

OutlineOutline

1. What are adaptive designs?1. What are adaptive designs?

Conventional fixed sample size design

Start Observe data

Clinical trial reality: gradual accumulation of data

Start Observe data

Adaptive design:

Use interim analyses to assess accumulating data

Adapt design for remainder of trial

Types of adaptive designs

Possible adaptations can include:

- “Up-and-down” type dose-finding

- Adaptive randomisation (rand. play-the-winner etc.)

- Sample size re-estimation based on nuisance parameter estimates

- Sample size re-estimation based on efficacy estimates (including ‘self-designing trials’)

- Early stopping for futility

- Early stopping for positive results

- Selection or modification of subgroups or treatments

- Stopping for safety or logistical reasons

Focus on methods for confirmatory trials:

- Sample size re-estimation based on nuisance parameter estimates

- Sample size re-estimation based on efficacy estimates (including ‘self-designing trials’)

- Early stopping for futility

- Early stopping for positive results

- Selection or modification of subgroups or treatments

2. Advantages and challenges2. Advantages and challengesAdvantages

Efficiency:

- reach conclusion with (on average) smaller sample size

- avoid wasting further resources on trials unlikely to yield useful results

- ensure trials are appropriately powered

- focus resources on evaluation of most promising treatments

Ethics:

- use right number of right patients on right treatments

Statistical challenges

Type I error rate

E.g. Interim analysis in phase III trial to compare two arms

Significant at 5% level – stop trial

Not significant – continue with trial

Probability of false positive at interim analysis = 5%

Overall probability of false positive > 5%

Other adaptations may also increase type I error rate

e.g. sample size increased after less promising interim data

Treatment effect estimation

Trial may stop because of extreme positive data

Conventional estimates will overestimate true treatment effect

Specialist statistical methodology is required

Logistical challenges

Up-front planning

Designs may be more ‘custom-made’

Design properties may need to be assessed prior to trial

e.g. by simulation studies

Management of unblinded dataBreaking of blind may lead to bias, limit recruitment or lead to lack of equipoiseRelease of information and decision-making process needs to be carefully considered

Conduct of interim analyses

Timely and accurate data management required

Trial modification

May require ethical approval

May require revision of patient information sheets

Randomisation and drug supply needs careful consideration

3. Example – Adaptive seamless design in MS3. Example – Adaptive seamless design in MS

SettingPrimary/secondary progressive Multiple Sclerosis

ChallengesNo current effective disease modifying therapySeveral potential novel drug therapies to evaluate

Outcomes‘Phase II’ Short-term MRI data (~6-12 months)‘Phase III’ Long-term disability scales (~2-3 years)

Clinical trials are very long and costly

Adaptive seamless phase II/III clinical trial Experimental treatments T1, ..., Tk

Control treatment T0

Select treatment(s) at interim analysis using MRI dataFinal analysis uses combination test to control overall type I error rate allowing for selection/multiple testing

Stage 1

T0

T1

T2

Tk

Stage 2

T0

T[1]

Select treatment(s)

2kT

Evaluation of design options

Choice of design optionssample size, timing of interim analysis, decision rule for selecting arms

Simulation studyestimate power to reject at least one false null hypothesisestimate selection probabilitiesbased on wide range of assumptions

treatment effect on primary outcometreatment effect on short-term outcomecorrelation between outcomesfrom extensive literature review

10,000 simulations for each of > 25,000 scenarios

Example simulation results3 experimental treatments

Interim analysis midway early

one effective treatment

one effective treatmentone partly effective

4. Implications for research funders4. Implications for research funders

Advantages

Adaptive designs could lead to efficiency gains

Resources are targeted most effectively

Challenges

Need to ensure appropriate methodology is used

Additional methodological development may be needed

May need to allow extra time/funding for design work and evaluation

More flexible trials may require more flexible funding model