Bayesian Adaptive Dose Finding Studies: Smaller, Stronger, FasterBayesian Adaptive Dose Finding

Studies: Smaller, Stronger, Faster

Scott M. Berryscott@berryconsultants.com

Scott M. Berryscott@berryconsultants.com

BERRY

STATISTICAL INNOVATION

CONSULTANTS

Dose Finding TrialDose Finding Trial

Generic example. All details hidden, but flavor is the same

“Delayed” Dichotomous Response Combine multiple efficacy + safety in

the dose finding decision Use utility approach for combining

various goals Multiple statistical goals Adaptive stopping rules

Generic example. All details hidden, but flavor is the same

“Delayed” Dichotomous Response Combine multiple efficacy + safety in

the dose finding decision Use utility approach for combining

various goals Multiple statistical goals Adaptive stopping rules

Statistical ModelStatistical Model

The statistical model captures the uncertainty in the process.

Capture data, quantities of interest, and forecast future data

Be “flexible,” (non-monotone?) but capture prior information on model behavior.

Invisible in the process

The statistical model captures the uncertainty in the process.

Capture data, quantities of interest, and forecast future data

Be “flexible,” (non-monotone?) but capture prior information on model behavior.

Invisible in the process

Empirical DataEmpirical Data

Observe Yij for subject i, outcome j

Yij = 1 if event, 0 otherwise

j = 1 is type #1 efficacy response ($$)j = 2 is type #2 efficacy response (FDA) j = 3 is minor safety event

Observe Yij for subject i, outcome j

Yij = 1 if event, 0 otherwise

j = 1 is type #1 efficacy response ($$)j = 2 is type #2 efficacy response (FDA) j = 3 is minor safety event

Efficacy EndpointsEfficacy EndpointsLet d be the dosePj(d) probability of event j=1,2;

Let d be the dosePj(d) probability of event j=1,2;

j(d) ~ N(, 2)

IG(2,2)N(–2,1) N(1,1)

G(1,1)

j d logPj d

1 Pj d

j d j j

d

d j

Safety EndpointSafety EndpointLet d be the dosePj(d) probability of safety j=3;

Let d be the dosePj(d) probability of safety j=3;

N(-2,1)N(1,1)

G(1,1)

j d logPj d

1 Pj d

j j

d

d j

Utility FunctionUtility Function

Multiple Factors:Monetary Profile (value on market)

FDA SuccessSafety Factors

Utility is critical: Defines ED?

Multiple Factors:Monetary Profile (value on market)

FDA SuccessSafety Factors

Utility is critical: Defines ED?

Utility FunctionUtility Function

MonetaryFDA Approval

P2(0) is prob Efficacy #2 success for d=0

U d U1 P1 d U2 P3 d U3 P2 0 ,P2 d

Monetary Utility (“Fake”)Monetary Utility (“Fake”)

U1 P1 P1 0.1

0.4

U2 P3 1 2 P3 1.5

U3: FDA SuccessU3: FDA Success

U3 P2 0 ,P2 d Pr d 0 250/arm trial

StatisticalSignificance

This is a posterior predictive calculation. The probabilityof trial success, averaged over the current posteriordistribution

Statistical + Utility OutputStatistical + Utility OutputE[U(d)]E[j(d)], V[j(d)]

E[Pj(d)], V[Pj(d)]

Pr[dj max U]

Pr[P2(d) > P0]Pr[ d >> 0 | 250/per arm) each

d

E[U(d)]E[j(d)], V[j(d)]

E[Pj(d)], V[Pj(d)]

Pr[dj max U]

Pr[P2(d) > P0]Pr[ d >> 0 | 250/per arm) each

d

AllocatorAllocator Goals of Phase II study? Find best dose? Learn about best dose? Learn about whole curve? Learn the minimum effective dose? Allocator and decisions need to reflect

this (if not through the utility function) Calculation can be an important issue!

Goals of Phase II study? Find best dose? Learn about best dose? Learn about whole curve? Learn the minimum effective dose? Allocator and decisions need to reflect

this (if not through the utility function) Calculation can be an important issue!

AllocatorAllocator Find best dose?

Learn about best dose?

Find best dose?

Learn about best dose?

Find the V* for each dose ==> allocation probs

V P1 d* P1 d

** V P2 d

* P0

V* w1V P1 d* P1 d

** w2V P2 d* P0

d* is the max utility dose, d** second best

Best Dose

2nd Best Dose

AllocatorAllocator

V*(d≠0) = V P1 d nd 1

w1 Pr d d* w2 Pr d d**

V P2 d nd 1

w2 Pr d d*

V*(d=0) = V P2 0 n0 1

w2

AllocatorAllocator

“Drop” any rd<0.05

Renormalize

“Drop” any rd<0.05

Renormalize

rd V * d V * d

d0

k

for all d

DecisionsDecisionsFind best dose? Learn about best dose?

Shut down allocator wj if stop!!!!

Stop trial when both happen

If Pr(P2(d*) >> P0) < 0.10 stop for futility

Find best dose? Learn about best dose?

Shut down allocator wj if stop!!!!

Stop trial when both happen

If Pr(P2(d*) >> P0) < 0.10 stop for futility

If found, stop:

If found, stop:

Pr(d = d*) > C1

Pr(P2(d*) >> P0)>C2

More Decisions?More Decisions?

Ultimate: EU(dosing) > EU(stopping)?Wait until significance?Goal of this study?Roll in to phase III: set up to do this,

though goal becomes w2 and w3

Utility and why? are critical and should be done--easy to ignore and say it is too hard.

Ultimate: EU(dosing) > EU(stopping)?Wait until significance?Goal of this study?Roll in to phase III: set up to do this,

though goal becomes w2 and w3

Utility and why? are critical and should be done--easy to ignore and say it is too hard.

SimulationsSimulations

Subject level simulationSimulate 2/day first 70 days, then

4/dayDelayed observation

exponential mean 10 daysAllocate + Decision every weekFirst 140 subjects 20/arm

Subject level simulationSimulate 2/day first 70 days, then

4/dayDelayed observation

exponential mean 10 daysAllocate + Decision every weekFirst 140 subjects 20/arm

Scenario #1Dose P1 P2 P3 P4 UTIL

0 0.05 0.06 0.05 0 0

0.25 0.05 0.10 0.06 0 0

0.5 0.08 0.13 0.07 0 0.063

1 0.12 0.17 0.08 0 0.323

2.5 0.15 0.20 0.09 0 0.457

5 0.18 0.23 0.10 0 0.532

10 0.25 0.30 0.11 0 0.656

Stopping Rules: C1 = 0.80, C2 = 0.90

MAX

181102

200210

182202

150535

190531

174423

183522

Nin

#1#2#3Nout

Dose ProbabilitiesDose Probabilities

0 .25 .5 1 2.5 5 10

P(>>Pbo) .18 .33 .27 .29 .67 .67

P(max) .01 .04 .06 .04 .33 .52

P(2nd) .03 .06 .10 .13 .35 .32

Alloc .06 .01 .02 .04 .06 .35 .46

201103

200210

182202

191541

190533

258727

245727

Nin

#1#2#3Nout

Dose ProbabilitiesDose Probabilities

0 .25 .5 1 2.5 5 10

P(>>Pbo) .12 .38 .36 .38 .92 .91

P(max) .00 .00 .02 .04 .41 .53

P(2nd) .00 .03 .06 .07 .47 .37

Alloc .00 .00 .02 .04 .09 .34 .51

211202

200210

192301

201540

210534

29972

11

316

113

17

Nin

#1#2#3Nout

Dose ProbabilitiesDose Probabilities

0 .25 .5 1 2.5 5 10

P(>>Pbo) .13 .39 .38 .26 .97 .85

P(max) .00 .02 .03 .01 .39 .55

P(2nd) .00 .03 .10 .05 .46 .35

Alloc .11 .00 .03 .10 .05 .46 .35

231204

200210

202400

211544

251540

361073

10

4510123

16

Nin

#1#2#3Nout

Dose ProbabilitiesDose Probabilities

0 .25 .5 1 2.5 5 10

P(>>Pbo) .16 .41 .38 .48 .93 .93

P(max) .00 .02 .03 .04 .26 .65

P(2nd) .00 .05 .07 .10 .49 .29

Alloc .00 .00 .08 .11 .18 .35 .28

261201

200210

202400

251546

262645

441373

12

5210134

15

Nin

#1#2#3Nout

Dose ProbabilitiesDose Probabilities

0 .25 .5 1 2.5 5 10

P(>>Pbo) .16 .40 .31 .41 .98 .89

P(max) .00 .02 .03 .06 .27 .63

P(2nd) .00 .06 .06 .12 .48 .28

Alloc .16 .00 .10 .04 .13 .26 .30

261206

200210

212403

261645

333745

521384

10

6115184

12

Nin

#1#2#3Nout

Dose ProbabilitiesDose Probabilities

0 .25 .5 1 2.5 5 10

P(>>Pbo) .13 .36 .32 .65 .96 .96

P(max) .00 .01 .01 .09 .08 .81

P(2nd) .00 .05 .05 .23 .52 .15

Alloc

Trial EndsTrial Ends

P(10-Dose max Util dose) = 0.907

P(10-Dose >> Pbo 250/arm) = 0.949

280 subjects: 32, 20, 24, 31, 38, 62, 73 per

arm

P(10-Dose max Util dose) = 0.907

P(10-Dose >> Pbo 250/arm) = 0.949

280 subjects: 32, 20, 24, 31, 38, 62, 73 per

arm

Operating CharacteristicsOperating Characteristics

Pbo 0.25 0.5 1 2.5 5 10

SS 39 21 25 37 63 89 110

Pmax --- 0.00 0.00 0.00 0.00 0.04 0.96

SS 66 66 66 66 66 66 66

Pmax --- 0.00 0.00 0.00 0.01 0.06 0.93

Operating CharacteristicsOperating Characteristics

Adaptive ConstantConstant/No Model

P(Sufficient) 0.936 0.810 0.700

P(Cap) 0.064 0.190 0.300

P(Futility) 0.000 0.000 0.000

P(10mg Best) 0.96 0.93 0.88

Mean SS 384 459 517

SD SS 186 224 235

Mean TDose 1754 1263 1420

Max TDose 4818 2370 2341

Scenario #2Scenario #2Dose P1 P2 P3 P4 UTIL

0 0.06 0.05 0.05 0 0

0.25 0.10 0.05 0.06 0 0

0.5 0.13 0.08 0.07 0 0.063

1 0.17 0.12 0.08 0 0.323

2.5 0.20 0.15 0.10 0 0.452

5 0.23 0.18 0.15 0 0.502

10 0.25 0.20 0.40 0 0.302

Stopping Rules: C1 = 0.80, C2 = 0.90

Operating CharacteristicsOperating Characteristics

Pbo 0.25 0.5 1 2.5 5 10

SS 71 27 41 81 137 172 164

Pmax --- 0.00 0.00 0.03 0.22 0.60 0.16

SS 100 100 100 100 100 100 100

Pmax --- 0.00 0.00 0.03 0.20 0.44 0.33

Operating CharacteristicsOperating Characteristics

Adaptive ConstantConstant/No Model

P(Sufficient) 0.314 0.266 0.286

P(Cap) 0.686 0.734 0.708

P(Futility) 0.000 0.000 0.006

P(5mg Best) 0.60 0.44 0.58

Mean SS 694 702 704

SD SS 193 190 182

Mean TDose 2954 1933 1937

Max TDose 4489 2455.25 2436

Simulation #3Simulation #3Dose P1 P2 P3 P4 UTIL

0 0.06 0.05 0.05 0 0

0.1 0.10 0.05 0.06 0 0

0.5 0.13 0.08 0.07 0 0.063

1 0.30 0.25 0.11 0 0.656

2.5 0.17 0.12 0.08 0 0.323

5 0.20 0.15 0.09 0 0.457

10 0.23 0.18 0.10 0 0.532

Stopping Rules: C1 = 0.80, C2 = 0.90

Operating CharacteristicsOperating Characteristics

Pbo 0.25 0.5 1 2.5 5 10

SS 53 23 28 119 52 76 102

Pmax --- 0.00 0.00 0.92 0.00 0.01 0.07

SS 87 87 87 87 87 87 87

Pmax --- 0.00 0.00 0.83 0.00 0.02 0.15

Operating CharacteristicsOperating Characteristics

Adaptive ConstantConstant/No Model

P(Sufficient) 0.906 0.596 0.708

P(Cap) 0.092 0.404 0.290

P(Futility) 0.002 0.000 0.002

P(1mg Best) 0.92 0.83 0.87

Mean SS 453 606 542

SD SS 187 205 225

Mean TDose 1663 1662 1491

Max TDose 3771 2384.25 2414.25

Scenario #4Scenario #4Dose P1 P2 P3 P4 UTIL

0 0.06 0.05 0.05 0 0

0.1 0.07 0.06 0.06 0 0

0.5 0.08 0.07 0.07 0 0

1 0.09 0.08 0.08 0 0

2.5 0.09 0.08 0.09 0 0

5 0.09 0.08 0.10 0 0

10 0.09 0.08 0.11 0 0

Stopping Rules: C1 = 0.80, C2 = 0.90

Operating CharacteristicsOperating Characteristics

Pbo 0.25 0.5 1 2.5 5 10

SS 92 91 75 66 76 83 90

Pmax --- 0.45 0.04 0.07 0.10 0.13 0.21

SS 84 84 84 84 84 84 84

Pmax --- 0.44 0.04 0.08 0.12 0.15 0.17

Operating CharacteristicsOperating Characteristics

Adaptive ConstantConstant/No Model

P(Sufficient) 0.004 0.006 0.030

P(Cap) 0.484 0.544 0.752

P(Futility) 0.512 0.450 0.218

Mean SS 574 589 699

SD SS 250 258 196

Mean TDose 1637 1615 1922

Max TDose 3223.5 2523.75 2467.75

Scenario #5Scenario #5Dose P1 P2 P3 P4 UTIL

0 0.06 0.05 0.05 0 0

0.1 0.06 0.05 0.05 0 0

0.5 0.06 0.05 0.05 0 0

1 0.06 0.05 0.05 0 0

2.5 0.06 0.05 0.05 0 0

5 0.06 0.05 0.05 0 0

Stopping Rules: C1 = 0.80, C2 = 0.90

Operating CharacteristicsOperating Characteristics

Pbo 0.25 0.5 1 2.5 5 10

SS 66 77 51 34 38 41 43

Pmax --- 0.90 0.01 0.02 0.02 0.02 0.03

SS 56 56 56 56 56 56 56

Pmax --- 0.86 0.01 0.02 0.03 0.03 0.05

Operating CharacteristicsOperating Characteristics

Adaptive ConstantConstant/No Model

P(Sufficient) 0.000 0.000 0.002

P(Cap) 0.122 0.190 0.362

P(Futility) 0.878 0.810 0.636

Mean SS 350 395 542

SD SS 215 241 241

Mean TDose 811 1086 1491

Max TDose 2404 2428.75 2455.5

Bells & WhistlesBells & Whistles Interest in QuantilesMinimum Effective Dose“Significance,” control type I errorSeamless phase II --> IIIPartial Interim Information“Biomarkers” of endpointContinuous, Poisson, Survival,

MixedContinuum of doses (IV)--little

additional n!!!

Interest in QuantilesMinimum Effective Dose“Significance,” control type I errorSeamless phase II --> IIIPartial Interim Information“Biomarkers” of endpointContinuous, Poisson, Survival,

MixedContinuum of doses (IV)--little

additional n!!!

ConclusionsConclusions

Approach, not answers or details!Shorter, smaller, stronger!Better for: Sponsor, Regulatory,

PATIENTS (in and out), ScienceWhy study?--adaptive can help

multiple needs.Adaptive Stopping Bid Step!

Approach, not answers or details!Shorter, smaller, stronger!Better for: Sponsor, Regulatory,

PATIENTS (in and out), ScienceWhy study?--adaptive can help

multiple needs.Adaptive Stopping Bid Step!