Lecture 20: Study Design and Sample Size Estimation in TTE Studies

Study Design and Sample Size

• Ideally we are involved in a study from the beginning

• As statisticians (an epidemiologist) part of our role is to ensure the study is designed to address the primary hypothesis under consideration– Ensure proper study design– Ensure appropriate sample size• Power and significance level

Study Design

• We’ve already talked (informally) about study design…

• Our job is to ask appropriate questions– What is the study population?– Primary hypothesis?– Is the event recurrent?– Competing risks?– …

Components of Sample Size

General Considerations in any hypothesis test1. Hypothesis to be tested2. Test statistic3. Size of the test (i.e. a)4. Desired power5. Assumed effect size

Specific for Time To Event Data

Additional considerations 1. Probability of an event during the study2. Expected rate of loss (i.e. censoring)3. Enrollment rate4. Competing risks

Basic Considerations

• Ensure precise specification of the hypothesis• Select a significance level and power

appropriate for the study• What is the test statistic that will be used to

test the hypothesis?– Many statistics have well known properties upon

which we base our calculations– Deviations from assumptions complicate these

calculations…

Null Hypothesis to be Tested

• Recall for log-rank test

• Where HR is assumed to be proportional for all t

0

0

0

:

:

: HR 1

A B

A B

A B

H S t S t t

H h t h t t

H h t h t t

Test-Statistic

• Can use either– Log-rank test (or some variation there of)– HR estimated from Cox PHM

• To test the null hypothesis of no difference consider the log of the hazard ratio– log(HR) ~ N when comparing two groups

Significance Level

• The probability that a statistical test will reject H0 when H0 is actually true– Significance = a

• Interpretation: For a given value under the null hypothesis, we’re going to reject the null in favor of the alternative in error (a)100% of the time.

Power

• The probability that a statistical test will reject H0 when H0 is false– Power=1-b

• Interpretation: For a given value under the alternative hypothesis, we’re going to correctly reject the null in favor of the alternative (1-b)100% of the time.

Choice of a and 1-b

• Generally choose a = 0.05• Other values can be used but should be

justified– Say choose a = 0.01, we want very string evidence

of a treatment effect– Alternatively a = 0.10 or 0.20 might be chosen for

something like a pilot study• Power generally set to 80-99%

Effect Size

• Generally assume proportional hazards• Hazard ratio:– Null: HR = 1– HR < 1 implies longer survival in treatment B– HR > 1 implies longer survival in treatment A

• Base sample size calculation on having sufficient power to detect minimum clinically important effect– For example, maybe a 30% reduction in incidence for Trt A

vs. Trt B (i.e. HR = 0.7) is clinically meaningful

Minimum Scientifically Important Difference

• Definition: the smallest difference which would mandate, in the absence of serious side effects and/or excessive cost, a change in scientific practice/ understanding.

• This is a scientific question, not a statistical question.

Significance, Power, and Sample Size

• Sample size impacted not only by significance level, power, and research question but also practical considerations– Number of available patients– Study duration– Cost

Basic Sample Size Formula• Generic formula for total # subjects/group

• Under restrictions

1 2 122A

Z ZN

1 2 0

1 2 1

and

A

P Z H

P Z H

Design Consideration in TTE Studies

• How will sample size be collected?– Enroll fixed number of patients and follow for

some specified period of time– Continue study until a sufficient number of events

have been observed

• Other considerations– Expected event rate in each group– How much loss to follow-up expected

Study Type

• Type I study– All subjects experience an event by the end of the study

• Type II study– A study terminates at fixed time T resulting in

administratively censored subjects.• Administrative censoring

– Right censoring that occurs of subject fails to experience the event prior to the end of the study

• Loss to follow-up– Occurs when a subject fails to complete the study for

reasons unrelated to the event of interest

Sample Size in Time to Event Data

• For many power calculations we specify significance level, power, variability, and our minimum clinically relevant difference to get our total N

• In TTE studies, it is easier to specify the number of events we need to observe rather than the total number of people

Simplest Case for TTE

• Trial comparing Treatment A to Treatment B• Simplest case assumes all subjects are

followed to the end of the study• Also assumes all subjects in within treatment

group have the same probability of experiencing the event

• Finally assumes hazard rates are proportional

Required Number of Events

• Za/2 and Zb are standard normal percentiles

• pA and pB are the proportion of subjects allocated to each treatment group– i.e. equal allocation both are ½

• D is the minimum clinically relevant difference we want to detect– In this case it is the log of the hazard ratio under the

alternative hypothesis

2 2

2 2

22events

lnA B A A B A

Z Z Z Z

p p p p

Example: Required # Events

• Suppose a = 0.05 and b = 0.10 (90% power)– za/2 = 1.96 and zb = 1.282

• Equal allocation– pApB = ¼

• Always round up.

Hazard Ratio # Events Required

0.8 845

0.7 331

0.5 88

How Many People?

• We’ve not yet discussed event rate(s)

• We’ve determined for 90% power at a significance level a = 0.05– To detect a 50% reduction in hazard, we need to

observe 88 events

• But… How many people do we actually need to enroll to get that many events?

A Simplification

• Consider only the case in which each patient is followed for some specified time period, T

• More general case– Patients recruited during accrual period, a– After recruitment, there is an additional follow-up

period, f– First patient followed a + f– Last patient followed for f– Requires slightly more complex power calculations

Calculating Number of People

• Need to consider probability of an event during the study period

• Once we have this, estimating the total number of people needed is easy

• NOTE: This still ignores loss to follow-up

# events

eventN

P

Overall Probability of Event

• Recall pA and pB are the proportion of subjects allocated to each treatment group

• SA(T) and SB(T) are the survival distributions for the two treatment groups

• How do we get values for SA(T) and SB(T)?

event 1 A A B BP p S T p S T

Estimating Survival Distributions

• Crude rate– How many events are expected in each group over the

course of the study?

• Alternatively, we could assume S(T) ~Exp(l)• Recall S(t) = exp{-lt}• Use the assumed distribution to calculate failure

probability– e.g. if l = 0.1/unit time, then S(1) = 0.905– Thus, assume by t = 1, 9.5% of subjects will have

experienced the event

Example 1

• One-year study (T = 1) with pA = pB, l = 0.1/yr for control group, and qA = 0.5

Example 2

• 5 year study (T = 5) with pA = pB, l = 0.1/yr for control group, and DA = 0.5

More General Case• Consider if subjects enroll over time– First patient followed a + f– Last patient followed for f

• Proportion of patients that will survive is the average survival curve from time f to a + f

• P(event) can be estimated by

161 4 0.5

1 1

event

A

control cont cont cont

trt control

control control trt trt

d S f S f a S f a

d d

P p d p d

Example 2 Revisited• 5 year study with pA = pB, l = 0.1/yr for

control group, and D = 0.5• But accrue patients for 2 years and follow for

the remaining 3 years

Beyond Basic Considerations

• Many factors may cause statistic to deviate from expected behavior– Loss to follow-up (non-administrative censoring)– Failure to comply with treatment– Non-Uniform patient entry– Non-constant hazard ratio

• Failure time differ greatly from exponential

– Competing risks…

• Violations generally require an increase in sample size to achieve desired power

Non-Uniform Entry

• Previously assumed uniform entry of subjects into the trial, which could be an error– For example patients enter trial in staggered

fashion– Sample size related to total number of person

years observed• Alternative “general” sample size equation

proposed by Lachin and Foulkes

Lachin and Foulkes Formula

• lcont and ltrt are hazard rates for each group• is overall hazard rate• f(l) is a component of the variance of s 2(l)

where

2

1 1 1 12

1trt cont trt trt cont cont

trt cont

N Z p p Z p p

trt trt cont contp p

12E

Non-Uniform Entry using L-F Formulation

• Assume patient entry times follow g(t)– If it is Uniform (for recruitment period a)

– If it is not Uniform• If recruitment is faster than expected, power will be

greater• If recruitment is slower than expected, power will be

reduced

1

211and

f ae eg t

a a

Example Alternate Distribution

• Truncated exponential entry distribution

1

2

0 , 01

11

1

t

a

a f a

a

eg t t a

e

e e

e

Example of Impact on Power

g Probability of Event in Trt

E(d | ltrt)

Probability of Event in Trt

E(d | ltrt)

Power (assuming Uniform entry)

N to maintain 90% power

0 0.496 0.638 0.900 378

-1 0.426 0.563 0.859 430

-2 0.387 0.520 0.832 468

-3 0.371 0.500 0.820 490

-4 0.359 0.488 0.813 502

-5 0.352 0.480 0.805 510

-6 0.348 0.477 0.801 516

Trial Conditions: ltrt = 0.2 & lcont = 0.3a = 3 & f = 2a = 0.05 (one-sided) & 1-b = 0.9

Loss To Follow-Up• Again, our early expressions assumed only

administrative censoring BUT clearly this is not always the case.

• The Lakin-Foulkes expression can also be adapted to address random right censoring

Right Censoring Formulation

• Easiest “case” – censoring times ~Exp(h) – Uniform entry into trial

1

2

1 1 12

1

1

, 1

1, , , ,

Entry Time Distrbution:

Distribution for Censoring: c

f a f

trt trt cont cont trt trt trt conttrt cont

g ta

F c e

e e

a

N Z p p Z p

2

1cont contp

Impact of LTFU on Sample SizeTrial Conditions:

ltrt = 0.2, lcont = 0.3, & htrt = hcont = ha = 3 & f = 2a = 0.05 (one-sided) & 1-b = 0.9entry times ~U(a)

h E(d | ltrt) E(d | lcont) E(x | ltrt , h) E(x | lcont , h) Power Sample Size

0 0.496 0.638 0 0 0.901 378

0.05 0.459 0.594 0.115 0.099 0.881 406

0.10 0.425 0.554 0.213 0.185 0.860 436

0.15 0.396 0.518 0.297 0.259 0.839 468

0.20 0.369 0.486 0.369 0.324 0.817 500

Competing Risk Setting

• Same idea- we want to compare 2 treatments but now we have some competing risk(s)

• Latouche et. al (2004) developed an approach for estimating sample size in data with competing risks– Extension of Schoenfeld formula– Also based on Fine and Gray model for competing

risks

Recall the Fine and Gray Model

• Model and Partial likelihood

1 0

1

1

1

1

exp

exp

exp

where : 1

log 1 0then

log 1 1

i

i

I e

n i

ijj R

i i j j i i

t X t X

XL

X

R j T T T T e

F t XHR

F t X

Competing Risk Sample Size

• Number of events

• Number of subjects

2

1 2

2logcont trt

Z Zevents

p p

2

1 2

2logcont trt

Z Zn

p p

Additional Considerations

• There are extensions for >2 groups

• If interim analyses are planned, these also need to be accounted for in the sample size analysis

• Multi-center trial?

• Non-proportional hazards– Methods by Halpern and Lakatos allows for non-

proportional hazards– Specify event rates for groups within specific time

intervals

Additional Considerations

• Non-compliance– Differs from LTFU in that these people don’t comply to

treatment but are still followed– There are adjustments that can be made to the Lakin-

Foulkes formula if non-compliance expected

• Covariate adjustments?– If balanced, adjustment for covariate shouldn’t impact

power– However, in cases of extreme in-balance, formulas are

not valid

If Complications Exist

• Consider a 3-stage approach– Use basic formula as first estimate– Refine sample size calculation based on likely

deviations from assumptions– If necessary, develop simulation to address more

complicated deviations

Implementation

• The Lakin and Foulkes implemented in the gsDesign package in R– Includes ability to power for stratified analysis– Can power for interim analyses

• Can use proc power in SAS– TwoSampleSurvival statement• Based on log-rank but allows for different weights

(Gehan, log-rank, or Tarone-Ware)

Implementation

• Alternative software?– PASS• Can power for binary or non-binary covariate• Can power for scenario where an additional correlated

covariate is considered

– Nquery• Basic calculation based on Schoenfeld equation

All R Packages Related to Survival Analysis

https://cran.r-project.org/web/views/Survival.html

Statistician Vs. Lab Researcher

https://www.youtube.com/watch?v=PbODigCZqL8

Next Time

1. Nate O’ConnellSequential Designs for Phase I Clinical Trials with Late-Onset Toxicities

2. Lutffiya MuhammadCovariate-adjusted non-parametric survival curve estimation

3. James SmallSimulating biologically plausible complex survival data

4. Cameron MillerTutorial in biostatistics: Competing risks and multi-state models

5. Jamie SpeiserBagging survival trees