the poppi 1 example: statistical comments
DESCRIPTION
The POPPI 1 Example: Statistical Comments. Thomas A. Louis, PhD Department of Biostatistics Johns Hopkins Bloomberg SPH [email protected]. Little nits regarding the testing setup. ...transformed as log(mean C-peptide+ 1 ) . Why “ 1 ?” - PowerPoint PPT PresentationTRANSCRIPT
T. A. Louis TrialNet WorkshopMarch 7, 2005
1
The POPPI 1 Example:The POPPI 1 Example:Statistical CommentsStatistical Comments
Thomas A. Louis, PhDDepartment of Biostatistics
Johns Hopkins Bloomberg [email protected]
T. A. Louis TrialNet WorkshopMarch 7, 2005
2
Little nits regarding the testing setupLittle nits regarding the testing setup...transformed as log(mean C-peptide+1).
• Why “1?”• A formal, Bayesian approach would automatically
(and implicitly) move away from log(0)
...Shapiro-Wilk test for normality of the residuals and the White test for homoscedasticity• How to choose level• These tests have low power• The alternative is always true• Properties of testitests can be poor
Either go with the Wilcoxon or use a Either go with the Wilcoxon or use a Bootstrap-based test (direct test or BCBootstrap-based test (direct test or BCaa CI-based) CI-based)
T. A. Louis TrialNet WorkshopMarch 7, 2005
3
Bigger issues with the designBigger issues with the design
T. A. Louis TrialNet WorkshopMarch 7, 2005
4
Have you missed an important contrast?Have you missed an important contrast?
• Size and power were determined for three pair-wise group comparisons:– Mycophenolate mofetil (MMF) vs control– MMF/DZB (Daclizumab) vs control – MMF versus MMF/DZB
• If the contrast:
(MMF + MMF/DZB) –2*Control
is statistically and clinically significant, will you regret not having included it in your framework for controlling type I error?
T. A. Louis TrialNet WorkshopMarch 7, 2005
5
Fundamental IssueFundamental Issue
• Analysis plan should separate “evidence” (the joint likelihood for the three randomization groups) from decisions based on the evidence
• Multiplicity should be addressed by an explicit loss function or a collection of loss functions
T. A. Louis TrialNet WorkshopMarch 7, 2005
6
Sample Size and PowerSample Size and Power
• Sample size (and follow-up time) need to address uncertainty in the relation between the surrogate
(C-peptide) and clinical endpoints
• “60 subjects per group”– “subjects” “patients” (or participants) – Hard to imagine that with the current state of
knowledge, 60/60/60 is sufficient– Is equal allocation efficient (statistically and for recruiting?)
T. A. Louis TrialNet WorkshopMarch 7, 2005
7
Sample Size and PowerSample Size and PowerAccounting for uncertaintiesAccounting for uncertainties
• A series of “what ifs” (e.g., what if = 1, 2, ...) generally underestimates the necessary sample size
• Need to account for uncertainty in , base rates, dropouts, biologically plausible clinical differences and other uncertainties
T. A. Louis TrialNet WorkshopMarch 7, 2005
8
T. A. Louis TrialNet WorkshopMarch 7, 2005
9
T. A. Louis TrialNet WorkshopMarch 7, 2005
10
T. A. Louis TrialNet WorkshopMarch 7, 2005
11
T. A. Louis TrialNet WorkshopMarch 7, 2005
12
T. A. Louis TrialNet WorkshopMarch 7, 2005
13
T. A. Louis TrialNet WorkshopMarch 7, 2005
14
(CV)2
T. A. Louis TrialNet WorkshopMarch 7, 2005
15
T. A. Louis TrialNet WorkshopMarch 7, 2005
16
0.0 0.8 1.3 1.9 2.50.0 0.8 1.3 1.9 2.5 CV CV
T. A. Louis TrialNet WorkshopMarch 7, 2005
17
0.0 0.8 1.3 1.9 2.50.0 0.8 1.3 1.9 2.5
CVCV
T. A. Louis TrialNet WorkshopMarch 7, 2005
18
Pre-workshop QuestionsPre-workshop Questions• How long should the treatment and follow-up period
in trials be? Sufficient to address the relevant treatment
comparisons. If a surrogate endpoint is not sufficiently validated, follow-up must includ clinical endpoints, in part to produce meaningful comparisons and in part to gain information on the relation between the surrogate and clinical endpoints.
• Can larger numbers of treatments be compared simultaneously with aggressive curtailment for futility?
Yes, but follow-up still must be sufficient to support dropping a treatment or declaring a winner.
T. A. Louis TrialNet WorkshopMarch 7, 2005
19
Pre-workshop QuestionsPre-workshop Questions
• Can short term trials be grafted onto long term trials – roll phase 2 patients into phase 3 studies? Yes!!
• Should trials be powered for large effect sizes or for more moderate effect sizes with monitoring guidelines that could terminate early for an unexpected large effect?
Definitely power for clinically important and biologically viable effect sizes. Monitoring will temper what may be large, maximal sample sizes, but if the size needs to be big, it needs to be big.
• Should trials be powered to rule out adverse effects? Yes, to rule out relatively high-incidence (S)AEs. Whether
to power to rule out small to moderate (S)AEs depends on the integrity of post-marketing surveillance and on external information.
T. A. Louis TrialNet WorkshopMarch 7, 2005
20
A culture changeA culture change
Successfully addressing and implementing the foregoing requires changes
in culture and bureaucracy, plus some statistical developments