judith goldberg medicres world congress 2014
Post on 22-Nov-2014
826 Views
Preview:
DESCRIPTION
TRANSCRIPT
Statistics in Clinical and Translational Research in Drug Development
Judith D. Goldberg, Sc.D.ProfessorDivision of BiostatisticsNew York University School of Medicine
MedicReS International Congress on Good Medical ResearchNew York, New York October 16, 2014
JD Goldberg MedicReS 10162014
Personal Perspectives from:
Pharmaceutical industry drug and device developmentNon profit health care research AcademiaFDA Advisory Committee MemberExpert Witness, Other
HistoryCurrent viewsFuture directions and challengesBioinformaticsBig DataPersonalized medicine
JD Goldberg MedicReS 10162014
Statistics in Clinical and Translational Research: Process
Planning Problem formulation
What is the question (hypothesis)?
Study designType of study? Comparison?What is the intervention? Outcome? For whom?When? For how long?Sample size?Data collection: forms design, database design,
procedures, timelinesContingency plans? early stopping?
Analysis plan
JD Goldberg MedicReS 10162014
Statistics in Clinical and Translational Research: Process
ImplementationStudy conductStudy progress
accrual, data and safety monitoring
Data management
Study Completion Study closeout Data analysis Interpretation Reporting
JD Goldberg MedicReS 10162014
JD Goldberg MedicReS 10162014
Environment [early 1970’s]
New statistical methods: logistic regression log linear models Cox proportional hazards model
Batch computing, IBM cards, card readers, sorters, tape back ups, …Statistical computing: SPSS, BMDP limited software
JD Goldberg MedicReS 10162014
Current EnvironmentBasic issues of study design, replication need to be addressedSoftware availability (R, SAS, STATA, …)Emphasis on speed, efficiency, accelerated developmentLarge amounts of data need special toolsMultiplicity makes usual p-values uninterpretable – false discovery rateAssumptions in pre-processing of data at multiple steps influence resultsAssumptions in analytic methods influence results
Changing RolesBasic statistical issues remain the same Focus on problem identification Collaborative involvement throughout
research process Planning Implementation Reporting
Statistical thinking has expanded Tools and methods have changed
Advances in science Explosion in amounts of data Enabled by advances in computing
JD Goldberg MedicReS 10162014
JD Goldberg MedicReS 10162014
Biostatistics in Drug Development:Today and Tomorrow
Issues: Basic issues the same Thinking has expanded Problems more complex High dimensional data –many variables, small numbers of
observations
Environment: Interdisciplinary research: TEAM SCIENCE
Challenges for statistics Expanded role in problem formulation, complex research process,
input into all stages of development Interactions with ‘bioinformatics’, informatics Data sharing, regulations (e.g, privacy) Combining data from multiple sources; warehousing Making explicit requirements for IT infrastructure to enable and
enhance research process
JD Goldberg MedicReS 10162014
New [and Old] Opportunities
Strategic input at all stages of drug development Compound screening Patent preparation ***
New study designs to address efficiency without compromising science Phase I/II; Phase II/III; adaptive designs Incorporation of biomarkers
Safety evaluations from early development through post marketingCombining data from multiple sourcesComparative effectiveness
Drug Development Paradigm
JD Goldberg MedicReS 10162014
Pre Clinical Development
Drug Discovery Compound Screening
High throughput in silico Animal models
New Opportunity Statistical methods for screening to
minimize false negative and false positive leads
Use of experimental designs to optimize screening and animal testing
JD Goldberg MedicReS 10162014
Patents
JD Goldberg MedicReS 10162014
Historically, little statistical involvementFile rapidly
Example: • Survival curves in mice calculated incorrectly
Led to major patent litigation ignorance– but should not happen
• Lab notebooks at issue as well labelled ‘fraud’ ------
Example:• Patent claims that two drugs given together are synergistic
Phase I
Investigator controlled treatment administration and structured observationsGenerally not randomized; can be circumstances where randomization is used
Objectives:Safety and tolerance; single and multiple doseDose finding (MTD- maximum tolerated dose that is
associated with serious but reversible side effects in a ‘sizeable’ proportion of patients ; use RPTD – recommended Phase II dose- one level down
Bioavailabilty – rate and extent to which active ingredient or therapeutic compound absorbed and available at site of action
Equivalence of formulations, drugs (bioequivalence)Special populations, drug –drug interactions, fed/fast
Exploratory - tentative answers Issues—ethics
Healthy volunteers vs patientsJD Goldberg MedicReS 10162014
Phase II Designs
Objective: Preliminary evidence of efficacy and side effects at fixed dose(s)
Parallel group randomized designs Uncontrolled single group
Objectives: Proof of concept, efficacy, mechanism, dose
ranging, pilot studies
JD Goldberg MedicReS 10162014
Phase II Objectives - continued
Estimate clinical endpoint with specified precision
Proportion of patients who respond
Average change from baseline in diastolic blood pressure
Proportion of patients with side effects
Proportion of patients who fail (failure rate)
Dose response JD Goldberg MedicReS
10162014
Types of Phase II Designs
Single arm uncontrolled trial with specified number of patients to estimate the response rate with specified precision
Example: If 20% is the lowest acceptable response rate for a new treatment, if there are no observed responses in 12 patients, then the exact binomial upper 95% confidence interval is 20%.
Randomized phase IISeamless Phase II/III
Response to pressure for more efficient study designs
JD Goldberg MedicReS 10162014
Phase II Single Arm Two Stage Designs (Simon, 1989)
p0 uninteresting level of response
p1 interesting level of response
If true probability of response is less than p0, then the chance of accepting treatment for further study is α
If true probability of response is greater than p1 , chance of rejecting treatment is β (1-power)JD Goldberg MedicReS 10162014
Simon Two Stage Design- cont.
Study ends at end of stage 1 only if the treatment appears ineffective
Stop early only for lack of efficacy
Stage 1: If r1 or fewer responses are observed in the first n1 patients, stop; otherwise continueStage 2: If r (total stage 1+ stage 2) responses are observed in n (total patients), continue to study the drug
JD Goldberg MedicReS 10162014
Adaptive Designs
Accumulating data as basis for modifying trial without impacting validity, integrity
JD Goldberg MedicReS 10162014
Possible Adaptations:
Early stop (futility, early rejection)Sample size re-assessmentTreatment allocation ratio changeTreatment arms changes (drop, add, modify)Change hypothesesChange study population (inclusion, exclusion)Change test statisticsCombine trials (eg, seamless Phase II/III)
JD Goldberg MedicReS 10162014
Adaptive Designs: Sample Size Re-assessment
When, howBlinded, unblindedFDA Draft Guidance (2010)
‘revisions based on blinded interim evaluations of data (eg, aggregate event rates based on aggregate event rates or variance of the endpoint are advisable procedures that can be considered , variance, discontinuation rates, baseline characteristics) do not introduce statistical bias into the study or into subsequent study revisions made by the same personnel. Certain blinded analysis based changes, such as sample size revisions planned at the design stage, but can also be applied when not planned from the study outset if the study has remained unequivocally blinded’. [13, lines 91-96].
JD Goldberg MedicReS 10162014
Seamless Phase II/III Designs
Goal: Combine treatment selection and confirmation into one trial to speed development During trial, choose optimal dose,
population based on interim data\ Surrogate marker, early data on
endpoint, primary endpoint Enrollment continues on selected dose,
treatment arm(s), and population
JD Goldberg MedicReS 10162014
Intention To Treat (ITT) Principle
Analyze all subjects randomized Include all eventsBeware of “look alikes” Modified ITT: Analyze subjects who
get some intervention Per Protocol: Analyze subjects who
comply according to the protocol
JD Goldberg MedicReS 10162014
Dynamic Treatment Strategies and SMART Trials
DTS: set of decision rules for management of patients Can be represented by time-indexed
mapping from patient state history and previous treatments into set of possible treatment strategies
SMART: experiment for comparing DTSs Randomizes to different treatment
branches that separate DTSsJD Goldberg MedicReS 10162014
SMART continued
ITT randomizes at start Treatment changes after initial
randomization are not randomized and analysis is over distribution of implied DTSs
DTS ITT converts to SMART by randomizing when would change treatment decisions‘sequential ignorability’ (generalizes Rubin ignorability in this context)
JD Goldberg MedicReS 10162014
SMART Analysis
G estimation, marginal means, optimal semi parametric estimator (Moodie, 2007)Patient information contributed to one or more DTS until patient leaves that DTSAlternative to baseline randomization among DTSs
JD Goldberg MedicReS 10162014
Choices in Design of Randomized Controlled Trials
Treatment RegimenControlsTypes of patients and severity of diseaseLevel of blindingParallel group or alternative designNumber and size of centersStratificationInterim Analysis/monitoringAdaptiveBayesian
Length of observation period, need for retreatmentMethods of treatment deliveryUnit of analysisOutcomes and their measures; measurement errorMeaningful effect size [statistical significance vs. clinical significance]
JD Goldberg MedicReS 10162014
Defining the QuestionDefined carefully in advanceMust be clinically relevantPrioritize into primary, secondary, …Design built around primary question(s)Superiority, non inferiority, equivalence of treatments with respect to outcomeEligibility criteria define population studied and inferences to be madeSurrogates desirable but riskyNeed the relevant measure of efficacy
JD Goldberg MedicReS 10162014
Who Should Be Studied? Homogeneous vs. Heterogeneous
• Well defined Not easily specified
• Mechanism of action Not know if all groups well known respond similarly
• No dilution of results Easier to recruit • Infer results specifically Easier to
generalize
JD Goldberg MedicReS 10162014
Outcome measures
Occurrence of event e.g., in-hospital mortalityTime to evente.g., time to death, time to heart failure
Mean level of responsee.g., VO2, 6 min walkMean change from baseline in key variable Response (yes/no)
JD Goldberg MedicReS 10162014
Data analysis
Descriptive data analysisSpecify in advance Primary Secondary Other Statistical approach
Exploratory
JD Goldberg MedicReS 10162014
Data analysis
Intention-to-treatBy exposuresSubgroupsAdjusted vs. Unadjusted
JD Goldberg MedicReS 10162014
What Data Should Be Analyzed?Basic Intention-to-Treat Principle Analyze what is randomized! All subjects randomized, all events during
follow-upRandomized control trial is the “gold” standard”
DefinitionsExclusions Screened but not randomized Affects generalizability but validity OK
Withdrawals from Analysis Randomized, but not included in data
analysis Possible to introduce bias!
JD Goldberg MedicReS 10162014
Patient Closeout
ICH E9 Glossary “Intention-to-treat principle - …It has
the consequence that subjects allocated to a treatment group should be followed up, assessed, and analyzed as members of that group irrespective of their compliance with the planned course of treatment.”
JD Goldberg MedicReS 10162014
Patient Withdrawn in Analysis
Common Practice - 1980s Over 3 years, 37/109 trials in New England Journal of
Medicine published papers with some patient data not included
Typical Reasons-Patient ineligible (in retrospect) -Noncompliance
-Competing events -Missing data
JD Goldberg MedicReS 10162014
Patient Withdrawn in Analysis-continued
Patient INELIGIBLE after randomization
Concern ineligible patients may dilute treatment effect
Temptation to withdraw ineligibles
Withdrawal of ineligible patients, post hoc, may introduce bias
JD Goldberg MedicReS 10162014
Sources of Bias in Clinical Trials
• Patient selection• Treatment assignment• Evaluation of patient outcomes• Dropouts, crossovers • Loss to follow up• Missing covariate data• Missing outcome data
Methods to Minimize Bias• Randomized Controls• Double blind (masked)• Analyze as randomized (intent to treat)JD Goldberg MedicReS 10162014
Betablocker Heart Attack Trial(JAMA, 1982)
3837 post MI patients randomized341 patients found by Central Review to be ineligibleResults
% MortalityPropranolol Placebo
Eligible 7.3 9.6Ineligible 6.7 11.3Total 7.2 9.8
In the ineligible patients, treatment works bestJD Goldberg MedicReS 10162014
Data Analysis Issues
Heterogeniety among patientsNon compliance Crossovers, dropouts Approaches:
Censoring at time of crossover, dropout Causal effects and principal stratification
methods Complier average causal effects (CACE)
Data Analysis Issues continued
Missing Data Outcomes
Dummy variable to indicate whether outcome observed or not vs covariates
Covariates Multiple imputation Inverse probability weighting
Propensity score adjustments for balanceSensitivity analyses
JD Goldberg MedicReS 10162014
JD Goldberg MedicReS 10162014
Example:
New Beta-blocker for Hypertension
Changed paradigm of initial treatment of mild-moderate hypertension from monotherapy to low dose combination new beta-blocker and diuretic (standard)Designed experiment for regulatory approval of new drug Preserved monotherapy study
Primary efficacy based on increasing dose and difference between maximum dose and placebo
Allowed study of combinations
JD Goldberg MedicReS 10162014
Combination Therapy in Hypertension: Bisoprolol + Hydochlorothiazide
3 x 4 factorial
clinical trial
Frishman, etal Arch Int Med, 1984
0 6.25 25
0 60 30 30
2.5 60 30 30
10 60 30 30
40 60 30 30
Hctz mg (Standard)
NewBisop mg
JD Goldberg MedicReS 10162014
Example:
Translational Research:‘Bench to Bedside’
Issues and Environment New laboratory science Explosion of data –genomics, proteomics,… Data management and computing Cross disciplinary collaboration
Study design Reduction of data within and across
domains Integration of diverse data domains
JD Goldberg MedicReS 10162014
Translational Research Studies:Biomarkers
Investigators are provided with small number of patient samples for their substudy in context of larger project (e.g., clinical trial)Issue:
Difficult to develop comprehensive, integrated analysis of disease across all domains of data
JD Goldberg MedicReS 10162014
Systematic Missing-At-Random (SMAR) Designs for Translational Research StudiesBelitskaya-Levy, Shao, and Goldberg (2008)*
Motivation: DOD Center of Excellence: Locally Advanced Breast CancerTreatment and Prognosis
Goal: Identification of characteristics that predict pathological response to treatment, progression, and survival
Based on clinical and laboratory data
genomics, molecular/biochemical markers, immunological, hormonal markers clinical, demographic, social, cultural data
Standard chemoradiation protocol and patient follow-up • Multi-ethnic cohorts• Multiple cancer centers world wide* The International Journal of Biostatistics: http://www.bepress.com/ijb/vol4/iss1/15
JD Goldberg MedicReS 10162014
LABC: Statistical Challengesin Design and Analysis
Large sample size required for primary, secondary endpoints Costly modern technologies for laboratory studies (time, money)Inability to measure all variables on all patients
JD Goldberg MedicReS 10162014
Statistical Solution: Systematic Missing-At-Random (SMAR) Design
Entire cohort is used for measurement of endpoints, important covariates, inexpensive variablesNested random subsamples of the cohort are used to measure more ‘expensive’ classes of variablesAs cost of collection increases, random subsamples are smaller
JD Goldberg MedicReS 10162014
LABC Design: Data Structure
Types of Variables
Numberof Patients
Clinical Genomics
Molecular markers
Immunology
Mutational analyses
Hormonal assays
n1+
n0+ + + + + +
* Stratified by center
JD Goldberg MedicReS 10162014
Stratified Missing-At-Random (SMAR) Designs
JD Goldberg MedicReS 10162014
SMAR Designs: AdvantagesPlanned Missingness [monotone missing]
enables integrated analysis of entire cohort with partially observed covariates across all domains of data
statistically efficient computationally efficient cost effective allocation of resources
SMAR data are Missing-At-Random [MAR] statistical likelihood based inference valid
SMAR designs are prospective allows evaluation of efficacy, safety of
treatment, survival, …
JD Goldberg MedicReS 10162014
SMAR Design: SummaryEnables integrated statistical analysis across all data domains Statistical theory holds
SMAR is MAR
Computationally efficient Obtain cell probability estimates once prior to EM
iteration
Can use outcome (Y) in calculation of cell probabilitiesCost effective
Designed experiments
Can handle: Discrete variables with multiple categories Large numbers of observations; large numbers of
variables Heavy missingness Two stage response dependent sampling to increase
power
JD Goldberg MedicReS 10162014
Example: Active Controlled Clinical Trials*
Compare new to standard treatmentDilemma: design for superiority or non-
inferiority uncertainty about projected efficacy
of new treatment simultaneous testing?
*Shao, Y., Mukhi, V., and Goldberg, J.D.: A Hybrid Bayesian-frequentist approach to evaluate clinical trial designs for tests of superiority and non-inferiority. Stat.in Medicine 27:504–519, 2008
JD Goldberg MedicReS 10162014
Specification of Study Objective
Decision to conduct a Superiority or Non-Inferiority trial 0 (preliminary estimate of *) and ε0 (pre-specified non-inferiority
margin)
If 0 >> ε – Design Superiority
If 0 < 0 or 0 < ε – Design Non Inferiority
JD Goldberg MedicReS 10162014
Objective
SuperiorityNull hypothesis H0(0): * ≤
0Alternative hypothesis H1(0): * >
0∆* = Pe – Pc
Non-inferiority Null hypothesis H0(-ε): * ≤ - ε
Alternative hypothesis H1(-ε): * > - ε
ε ( > 0) : pre-specified non-inferiority margin
JD Goldberg MedicReS 10162014
How to design?
Single stageNI - Sup : Test non-inferiority; If non-
inferior then test superioritySup - NI : Test superiority; If superiority
fails then test non-inferiority
Adaptive or group sequential
JD Goldberg MedicReS 10162014
Single-stage Simultaneous Testing
Is it appropriate to conduct multiple tests?
Is overall type I error rate controlled?
Is power adequate?
Are the discoveries reproducible?
JD Goldberg MedicReS 10162014
Hybrid Bayesian - Frequentist Approach
[Mukhi, Shao, Goldberg]
Method: Specification of uncertainty using
distribution and Bayes formula Classical endpoint analysis
JD Goldberg MedicReS 10162014
Advantages:Hybrid Approach
Overall type I error rate is controlled Using Closed Testing Principle
Pre-specification of ε0 (non-inferiority margin) is necessaryPowerNI adequacy depends on 0 (preliminary estimate of difference) and ε0 Can plan to conduct simultaneous tests under reasonable scenarios
JD Goldberg MedicReS 10162014
Example: Patent Litigation3 clinical trials to compare 2 devices I: first in man randomized trial of 2
devices evaluated at 6, 12 months; ex US
II: randomized 2 group, evaluated at 6, 24 months; active control; single blind; ex US
III: randomized 2 group; randomized within group to 8 month evaluation (invasive); US
Different control arms
Patent Claims
all require in part that the drug delivery device has
“an in-stent diameter stenosis at 12 months . . . less than about 22%, as measured by quantitative coronary angiography.
JD Goldberg MedicReS 10162014
Example: Patent Claim of Synergy Based on Randomized Trial Data
Endpoint
Sumatriptan & Naproxen
Sumatriptan Naproxen Placebo
n % n % n % n %
Sustained Response
115 250 46.0 66 229 28.8 61 247 24.7 41 241 17.0
Sustained Pain Free
63 250 25.2 25 229 10.9 29 247 11.7 12 241 5.0
Pain Response
250 65 229 49 247 46 241 27
JD Goldberg MedicReS 10162014
Trials designed to test combination and each agent against placebo
Not designed to test for interaction
JD Goldberg MedicReS 10162014
Inclusion Criteria for Clinical Trials
Lesion
Type
Lesion
Length
Number of
Lesions
Percent
Diameter
Stenosis
Vessel Reference
Diameter
SPIRIT I de novo <18 mm 1 >50% 3.0mm
SPIRIT II de novo < 28mm 2 50% - 99% 2.5-4.25mm
SPIRIT III de novo <28mm 1 or 2 50% - 99% 2.5-3.75mm
.
And Active Control Arms Differed
JD Goldberg MedicReS 10162014
Comparison of Studies
%
Diabetic
%
Male
Proportion of
Patients With
Multiple Stents
Follow -Up
Evaluation
Time
Percent of
Patients with
Follow-up
Evaluation
SPIRIT I 11% 70.1% 1 Stent – 100% 6 mos.
12 mos.
75%
74.1%
SPIRIT II 20.2% 70.9% 1 Stent – 70%
2 Stents – 23%
3 Stents – 5%
4 Stents – 2%
6 mos.
24 mos.
74.3%
75%
SPIRIT III 29.6% 70.1% 1 Stent – 83%
2 Stents – 15%
3 Stents – 1%
4 Stents – 1%
8 mos. 80%
Study Design/Patient Populations
JD Goldberg MedicReS 10162014
Angiographic Evaluation Times and Patient Numbers
Study 6 months
8 months
12 months
24 months
I 23 22
II 223 85
III 302
JD Goldberg MedicReS 10162014
Analysis
Combined data from all 3 trials with mixed effects regression models Differences between two devices
Flawed because of study differencesPatent case won on ‘first principles’ Data not combinable Different evaluation times Different patient populations
JD Goldberg MedicReS 10162014
Example:Multicenter Randomized Clinical Trial PVSG-01: 32P vs Phlebotomy vs Chlorambucil
Issues and Environment: Multiple endpoints Long term follow-up Changes in treatment, supportive care over time Multiple analyses – ‘adjust’? ‘Intent to treat’ – not invented yet Interim stopping rules- primitive Data Safety Monitoring- ad hoc
Results: Early stopping of treatment arm (chlorambucil) Major impact on treatment of disease
JD Goldberg MedicReS 10162014
Cumulative Survival by Treatment: PVSG-01
Berk, Goldberg, et al, NEJM, 1981
JD Goldberg MedicReS 10162014
Leukemia-free Survival from Randomization
Hazard FunctionFrom Randomization
Berk, Goldberg, et al, NEJM, 1981
JD Goldberg MedicReS 10162014
Cumulative Survival by Treatment: 20 year data
From randomization
Conditional on surviving 7 years
Berk, Wasserman, Fruchtman, and Goldberg, Chap. 15, Polycythemia and the Myeloproliferative Disorders, ed. Wasserman, et al, Saunders, 1995.
JD Goldberg MedicReS 10162014
Examples:New areas for statistical collaboration and methodology development
ProteomicsImagingBiomarkersGenetics, gene-environment interactions
-----------------------------------------------------------Adaptive clinical trial designs, other ‘new’ designsSafety assessmentCombining data from multiple sourcesComparative effectiveness research…
JD Goldberg MedicReS 10162014
Where next?
Need for collaboration with scientists greater than ever throughout research process from inceptionContinue to exploit new technologiesContinue to make explicit the IT requirements for infrastructure to enable new approachesContinue to expand role of biostatistics in drug development
Includes compound screening, high throughput technologies
Clinical translational research including clinical trials (controlled and uncontrolled), meta-analysis, safety evaluation, comparative effectiveness research
Continue to stretch the boundaries of statistics and statistical thinkingStrategic input into drug development from compound identification, patent development, post marketing effectiveness and safety evaluation
JD Goldberg MedicReS 10162014
Thank you to collaborators and colleagues:
Health Insurance Plan of Greater New YorkMount Sinai School of MedicineLederle Laboratories, American Cyanamid D. Alemayehu, K. Koury, …
Bristol-Myers SquibbNew York University School of Medicine Y. Shao, M. Liu, I. Belitskaya-Levy, V. Mukhi,
…
Herman P. Friedman…
JD Goldberg MedicReS 10162014
Currently supported in part by:
NYU Cancer Center Support Grant:NCI P30 CA16087
NYU Clinical Translational Science Award: 1UL1RR029893MPD Research Consortium: P01 CA108671 Locally Advanced Breast Cancer Center of Excellence: DOD W81XWH-04-2-0905
top related