common pitfalls in clinical trials_010511
TRANSCRIPT
8/6/2019 Common Pitfalls in Clinical Trials_010511
http://slidepdf.com/reader/full/common-pitfalls-in-clinical-trials010511 1/28
8/6/2019 Common Pitfalls in Clinical Trials_010511
http://slidepdf.com/reader/full/common-pitfalls-in-clinical-trials010511 2/28
8/6/2019 Common Pitfalls in Clinical Trials_010511
http://slidepdf.com/reader/full/common-pitfalls-in-clinical-trials010511 3/28
HRT trial
Hormone replacement therapy
Multiple observational epidemiological study show benefit
Three clinical trials show evenharmful
8/6/2019 Common Pitfalls in Clinical Trials_010511
http://slidepdf.com/reader/full/common-pitfalls-in-clinical-trials010511 4/28
Outcome Treatment
Attributes/
confounders
Randomization
blocks this
8/6/2019 Common Pitfalls in Clinical Trials_010511
http://slidepdf.com/reader/full/common-pitfalls-in-clinical-trials010511 5/28
NASCIS trial
National Acute Spinal Cord Injury Studies
Methyprednisolone Rx in acute spinal cord injury
CAST trialCardiac arrhythmia suppression trial
Treat PVC or not?
CONVINCE trial
Controlled Onset Verapamil Investigation of Cardiovascular Endpoints
Stopped prematurely for commercial reason
8/6/2019 Common Pitfalls in Clinical Trials_010511
http://slidepdf.com/reader/full/common-pitfalls-in-clinical-trials010511 6/28
Design
Analysis
Presentation
8/6/2019 Common Pitfalls in Clinical Trials_010511
http://slidepdf.com/reader/full/common-pitfalls-in-clinical-trials010511 7/28
Ethical issue
Phase of trials
Sampling issue
Randomization issue
Blinding/masking
Measurement issue
Power issue
8/6/2019 Common Pitfalls in Clinical Trials_010511
http://slidepdf.com/reader/full/common-pitfalls-in-clinical-trials010511 8/28
Adherence issue
Multiple comparison
Statistical error
8/6/2019 Common Pitfalls in Clinical Trials_010511
http://slidepdf.com/reader/full/common-pitfalls-in-clinical-trials010511 9/28
Inappropriate or poor reporting of results
Wrong or poor interpretation of results
8/6/2019 Common Pitfalls in Clinical Trials_010511
http://slidepdf.com/reader/full/common-pitfalls-in-clinical-trials010511 10/28
Fail to recognize roles of Researcher vs.Physician
Risk/benefit equipoise
Employee/vulnerable population
Responsibility of investigators
Collaboration with pharmaceuticals
Fail to understand the related regulation
8/6/2019 Common Pitfalls in Clinical Trials_010511
http://slidepdf.com/reader/full/common-pitfalls-in-clinical-trials010511 11/28
Fail to form DSMB
Early termination/extended recruitment
What if Placebo not ethical?
Equivalence trial
Active control
Effectiveness issue
CONVINCE trialNon-inferiority trial
CAST trial
VAL/HCTZ (Valsartin/Hydrochlorthiazide) vs. Amlodipine
0% 10 %
Ho Ha
0% 20 %
Ho Ha
0% 15 %
Ho Ha
!"
33% reduction
50% reduction
67% reduction
8/6/2019 Common Pitfalls in Clinical Trials_010511
http://slidepdf.com/reader/full/common-pitfalls-in-clinical-trials010511 12/28
Alternative RCTType
Null Hypothesis Hypothesis
Traditional New = old New ≠ old
Equivalence New < old + δ New ≥ old + δ
Non-inferiority New ⊀ old New = old
8/6/2019 Common Pitfalls in Clinical Trials_010511
http://slidepdf.com/reader/full/common-pitfalls-in-clinical-trials010511 13/28
8/6/2019 Common Pitfalls in Clinical Trials_010511
http://slidepdf.com/reader/full/common-pitfalls-in-clinical-trials010511 14/28
8/6/2019 Common Pitfalls in Clinical Trials_010511
http://slidepdf.com/reader/full/common-pitfalls-in-clinical-trials010511 15/28
8/6/2019 Common Pitfalls in Clinical Trials_010511
http://slidepdf.com/reader/full/common-pitfalls-in-clinical-trials010511 16/28
Fail to report inclusion/exclusion criteria
Fail to obtain equipoise between:
Homogeneity
Generalizability
Modified Zelen’s design
Double randomized consent trial
8/6/2019 Common Pitfalls in Clinical Trials_010511
http://slidepdf.com/reader/full/common-pitfalls-in-clinical-trials010511 17/28
Method of randomization not clearly stated
Simple, Restricted, or Adaptive, which & why?
How to have good randomization?
Independent group
Reproducible, unpredictable
Adherence, blinding
Verify, audit, periodical check
8/6/2019 Common Pitfalls in Clinical Trials_010511
http://slidepdf.com/reader/full/common-pitfalls-in-clinical-trials010511 18/28
Hard to mask for device trial
Patient bias (placebo/sham effect)
Investigator bias
Evaluator bias
PROBE design (Prospective randomizedopen blinded end-point)
8/6/2019 Common Pitfalls in Clinical Trials_010511
http://slidepdf.com/reader/full/common-pitfalls-in-clinical-trials010511 19/28
Instrument utilization
Even VAS not optimal for pain
Objective is better: heart rate
Suboptimal measurement
Surrogate outcome/exposure
CAST trial
Duration of follow up
8/6/2019 Common Pitfalls in Clinical Trials_010511
http://slidepdf.com/reader/full/common-pitfalls-in-clinical-trials010511 20/28
Fail to estimation recruitment feasibility
Fail to report withdrawals
Fail to calculation for interim analysis
Fail to use efficient trial
Non-inferiority trial
CAST for one-sided issue
8/6/2019 Common Pitfalls in Clinical Trials_010511
http://slidepdf.com/reader/full/common-pitfalls-in-clinical-trials010511 21/28
As-treated analysis
Compare who actually received treatment vs. control
Ignore randomization
Per-protocol analysis
Compare who comply with assigned treatment
Compare compliers in treatment group with full control group
Intention-to-treat analysisStandard estimate
Ignore compliance just use randomization
Instrumental variable/complier average causal effect (IV/CACE)
8/6/2019 Common Pitfalls in Clinical Trials_010511
http://slidepdf.com/reader/full/common-pitfalls-in-clinical-trials010511 22/28
Failure to include a multiple-comparisoncorrection
NASCIS
Inflation of Type I error
Post-hoc subgroup analysis
8/6/2019 Common Pitfalls in Clinical Trials_010511
http://slidepdf.com/reader/full/common-pitfalls-in-clinical-trials010511 23/28
Typical errors with Student’s t-test
Failure to prove test assumptions
Use of an unpaired t-test for paired data or vice versa
Unequal sample sizes for paired t-test
Improper multiple pair-wise comparisons
8/6/2019 Common Pitfalls in Clinical Trials_010511
http://slidepdf.com/reader/full/common-pitfalls-in-clinical-trials010511 24/28
Typical errors with Chi-square tests
No Yates-continuity correction reported if small numbers
Use of chi-square when expected numbersin a cell <5
No explicit statement of tested Null-Hypotheses
8/6/2019 Common Pitfalls in Clinical Trials_010511
http://slidepdf.com/reader/full/common-pitfalls-in-clinical-trials010511 25/28
Report p-values only, no confidence intervals
CIs given for each group rather than forcontrasts
“p = NS”, “p <0.05” or other arbitrary thresholds instead of reporting exact p-values
Numerical information given to an unrealisticlevel of precisions
8/6/2019 Common Pitfalls in Clinical Trials_010511
http://slidepdf.com/reader/full/common-pitfalls-in-clinical-trials010511 26/28
“Non significant” interpreted as “no effect”, or “no difference”
Drawing conclusions not supported by data
Significance claimed without data analysis or statistical testmentioned
Disregard for Type II error when reporting non-significantresults
Missing discussion of the problem of multiple significance testing if done
Failure to discuss sources of potential bias and confoundingfactors
8/6/2019 Common Pitfalls in Clinical Trials_010511
http://slidepdf.com/reader/full/common-pitfalls-in-clinical-trials010511 27/28
What we need is a brand new idea that has been thoroughly tested!
8/6/2019 Common Pitfalls in Clinical Trials_010511
http://slidepdf.com/reader/full/common-pitfalls-in-clinical-trials010511 28/28
Questions?