![Page 1: Design & Interpretation of Randomized Trials: A Clinician’s Perspective Francis KL Chan Department of Medicine & Therapeutics CUHK](https://reader035.vdocuments.mx/reader035/viewer/2022062320/56649cdc5503460f949a6ca1/html5/thumbnails/1.jpg)
Design & Interpretation of Randomized Trials:A Clinician’s PerspectiveFrancis KL ChanDepartment of Medicine & TherapeuticsCUHK
![Page 2: Design & Interpretation of Randomized Trials: A Clinician’s Perspective Francis KL Chan Department of Medicine & Therapeutics CUHK](https://reader035.vdocuments.mx/reader035/viewer/2022062320/56649cdc5503460f949a6ca1/html5/thumbnails/2.jpg)
Common problems of RCTs
Originality
Hypothesis
Allocation concealment & randomization
Evaluation of baseline data
“Intention-to-treat” analysis
Subgroup analysis
![Page 3: Design & Interpretation of Randomized Trials: A Clinician’s Perspective Francis KL Chan Department of Medicine & Therapeutics CUHK](https://reader035.vdocuments.mx/reader035/viewer/2022062320/56649cdc5503460f949a6ca1/html5/thumbnails/3.jpg)
Is the study original?
Ground breaking research?
Does this work add to the literature in any way?
Bigger, longer?
More rigorous methodology?
Results add to a meta-analysis of previous studies?
Different population (age, sex, ethnic groups)?
![Page 4: Design & Interpretation of Randomized Trials: A Clinician’s Perspective Francis KL Chan Department of Medicine & Therapeutics CUHK](https://reader035.vdocuments.mx/reader035/viewer/2022062320/56649cdc5503460f949a6ca1/html5/thumbnails/4.jpg)
Hypothesis & End pointMany RCTs did not explicitly state their study
hypotheses
“The aim of this study was to compare the efficacy of a new treatment with the standard treatment…”
Hypothesis 1:
Treatment A is superior to the standard treatment
Hypothesis 2:
Treatment A is equivalent to the standard treatment
![Page 5: Design & Interpretation of Randomized Trials: A Clinician’s Perspective Francis KL Chan Department of Medicine & Therapeutics CUHK](https://reader035.vdocuments.mx/reader035/viewer/2022062320/56649cdc5503460f949a6ca1/html5/thumbnails/5.jpg)
Hypothesis?
Sample size estimation
None!
![Page 6: Design & Interpretation of Randomized Trials: A Clinician’s Perspective Francis KL Chan Department of Medicine & Therapeutics CUHK](https://reader035.vdocuments.mx/reader035/viewer/2022062320/56649cdc5503460f949a6ca1/html5/thumbnails/6.jpg)
Failure to detect a difference
=Equivalence?
![Page 7: Design & Interpretation of Randomized Trials: A Clinician’s Perspective Francis KL Chan Department of Medicine & Therapeutics CUHK](https://reader035.vdocuments.mx/reader035/viewer/2022062320/56649cdc5503460f949a6ca1/html5/thumbnails/7.jpg)
Superiority Trial
The new treatment (µN) is superior to the
standard treatment (µS) if the difference
exceeds by a clinically important amount ().
Test hypothesis (H): µN - µS>
![Page 8: Design & Interpretation of Randomized Trials: A Clinician’s Perspective Francis KL Chan Department of Medicine & Therapeutics CUHK](https://reader035.vdocuments.mx/reader035/viewer/2022062320/56649cdc5503460f949a6ca1/html5/thumbnails/8.jpg)
Equivalence trial
The new treatment is equivalent to the
standard treatment if the maximal allowable
difference does not exceed by a clinically
important amount.
![Page 9: Design & Interpretation of Randomized Trials: A Clinician’s Perspective Francis KL Chan Department of Medicine & Therapeutics CUHK](https://reader035.vdocuments.mx/reader035/viewer/2022062320/56649cdc5503460f949a6ca1/html5/thumbnails/9.jpg)
Equivalence trial
- 0 +
Equivalent
Difference
Not equivalent
Favors new treatment
Not equivalent
Favors standard
treatment
New agent is not inferior to the
standard
![Page 10: Design & Interpretation of Randomized Trials: A Clinician’s Perspective Francis KL Chan Department of Medicine & Therapeutics CUHK](https://reader035.vdocuments.mx/reader035/viewer/2022062320/56649cdc5503460f949a6ca1/html5/thumbnails/10.jpg)
Assume non-inferiority if the lower limit of 95% CI is
less than –5%,N=904 per group!
![Page 11: Design & Interpretation of Randomized Trials: A Clinician’s Perspective Francis KL Chan Department of Medicine & Therapeutics CUHK](https://reader035.vdocuments.mx/reader035/viewer/2022062320/56649cdc5503460f949a6ca1/html5/thumbnails/11.jpg)
Allocation concealment & randomization
Concealment of allocation
(investigators and patients not knowing the assigned
treatment before randomization)
Was treatment assigned by an independent staff?
What was the method of allocation concealment?
contact with central office
blinded packages
sealed (opaque) envelopes
![Page 12: Design & Interpretation of Randomized Trials: A Clinician’s Perspective Francis KL Chan Department of Medicine & Therapeutics CUHK](https://reader035.vdocuments.mx/reader035/viewer/2022062320/56649cdc5503460f949a6ca1/html5/thumbnails/12.jpg)
Allocation concealment
NEJM, JAMA, Lancet, BMJ (N=50, Jul - Sep 97)
44%
56%
YesNo
![Page 13: Design & Interpretation of Randomized Trials: A Clinician’s Perspective Francis KL Chan Department of Medicine & Therapeutics CUHK](https://reader035.vdocuments.mx/reader035/viewer/2022062320/56649cdc5503460f949a6ca1/html5/thumbnails/13.jpg)
Comparison of baseline data
Chan et al. Lancet 1997
Does P>0.05 indicate
comparability of treatment groups?
![Page 14: Design & Interpretation of Randomized Trials: A Clinician’s Perspective Francis KL Chan Department of Medicine & Therapeutics CUHK](https://reader035.vdocuments.mx/reader035/viewer/2022062320/56649cdc5503460f949a6ca1/html5/thumbnails/14.jpg)
Baseline data
Azathioprine Placebo
Mean age 54.7 54.9
Serum bilirubin (mol/L) 37.2 30.9
Stage I disease % 14 12
Stage II disease % 44 43
Stage III disease % 15 15
Stage IV disease % 27 30
Christensen et al. Gastro 1985
Effect of azathioprine on the survival of patients with primary biliary cirrhosis
![Page 15: Design & Interpretation of Randomized Trials: A Clinician’s Perspective Francis KL Chan Department of Medicine & Therapeutics CUHK](https://reader035.vdocuments.mx/reader035/viewer/2022062320/56649cdc5503460f949a6ca1/html5/thumbnails/15.jpg)
Baseline data
Azathioprine Placebo
Mean age 54.7 54.9
Serum bilirubin (mol/L) 37.2 30.9
Stage I disease % 14 12
Stage II disease % 44 43
Stage III disease % 15 15
Stage IV disease % 27 30
Christensen et al. Gastro 1985
Effect of azathioprine on the survival of patients with primary biliary cirrhosis
![Page 16: Design & Interpretation of Randomized Trials: A Clinician’s Perspective Francis KL Chan Department of Medicine & Therapeutics CUHK](https://reader035.vdocuments.mx/reader035/viewer/2022062320/56649cdc5503460f949a6ca1/html5/thumbnails/16.jpg)
UnadjustedP=0.10
Adjusted for bilirubin
P=0.01
![Page 17: Design & Interpretation of Randomized Trials: A Clinician’s Perspective Francis KL Chan Department of Medicine & Therapeutics CUHK](https://reader035.vdocuments.mx/reader035/viewer/2022062320/56649cdc5503460f949a6ca1/html5/thumbnails/17.jpg)
P=0.04
P=0.02
Columbus Investigators. NEJM 1997
![Page 18: Design & Interpretation of Randomized Trials: A Clinician’s Perspective Francis KL Chan Department of Medicine & Therapeutics CUHK](https://reader035.vdocuments.mx/reader035/viewer/2022062320/56649cdc5503460f949a6ca1/html5/thumbnails/18.jpg)
Significant imbalance may not affect outcome
Comparison of baseline data
Non-significant imbalance may affect outcome
Significance tests for baseline differences are inappropriate.
![Page 19: Design & Interpretation of Randomized Trials: A Clinician’s Perspective Francis KL Chan Department of Medicine & Therapeutics CUHK](https://reader035.vdocuments.mx/reader035/viewer/2022062320/56649cdc5503460f949a6ca1/html5/thumbnails/19.jpg)
Significance tests for baseline differences
Chan et al. Lancet 1997
INAPPROPRIA
TE
![Page 20: Design & Interpretation of Randomized Trials: A Clinician’s Perspective Francis KL Chan Department of Medicine & Therapeutics CUHK](https://reader035.vdocuments.mx/reader035/viewer/2022062320/56649cdc5503460f949a6ca1/html5/thumbnails/20.jpg)
Significant imbalance may not affect outcome
Comparison of baseline data
Non-significant imbalance may affect outcome
Significance tests for baseline differences are inappropriate.
Table of baseline data should focus on factors affecting outcome.
![Page 21: Design & Interpretation of Randomized Trials: A Clinician’s Perspective Francis KL Chan Department of Medicine & Therapeutics CUHK](https://reader035.vdocuments.mx/reader035/viewer/2022062320/56649cdc5503460f949a6ca1/html5/thumbnails/21.jpg)
45 baseline factors!
![Page 22: Design & Interpretation of Randomized Trials: A Clinician’s Perspective Francis KL Chan Department of Medicine & Therapeutics CUHK](https://reader035.vdocuments.mx/reader035/viewer/2022062320/56649cdc5503460f949a6ca1/html5/thumbnails/22.jpg)
Significant imbalance may not affect outcome
Comparison of baseline data
Non-significant imbalance may affect outcome
Significance tests for baseline differences are inappropriate.
Table of baseline data should focus on factors affecting outcome.
Analysis adjusted for baseline factors that are known to strongly influence the outcome (Covariate-adjusted analysis).
Analysis of covariance for a quantitative outcome
Logistic regression for a binary response
Cox’s-proportional hazard model for time-to-event data
![Page 23: Design & Interpretation of Randomized Trials: A Clinician’s Perspective Francis KL Chan Department of Medicine & Therapeutics CUHK](https://reader035.vdocuments.mx/reader035/viewer/2022062320/56649cdc5503460f949a6ca1/html5/thumbnails/23.jpg)
“Intention-To-Treat” Analysis
“…results were analyzed according to the ITT principle.”
Question:
How were missing outcomes/ protocol violators
handled in the so called “ITT” analysis?
![Page 24: Design & Interpretation of Randomized Trials: A Clinician’s Perspective Francis KL Chan Department of Medicine & Therapeutics CUHK](https://reader035.vdocuments.mx/reader035/viewer/2022062320/56649cdc5503460f949a6ca1/html5/thumbnails/24.jpg)
“Intention-To-Treat” Analysis
Endpt
Savage et al. NEJM 1997
![Page 25: Design & Interpretation of Randomized Trials: A Clinician’s Perspective Francis KL Chan Department of Medicine & Therapeutics CUHK](https://reader035.vdocuments.mx/reader035/viewer/2022062320/56649cdc5503460f949a6ca1/html5/thumbnails/25.jpg)
Minimize missing response on primary outcome
Recommendations for ITT Analysis
Follow up subjects who withdraw early
Investigate & report the effect of missing response
Report all deviations and missing response
![Page 26: Design & Interpretation of Randomized Trials: A Clinician’s Perspective Francis KL Chan Department of Medicine & Therapeutics CUHK](https://reader035.vdocuments.mx/reader035/viewer/2022062320/56649cdc5503460f949a6ca1/html5/thumbnails/26.jpg)
Subgroup AnalysisRandomised trial of home-based psychosocial nursing intervention for patients recovering from myocardial infarction. Frasure-Smith et al. Lancet 1997
“…The poor overall outcome for women, and the possible harmful impact of the intervention on women, underlie the need for…”
![Page 27: Design & Interpretation of Randomized Trials: A Clinician’s Perspective Francis KL Chan Department of Medicine & Therapeutics CUHK](https://reader035.vdocuments.mx/reader035/viewer/2022062320/56649cdc5503460f949a6ca1/html5/thumbnails/27.jpg)
Effect of antenatal dexamethasone administration on the prevention of respiratory distress syndrome.
Am J Obstet Gynecol 1981;141:276-87.
Subgroup Analysis
Steroid Placebo P
Pre-ecclampsia 21.2%
(7/33)
27.3%
(9/33)
0.57
No pre-ecclampsia 7.9%
(21/267)
14.1%
(37/262)
0.021
![Page 28: Design & Interpretation of Randomized Trials: A Clinician’s Perspective Francis KL Chan Department of Medicine & Therapeutics CUHK](https://reader035.vdocuments.mx/reader035/viewer/2022062320/56649cdc5503460f949a6ca1/html5/thumbnails/28.jpg)
Effect of antenatal dexamethasone administration on the prevention of respiratory distress syndrome.
Am J Obstet Gynecol 1981;141:276-87.
Subgroup Analysis
Steroid Placebo P
Pre-ecclampsia 21.2%
(7/33)
27.3%
(9/33)
0.57
No pre-ecclampsia 7.9%
(21/267)
14.1%
(37/262)
0.021
Difference 6.1%
![Page 29: Design & Interpretation of Randomized Trials: A Clinician’s Perspective Francis KL Chan Department of Medicine & Therapeutics CUHK](https://reader035.vdocuments.mx/reader035/viewer/2022062320/56649cdc5503460f949a6ca1/html5/thumbnails/29.jpg)
Effect of antenatal dexamethasone administration on the prevention of respiratory distress syndrome.
Am J Obstet Gynecol 1981;141:276-87.
Subgroup Analysis
Steroid Placebo P
Pre-ecclampsia 21.2%
(7/33)
27.3%
(9/33)
0.57
No pre-ecclampsia 7.9%
(21/267)
14.1%
(37/262)
0.021
Difference 6.1%
Difference 6.2%
P value depends on effect size & SE
![Page 30: Design & Interpretation of Randomized Trials: A Clinician’s Perspective Francis KL Chan Department of Medicine & Therapeutics CUHK](https://reader035.vdocuments.mx/reader035/viewer/2022062320/56649cdc5503460f949a6ca1/html5/thumbnails/30.jpg)
Evaluation of Subgroup Analysis
Tests of interaction (assess whether a treatment
effect differs between subgroups) rather than
subgroup P values
Diff in Subgroup A – Diff in Subgroup B
SE of the above Diff Z =
![Page 31: Design & Interpretation of Randomized Trials: A Clinician’s Perspective Francis KL Chan Department of Medicine & Therapeutics CUHK](https://reader035.vdocuments.mx/reader035/viewer/2022062320/56649cdc5503460f949a6ca1/html5/thumbnails/31.jpg)
Trial of vitamin D supplements in pregnancy to prevent infant hypocalcemia. BMJ 1980;281:11-4.
Infant plasma calcium (mg/100ml)
Mean SEM P
Vit D 9.20 0.085 Bottle-fed
Placebo 8.78 0.076
0.0006
Vit D 9.79 0.146 Breast-fed Placebo 9.64 0.125
0.4
Interaction TestDifference = 0.42
Difference = 0.15
0.42 – 0.15 = 0.27
SE of this Diff = 0.22
Z = Diff / SE = 1.23
P = 0.2No evidence th
at the effe
ct of V
it D is
different
between bottle
-fed and breast-fe
d infants
![Page 32: Design & Interpretation of Randomized Trials: A Clinician’s Perspective Francis KL Chan Department of Medicine & Therapeutics CUHK](https://reader035.vdocuments.mx/reader035/viewer/2022062320/56649cdc5503460f949a6ca1/html5/thumbnails/32.jpg)
General points regarding subgroup analysis
Emphasis should remain on overall comparison
More convincing if confined to a limited number of pre-specified subgroup hypothesis
Rely on interaction tests, not P values
View subgroup findings as exploratory (to be confirmed in subsequent trials)