![Page 1: Design and Analysis of Clinical Study 12. Meta-analysis Dr. Tuan V. Nguyen Garvan Institute of Medical Research Sydney, Australia](https://reader035.vdocuments.mx/reader035/viewer/2022081512/56649eef5503460f94bff094/html5/thumbnails/1.jpg)
Design and Analysis of Clinical Study 12. Meta-analysis
Dr. Tuan V. Nguyen
Garvan Institute of Medical Research
Sydney, Australia
![Page 2: Design and Analysis of Clinical Study 12. Meta-analysis Dr. Tuan V. Nguyen Garvan Institute of Medical Research Sydney, Australia](https://reader035.vdocuments.mx/reader035/viewer/2022081512/56649eef5503460f94bff094/html5/thumbnails/2.jpg)
Overview
• What is meta-analysis• Two types of data• Statistical procedures
![Page 3: Design and Analysis of Clinical Study 12. Meta-analysis Dr. Tuan V. Nguyen Garvan Institute of Medical Research Sydney, Australia](https://reader035.vdocuments.mx/reader035/viewer/2022081512/56649eef5503460f94bff094/html5/thumbnails/3.jpg)
Why Meta-analysis/Systematic Reviews?
• “. . . the mass of new information makes it difficult for practicing physicians to follow the literature in all areas that might be relevant to their practices. New methods to synthesize and present information from widely dispersed publications are needed . . . .”
Jerome Kassirer. Clinical trials and meta-analysis: what do they do for us? N Engl J Med 1992; 327:273-4.
![Page 4: Design and Analysis of Clinical Study 12. Meta-analysis Dr. Tuan V. Nguyen Garvan Institute of Medical Research Sydney, Australia](https://reader035.vdocuments.mx/reader035/viewer/2022081512/56649eef5503460f94bff094/html5/thumbnails/4.jpg)
Why Need Meta-analysis? Information Explosion
• 10-fold Increase in Number of Professional Journals
• Psychology Journals: 91 (1951) --> 1,175 (1992)
• • Math Science Journals:
91 (1953) --> 920 (1992)
• Biomedical Journals: 2,300 (1940)--> 23,000 (1993)
![Page 5: Design and Analysis of Clinical Study 12. Meta-analysis Dr. Tuan V. Nguyen Garvan Institute of Medical Research Sydney, Australia](https://reader035.vdocuments.mx/reader035/viewer/2022081512/56649eef5503460f94bff094/html5/thumbnails/5.jpg)
Problem – Conflicting Information
• Not only is there more information, but . . .
• Not all information is of equal quality
• Information does not necessarily = evidence
• There is often conflicting information & reports Traditional narrative reviews can be very “impressionistic”
![Page 6: Design and Analysis of Clinical Study 12. Meta-analysis Dr. Tuan V. Nguyen Garvan Institute of Medical Research Sydney, Australia](https://reader035.vdocuments.mx/reader035/viewer/2022081512/56649eef5503460f94bff094/html5/thumbnails/6.jpg)
Problems With Traditional Literature Reviews Addressed in Meta-analysis
• Selective inclusion of studies, often based on the reviewer's own impressionistic view of the quality of the study
• Differential subjective weighting of studies in the interpretation of a set of findings
• Misleading interpretations of study findings • Failure to examine characteristics of the studies as
potential explanations for disparate or inconsistent results across studies
• Failure to examine moderating variables in the relationship under examination
![Page 7: Design and Analysis of Clinical Study 12. Meta-analysis Dr. Tuan V. Nguyen Garvan Institute of Medical Research Sydney, Australia](https://reader035.vdocuments.mx/reader035/viewer/2022081512/56649eef5503460f94bff094/html5/thumbnails/7.jpg)
Rationale for Systematic Reviews
• “provide summaries of what we know, and do not know, that are as free from bias as possible.” (Chalmers et al 1999)
• “research that uses explicit & transparent methods to synthesise relevant studies, allowing others to comment on, criticise or attempt to replicate the conclusions reached. Systematic reviews follow same set of procedures as any individual study, & are often reported in the same way. . . .” (Petrsino et al 1999)
![Page 8: Design and Analysis of Clinical Study 12. Meta-analysis Dr. Tuan V. Nguyen Garvan Institute of Medical Research Sydney, Australia](https://reader035.vdocuments.mx/reader035/viewer/2022081512/56649eef5503460f94bff094/html5/thumbnails/8.jpg)
4 Basic Questions That a SR/MA Tries to Answer
• Are the results of the different studies similar?
• To the extent that they are similar, what is the best overall estimate of effect?
• How precise and robust is this estimate?
• Can dissimilarities be explained?
Lau J, Ioannidis JPA, Schmid CH. Quantitative Synthesis in Systematic Reviews. Annals of Internal Medicine 1997; 127:820-826.
![Page 9: Design and Analysis of Clinical Study 12. Meta-analysis Dr. Tuan V. Nguyen Garvan Institute of Medical Research Sydney, Australia](https://reader035.vdocuments.mx/reader035/viewer/2022081512/56649eef5503460f94bff094/html5/thumbnails/9.jpg)
What is a Systematic Review?
• Assemble the most complete dataset feasible, with involvement of investigators
• Analyse results of eligible studies. Use statistical synthesis of data (meta-analysis) if appropriate & possible
• Perform sensitivity analyses, if appropriate & possible (including subgroup analyses)
• Prepare a structured report of the review, stating aims, describing materials & methods, & reporting results
![Page 10: Design and Analysis of Clinical Study 12. Meta-analysis Dr. Tuan V. Nguyen Garvan Institute of Medical Research Sydney, Australia](https://reader035.vdocuments.mx/reader035/viewer/2022081512/56649eef5503460f94bff094/html5/thumbnails/10.jpg)
Cochrane Library
• Cochrane Library CD (& WWW)
• Cochrane Database of Systematic Reviews (CDSR)
• Database of Abstracts of Reviews of Effectiveness (DARE)
• Cochrane Central Register of Controlled Trials (CENTRAL)
• Cochrane Review Methodology Database
• Health Technology Assessment DB (HTA)
• NHS Economic/Evaluation Database (NHS EED)
![Page 11: Design and Analysis of Clinical Study 12. Meta-analysis Dr. Tuan V. Nguyen Garvan Institute of Medical Research Sydney, Australia](https://reader035.vdocuments.mx/reader035/viewer/2022081512/56649eef5503460f94bff094/html5/thumbnails/11.jpg)
Search Strategy – References & Databases
• Studies were identified from – Cochrane Airways Group's Special Register of Controlled
Trials comprised of references from
– MEDLINE (1966-2000)
– EMBASE (1980-2000)
– CINAHL (1982-2000)
• hand searched airways-related journals • PsychINFO • Reference lists from relevant review articles that were
identified (ancestry approach
![Page 12: Design and Analysis of Clinical Study 12. Meta-analysis Dr. Tuan V. Nguyen Garvan Institute of Medical Research Sydney, Australia](https://reader035.vdocuments.mx/reader035/viewer/2022081512/56649eef5503460f94bff094/html5/thumbnails/12.jpg)
Search Strategy - Terms
• Congestive Heart Failure OR Heart Failure* AND • clinical trial* OR beta blocker*• placebo* OR trial* OR random* OR double-blind OR
double blind OR single-blind OR single blind OR controlled study OR comparative study.
![Page 13: Design and Analysis of Clinical Study 12. Meta-analysis Dr. Tuan V. Nguyen Garvan Institute of Medical Research Sydney, Australia](https://reader035.vdocuments.mx/reader035/viewer/2022081512/56649eef5503460f94bff094/html5/thumbnails/13.jpg)
Identification of Trials
• Potentially relevant studies from literature search and hand searches
• Excluded on basis of abstract, e.g., not randomised or controlled clinical trials Articles selected for full text review
• Excluded after full text review • Eligible trials
![Page 14: Design and Analysis of Clinical Study 12. Meta-analysis Dr. Tuan V. Nguyen Garvan Institute of Medical Research Sydney, Australia](https://reader035.vdocuments.mx/reader035/viewer/2022081512/56649eef5503460f94bff094/html5/thumbnails/14.jpg)
Main Outcome Measures
• Mortality / death
![Page 15: Design and Analysis of Clinical Study 12. Meta-analysis Dr. Tuan V. Nguyen Garvan Institute of Medical Research Sydney, Australia](https://reader035.vdocuments.mx/reader035/viewer/2022081512/56649eef5503460f94bff094/html5/thumbnails/15.jpg)
Beta-blocker and Congestive Heart Failure
Study(i)
Beta-blocker Placebo
N1 Deaths (d1) N2 Deaths (d2)
1 25 5 25 6
2 9 1 16 2
3 194 23 189 21
4 25 1 25 2
5 105 4 34 2
6 320 53 321 67
7 33 3 16 2
8 261 12 84 13
9 133 6 145 11
10 232 2 134 5
11 1327 156 1320 228
12 1990 145 2001 217
13 214 8 212 17
Tổng cộng 4879 420 4516 612
![Page 16: Design and Analysis of Clinical Study 12. Meta-analysis Dr. Tuan V. Nguyen Garvan Institute of Medical Research Sydney, Australia](https://reader035.vdocuments.mx/reader035/viewer/2022081512/56649eef5503460f94bff094/html5/thumbnails/16.jpg)
Model of Meta-analysis
• For each study– Relative risk
– Variance and standard error of logRR
1 1 1 2 2 2
1 1 1 1log iSE RR
d N d d N d
1
2i
pRR
p
1 1 1 2 2 2
1 1 1 1var log iRR
d N d d N d
log 1.96 loge RR SE RR
– 95% confidence interval of RR
– Weight
1
var logii
WRR
![Page 17: Design and Analysis of Clinical Study 12. Meta-analysis Dr. Tuan V. Nguyen Garvan Institute of Medical Research Sydney, Australia](https://reader035.vdocuments.mx/reader035/viewer/2022081512/56649eef5503460f94bff094/html5/thumbnails/17.jpg)
Model of Meta-analysis
• For all studies
– Overall relative risk
loglog
i i
i
W RRRR
W
– Variance and standard error
1var log
i
RRW
1log
i
SE RRW
– 95% confidence interval
log 1.96 logRR SE RR
![Page 18: Design and Analysis of Clinical Study 12. Meta-analysis Dr. Tuan V. Nguyen Garvan Institute of Medical Research Sydney, Australia](https://reader035.vdocuments.mx/reader035/viewer/2022081512/56649eef5503460f94bff094/html5/thumbnails/18.jpg)
Meta-analysis: an example
Study p1 p2 RRi logRRi Var[logRR] Wi Wi×log[RRi]
1 0.200 0.240 0.833 -0.182 0.264 3.79 -0.69
2 0.111 0.125 0.889 -0.118 1.304 0.77 -0.09
3 0.119 0.111 1.067 0.065 0.079 12.61 0.82
4 0.040 0.080 0.500 -0.693 1.415 0.71 -0.49
5 0.038 0.059 0.648 -0.434 0.709 1.41 -0.61
6 0.166 0.209 0.794 -0.231 0.026 38.30 -8.86
7 0.091 0.125 0.727 -0.318 0.729 1.37 -0.44
8 0.046 0.155 0.297 -1.214 0.142 7.03 -8.54
9 0.045 0.076 0.595 -0.520 0.242 4.13 -2.15
10 0.009 0.037 0.231 -1.465 0.688 1.45 -2.13
11 0.118 0.1730.681 -0.385 0.009 110.78 -42.63
12 0.073 0.108 0.672 -0.398 0.010 96.13 -38.23
13 0.037 0.080 0.466 -0.763 0.174 5.75 -4.39
284.24 -108.42
![Page 19: Design and Analysis of Clinical Study 12. Meta-analysis Dr. Tuan V. Nguyen Garvan Institute of Medical Research Sydney, Australia](https://reader035.vdocuments.mx/reader035/viewer/2022081512/56649eef5503460f94bff094/html5/thumbnails/19.jpg)
Meta-analysis: an example
95% CI of logRR = -0.38 ± 1.96×0.06 = -0.498, -0.265
95% of RR: exp(-0.498) = 0.61 to exp(-0.265) = 0.77
log108.42
log 0.38284.24
i i
i
W RRwRR
W
1 1log 0.0035
284.24i
Var wRRW
1log 0.0035 0.06
i
SE wRRW
![Page 20: Design and Analysis of Clinical Study 12. Meta-analysis Dr. Tuan V. Nguyen Garvan Institute of Medical Research Sydney, Australia](https://reader035.vdocuments.mx/reader035/viewer/2022081512/56649eef5503460f94bff094/html5/thumbnails/20.jpg)
Meta-analysis using R
library(meta) n1 <- c(25.9.194.25.105.320.33.261.133.232.1327.1990.214)
d1 <- c(5.1.23.1.4.53.3.12.6.2.156.145.8)n2 <- c(25.16.189.25.34.321.16.84.145.134.1320.2001.212)
d2 <- c(6.2.21.2.2.67.2.13.11.5.228.217.17)
bb <- data.frame(n1.d1.n2.d2)
res <- metabin(d1.n1.d2.n2.data=bb.sm=”RR”.meth=”I”)
res
plot(res. lwd=3)
![Page 21: Design and Analysis of Clinical Study 12. Meta-analysis Dr. Tuan V. Nguyen Garvan Institute of Medical Research Sydney, Australia](https://reader035.vdocuments.mx/reader035/viewer/2022081512/56649eef5503460f94bff094/html5/thumbnails/21.jpg)
Meta-analysis using R
> res RR 95%-CI %W(fixed) %W(random)1 0.8333 [0.2918; 2.3799] 1.26 1.262 0.8889 [0.0930; 8.4951] 0.27 0.273 1.0670 [0.6116; 1.8617] 4.47 4.474 0.5000 [0.0484; 5.1677] 0.25 0.255 0.6476 [0.1240; 3.3814] 0.51 0.516 0.7935 [0.5731; 1.0986] 13.08 13.087 0.7273 [0.1346; 3.9282] 0.49 0.498 0.2971 [0.1410; 0.6258] 2.49 2.499 0.5947 [0.2262; 1.5632] 1.48 1.4810 0.2310 [0.0454; 1.1744] 0.52 0.5211 0.6806 [0.5635; 0.8221] 38.81 38.8112 0.6719 [0.5496; 0.8214] 34.31 34.3113 0.4662 [0.2056; 1.0570] 2.07 2.07Number of trials combined: 13 RR 95%-CI z p.valueFixed effects model 0.6821 [0.6064; 0.7672] -6.3741 < 0.0001Random effects model 0.6821 [0.6064; 0.7672] -6.3741 < 0.0001Quantifying heterogeneity:tau^2 = 0; H = 1 [1; 1.45]; I^2 = 0% [0%; 52.6%]Test of heterogeneity: Q d.f. p.value 11 12 0.5292
![Page 22: Design and Analysis of Clinical Study 12. Meta-analysis Dr. Tuan V. Nguyen Garvan Institute of Medical Research Sydney, Australia](https://reader035.vdocuments.mx/reader035/viewer/2022081512/56649eef5503460f94bff094/html5/thumbnails/22.jpg)
Forest Plot
0.05 0.10 0.20 0.50 1.00 2.00 5.00 10.00Relative Risk
1
2
3
4
5
6
7
8
9
10
11
12
13
![Page 23: Design and Analysis of Clinical Study 12. Meta-analysis Dr. Tuan V. Nguyen Garvan Institute of Medical Research Sydney, Australia](https://reader035.vdocuments.mx/reader035/viewer/2022081512/56649eef5503460f94bff094/html5/thumbnails/23.jpg)
An Inverted Funnel Plot to Detect Publication Bias
![Page 24: Design and Analysis of Clinical Study 12. Meta-analysis Dr. Tuan V. Nguyen Garvan Institute of Medical Research Sydney, Australia](https://reader035.vdocuments.mx/reader035/viewer/2022081512/56649eef5503460f94bff094/html5/thumbnails/24.jpg)
An Inverted Funnel Plot to Detect Publication Bias
![Page 25: Design and Analysis of Clinical Study 12. Meta-analysis Dr. Tuan V. Nguyen Garvan Institute of Medical Research Sydney, Australia](https://reader035.vdocuments.mx/reader035/viewer/2022081512/56649eef5503460f94bff094/html5/thumbnails/25.jpg)
Heterogeneity
• Common, to be expected, not the exception • Should do test for homogeneity, but . . . interpret
heterogeneity cautiously in spirit of exploratory data analysis – Exploring sources of heterogeneity can lead to insights
about modification of apparent associations by various aspects of
– Study design
– Exposure measurements
– Study populations
![Page 26: Design and Analysis of Clinical Study 12. Meta-analysis Dr. Tuan V. Nguyen Garvan Institute of Medical Research Sydney, Australia](https://reader035.vdocuments.mx/reader035/viewer/2022081512/56649eef5503460f94bff094/html5/thumbnails/26.jpg)
Heterogeneity
• Relations discovered in process of exploring heterogeneity may be useful in planning & carrying out new studies
• Excluding outliers solely on basis of disagreement with other studies can lead to seriously biased summary estimates (avoid)
• Easier to interpret sources of heterogeneity when identified in advance of data analysis (not when suggested only by data)
![Page 27: Design and Analysis of Clinical Study 12. Meta-analysis Dr. Tuan V. Nguyen Garvan Institute of Medical Research Sydney, Australia](https://reader035.vdocuments.mx/reader035/viewer/2022081512/56649eef5503460f94bff094/html5/thumbnails/27.jpg)
Fixed & Random Effects
• Fixed effects models assume that an intervention has a single true effect
• Random effects models assume that an effect may vary across studies
![Page 28: Design and Analysis of Clinical Study 12. Meta-analysis Dr. Tuan V. Nguyen Garvan Institute of Medical Research Sydney, Australia](https://reader035.vdocuments.mx/reader035/viewer/2022081512/56649eef5503460f94bff094/html5/thumbnails/28.jpg)
Random Effects
• Assumes sample of studies randomly drawn from population of studies
• This is NOT typically true because: – All trials are included
– Trials are systematically (e.g., conveniently) sampled and
not randomly sampled
![Page 29: Design and Analysis of Clinical Study 12. Meta-analysis Dr. Tuan V. Nguyen Garvan Institute of Medical Research Sydney, Australia](https://reader035.vdocuments.mx/reader035/viewer/2022081512/56649eef5503460f94bff094/html5/thumbnails/29.jpg)
Random Effects
• Primary value of M-A is in search for predictors of between-study heterogeneity
• Random-effects summary is last resort only when
predictors or causes of between-study heterogeneity cannot be identified
• Random-effects can conceal fact that summary estimate or fitted model is poor summary of the data Sander Greenland.
Am J Epidemiol 1994;140;290-6.
![Page 30: Design and Analysis of Clinical Study 12. Meta-analysis Dr. Tuan V. Nguyen Garvan Institute of Medical Research Sydney, Australia](https://reader035.vdocuments.mx/reader035/viewer/2022081512/56649eef5503460f94bff094/html5/thumbnails/30.jpg)
Random Effects
• Sometimes needed, but more sensitive to publication bias than fixed-effects
• Random effects weights vary less across studies than fixed-effects weights
• W = 1/v versus w = 1/(v + t2) • Leads to reduced variation in weights • Thus smaller studies given larger relative weights when
random effects models used • Thus influenced more strongly by any tendency NOT to
publish small statistically insignificant studies biased estimate, spuriously strong associations
![Page 31: Design and Analysis of Clinical Study 12. Meta-analysis Dr. Tuan V. Nguyen Garvan Institute of Medical Research Sydney, Australia](https://reader035.vdocuments.mx/reader035/viewer/2022081512/56649eef5503460f94bff094/html5/thumbnails/31.jpg)
Random Effects
• Fixed effects weights vs. random effects weights • W = 1/v versus w = 1/(v + t2) • Identical when there is little or no between study variation • When differ, confidence intervals are larger for random-
effects than fixed effects • Smaller studies given larger relative weights in random
effects models & > influence • Conversely, influence of larger studies is less • May result in type II (beta error), e.g., Finding no
significant difference when one truly exists
![Page 32: Design and Analysis of Clinical Study 12. Meta-analysis Dr. Tuan V. Nguyen Garvan Institute of Medical Research Sydney, Australia](https://reader035.vdocuments.mx/reader035/viewer/2022081512/56649eef5503460f94bff094/html5/thumbnails/32.jpg)
Methodologic Choices & Their Implications in Dealing With Heterogeneous Data in a Meta-analysis
Lau J, Ioannidis JPA, Schmid CH. Quantitative Synthesis in Systematic Reviews. Annals of Internal Medicine 1997; 127:820-826.