meta-analysis principles and practice in cardiovascular research giuseppe biondi zoccai istituto di...
TRANSCRIPT
Meta-analysisMeta-analysisPrinciples and practice in Principles and practice in
cardiovascular researchcardiovascular research
Giuseppe Biondi ZoccaiGiuseppe Biondi Zoccai
Istituto di Cardiologia, Università di TorinoIstituto di Cardiologia, Università di Torino
DisclosureDisclosure
• No funding or conflict of interest to declare
IndexIndex
• Introduction & definitions
• Scientific hierarchy
• The Cochrane Collaboration
• Structured approach to systematic reviews
• Additional topics
• Statistical packages
• Further examples
Exponential increase in Exponential increase in PubMed citationsPubMed citations
0
2500
5000
7500
10000
12500
15000
-1975 1976-1980
1981-1985
1986-1990
1991-1995
1996-2000
2001-2005
systematic review systematic overview meta-analysis any
PubMed search strategy: ("2001"[PDAT] : "2005"[PDAT]) AND (("systematic"[title/abstract] AND "review"[title/abstract]) OR ("systematic"[title/abstract] AND "overview"[title/abstract]) OR ("meta-analysis"[title/abstract] OR "meta-analyses"[title/abstract]))
Famous quotesFamous quotes
“If I have seen further it is by standing on the shoulders of giants”
Isaac Newton
“The great advances in science usually result from new tools rather than from new doctrines”
Freeman Dyson
Famous quotesFamous quotes
“I like to think of the meta-analytic process as similar to being in a helicopter. On the ground individual trees are visible with high resolution. This resolution diminishes as the helicopter rises, and in its place we begin to see patterns not visible from the ground”
Ingram Olkin
Baby steps of meta-analysisBaby steps of meta-analysis• 1904 - Karl Pearson (UK): correlation between inoculation of
vaccine for typhoid fever and mortality across apparently conflicting studies
• 1931 – Leonard Tippet (UK): comparison of differences between and within farming techniques on agricultural yield adjusting for sample size across several studies
• 1937 – William Cochran (UK): combination of effect sizes across different studies of medical treatments
• 1970s – Robert Rosenthal and Gene Glass (USA), Archie Cochrane (UK): combination of effect sizes across different studies of, respectively, educational and psychological treatments
• 1980s – exponential development/use of meta-analytic methods
Minimal glossaryMinimal glossary• Review: viewpoint on a subject quoting different primary authors
• Overview: as above
• Qualitative review: deliberately avoids a systematic approach
• Systematic review: deliberately uses a systematic approach to study
search, selection, abstraction, appraisal and pooling
• Quantitative review: uses quantitative methods to appraise or synthesize
data
• Meta-analysis: uses specific statistical methods for data pooling and/or
exploratory analysis
• Individual patient data meta-analysis: uses specific stastistical
methods for data pooling or exploration exploiting individual patient data
→ Our goal: systematic review (systematic review (± ± meta-analysis)meta-analysis)
Qualitative reviewQualitative review
Systematic review and meta-analysesSystematic review and meta-analyses
• What is a systematic review?
– A systematic appraisal of the methodological quality,
clinical relevance and consistency of published
evidence on a specific clinical topic in order to provide
clear suggestions for a specific healthcare problem
• What is a meta-analysis?
– A quantitative synthesis that, preserving the identity of
individual studies, tries to provide an estimate of the
overall effect of an intervention, exposure, or diagnostic
strategy
Systematic reviewSystematic review
Systematic review and meta-analysesSystematic review and meta-analyses
Individual patient dataIndividual patient data meta-analysis meta-analysis
• Ideally should be a systematic review and meta-analysis based on individual patient data
• Major pros:– a unique database containing primary studies is
created and used (consistency checks and homogenous variables are created)
– the same analytical tools can be used across studies– subgroup analyses can be performed even for groups
that were not reported in the original publications• Major cons:
– some studies may have to be excluded (publication bias) because original authors may not provide source data
– poses major logistical and financial challenges
Individual patient data meta-analysisIndividual patient data meta-analysis
BMJ 2002
Systematic review and Systematic review and meta-regressionmeta-regression
• A meta-regression employs meta-analytic methods to explore the impact of covariates or moderators on the main effect measure or on other
• All the limitations of non-RCT studies applies, and thus they should mainly be regarded as hypothesis generating
Meta-regressionMeta-regression
Cumulative and prospective Cumulative and prospective meta-analysesmeta-analyses
• A cumulative meta-analysis recomputes and plots the pooled effect estimate every time a new study is added
• A prospective meta-analysis is based on a specific a priori protocol for its conduct, analysis, and reporting, and may use also a cumulative design
Cumulative meta-analysisCumulative meta-analysis
Antman et al, JAMA 1992
Cumulative meta-analysisCumulative meta-analysis
ProsPros• Systematic searches for clinical evidence
• Explicit and standardized methods for search and
selection of evidence sources
• Thorough appraisal of the internal validity of primary
studies
• Quantitative synthesis with increased statistical power
• Increased external validity by appraising the effect of
an intervention (exposure) across different settings
• Test subgroup hypotheses
• Explore clinical and statistical heterogeneit
Lau et al, Lancet 1998
ConsCons• “Exercise in mega-silliness”
• “Mixing apples with oranges”
• Not original research
• Big RCTs definitely better
• Pertinent studies might not be found, or may be of low
quality or internal validity
• Publication and small study bias
• Average effect largely unapplicable to individuals
Lau et al, Lancet 1998
ConsCons
Smith et al, BMJ 2003
Biondi-Zoccai
et al, BMJ 2006
IntroductionIntroduction
• Introduction & definitions
• Scientific hierarchy
• The Cochrane Collaboration
• Structured approach to systematic reviews
• Additional topics
• Statistical packages
• Further examples
EBM hierarchy of evidenceEBM hierarchy of evidence1.1. N of 1 randomized controlled trialN of 1 randomized controlled trial
2.2. Systematic reviews of homogeneous randomized trialsSystematic reviews of homogeneous randomized trials
3.3. Single (large) randomized trialSingle (large) randomized trial
4.4. Systematic review of homogeneous observational Systematic review of homogeneous observational studies addressing patient-important outcomesstudies addressing patient-important outcomes
5.5. Single observational study addressing patient-important Single observational study addressing patient-important outcomesoutcomes
6.6. Physiologic studies Physiologic studies (eg blood pressure, cardiac output, (eg blood pressure, cardiac output, exercise capacity, bone density, and so forth)exercise capacity, bone density, and so forth)
7.7. Unsystematic clinical observationsUnsystematic clinical observations
Guyatt and Rennie, Users’ guide to the medical literature, 2002
Parallel hierarchy of scientific Parallel hierarchy of scientific studies in cardiovascular medicinestudies in cardiovascular medicine
Biondi-Zoccai, Ital Heart J 2003
Qualitative reviews
Systematic reviews
Meta-analyses from individual studies
Meta-analyses from individual patient data
Case reports and series
Observational studies
Observational controlled studies
Randomized controlled trials
Multicenter randomized controlled trials
Benson et al,
NEJM 2000
IntroductionIntroduction
• Introduction & definitions
• Scientific hierarchy
• The Cochrane Collaboration
• Structured approach to systematic reviews
• Additional topics
• Statistical packages
• Further examples
The Cochrane CollaborationThe Cochrane Collaboration
Mission Statement:
The Cochrane Collaboration is an world-wide organisation that aims to help people make well informed decisions about healthcare by preparing, maintaining and promoting the accessibility of systematic reviews of the effects of healthcare interventions
The Cochrane CollaborationThe Cochrane Collaboration
• About 6000 contributors
• 50 Collaborative Review Groups (CRGs)
• 12 Centres throughout the world
• 9 Fields
• 11 Methods Groups
• 1 Consumer Network
• Campbell Collaboration
The Cochrane CollaborationThe Cochrane CollaborationObjectives:• Collaboration• Building on the enthusiasm of individuals• Avoiding duplication• Minimising bias• Keeping up to date• Striving for relevance• Promoting access• Ensuring quality• Continuity
The Cochrane CollaborationThe Cochrane Collaboration• Cochrane Database of Systematic Reviews (CDSR) –
contains Cochrane systematic reviews
• Database of Abstracts of Reviews of Effectiveness (DARE) – contains abstracts of non-Cochrane reviews
• Cochrane Central Controlled Trials Register (CENTRAL) – contains titles or abstracts of RCTs from multiple sources
• Cochrane Database of Methodology Reviews – contains Cochrane reviews of methods papers
• Cochrane Methodology Register (CMR) – contains abstracts of non-Cochrane methods papers
• Health Technology Assessment Database (HTA) – contains abstracts of HTA papers
• NHS Economic Evaluation Database (NHS EED) – contains abstracts of economic analysis papers
IntroductionIntroduction
• Introduction & definitions
• Scientific hierarchy
• The Cochrane Collaboration
• Structured approach to systematic reviews
• Additional topics
• Statistical packages
• Further examples
Algorithm for systematic reviewsAlgorithm for systematic reviews
• Definition of question and hypothetical solution
• Prospective design of the systematic review
• Problem formulation (population, intervention or
exposure, comparison, outcome [PICO])
• Data search
• Data abstraction and appraisal
• Data analysis ± quantitative synthesis
• Result interpretation and dissemination
Biondi-Zoccai et al, Ital Heart J 2004
FE
ED
-BA
CK
ON
HY
PO
TH
ES
ISF
EE
D-B
AC
K O
N H
YP
OT
HE
SIS
Definition of question and Definition of question and prospective designprospective design
• The clinical question should be clearly
stated, being as much explicit as possible
• The review should be designed in as much
details as possible, and yet with a limited a
priori knowledge of the subject
Biondi-Zoccai et al, Ital Heart J 2004
Problem formulation according Problem formulation according to the PICO approachto the PICO approach
• PPopulation of interest - eg elderly male >2 weeks after
myocardial infarction)
• IIntervention (or exposure) – eg intracoronary
infusion of progenitor blood cells
• CComparison – eg patients treated with progenitor cells vs
standard therapy
• OOutcome(s) – eg change in echocardiographic left ventricular
ejection fraction from discharge to 6-month control
Biondi-Zoccai et al, Ital Heart J 2004
Data searchData search
• After definition of question according to
PICO approach, the appropriate key-words
are used to search several databases
• Useful resources: BioMedCentral, CENTRAL,
clinicaltrials.gov, EMBASE, LILACS, and
PubMed
• Conference proceedings
• Cross-referencing (snowballing)
• Contact with experts
Study searchStudy search
Biondi-Zoccai et al, Int J Epidemiol 2005
A simple PubMed strategy for clinical studies on LM PCI:left AND main AND coronary AND stent* NOT case reports [pt] NOT review [pt] NOT editorial [pt]
A complex PubMed strategy for randomized clinical trials on invasive vs conservative strategies in ACS:(randomized controlled trial[pt] OR controlled clinical trial[pt] OR randomized controlled trials[mh] OR random allocation[mh] OR double-blind method[mh] OR single-blind method[mh] OR clinical trial[pt] OR clinical trials[mh] OR (clinical trial[tw] OR ((singl*[tw] OR doubl*[tw] OR trebl*[tw] OR tripl*[tw]) AND (mask*[tw] OR blind[tw])) OR (latin square[tw]) OR placebos[mh] OR placebo*[tw] OR random*[tw] OR research design[mh:noexp] OR comparative study[mh] OR evaluation studies[mh] OR follow-up studies[mh] OR prospective studies[mh] OR cross-over studies[mh] OR control*[tw] OR prospectiv*[tw] OR volunteer*[tw]) NOT (animal[mh] NOT human[mh]) NOT (comment[pt] OR editorial[pt] OR meta-analysis[pt] OR practice-guideline[pt] OR review[pt])) AND ((invasive OR conservative AND (coronary OR unstable angina OR acute coronary syndrome* OR unstable coronary syndrome* OR myocardial infarction)))
Study searchStudy search
Reveiz et al, J Clin Epidemiol 2006
Study selectionStudy selection
• 1st - screening of titles and abstracts
• 2nd – potentially pertinent citations are then
retrieved as full reports and appraised
according to prespecified and explicit
inclusion/exclusion criteria
• 3rd – studies fullfilling both inclusion and
exclusion criteria, are then included in the
systematic review
Andreotti et al,
Eur Heart J 2005
Data abstraction and appraisalData abstraction and appraisal
• Abstraction of outcomes and moderator
variables, possibly on prespecified data form
• Appraisal of the internal validity of primary
studies (eg the risk of selection, performance,
adjudication and attrition bias)
• Performed by single vs multiple reviewers, with
divergences resolved by consensus (possibly
after formal tests for agreement)
Data extractionData extraction
Buscemi et al, J Clin Epidemiol 2006
Internal validity of primary studiesInternal validity of primary studies
• Many scales for the quality of included studies have been reported, but none is reliable or robust
• The recommended approach is to individually appraise the potential risk of the 4 biases (eg A-low, B-moderate, C-high, D-unclear from reported data):
– Selection bias (one group is different than the other)
– Performance bias (treatment is systematically different)
– Adjudication bias (outcome adjudication is selectively
different)
– Attrition bias (follow-up duration or completeness is
different)
Quality scales are unreliableQuality scales are unreliable
Quality scales are unreliableQuality scales are unreliable
Internal validity of primary studiesInternal validity of primary studies
Hill et al, Eur Heart J 2004
Data synthesisData synthesis
• Quantitative data synthesis is central to
the practice of meta-analysis, and is based
on a major assumptio:
individual studies that are going to be
pooled are relatively homogeneous, both
clinically and statistically, to provide a
meaningful central tendency effect
estimate
Effect sizes and p valuesEffect sizes and p valuesForms of research findings suitable to meta-analysis:• Central tendency research:
– incidence or prevalence rates– mean (standard error)
• Pre-post contrasts:– changes in continuous or categorical variables
• Group contrasts:– experimentally created groups:
• comparison of outcomes between experimental and control groups
– naturally or non-experimentally occurring groups• treatment, prognostic or diagnostic features
• Association between variables:– correlation coefficients– regression coefficients
Effect sizes and p valuesEffect sizes and p values• The effect size makes meta-analysis possible:
– it is the “dependent variable”– it standardizes findings across studies such that they can be
directly compared
• Any standardized index can be an “effect size” as long as it meets the following:– is comparable across studies (generally requires
standardization)– represents the magnitude and direction of the relationship of
interest– is independent of sample size
• We identify as p values (for effect) the measures of alpha error for hypothesis testing
Relative risksRelative risks
• Relative risks (RR) are defined as the ratio of incidence rates, and are thus used for dichotomic variables)
• What is the meaning of RR:– RR=1 means no difference in risk– RR<1 means reduced risk in group 1 vs 2– RR>1 means increased risk in group 1 vs 2
• RRs are easier to interpret but are less userfriendly from a statistical point of view (RRAvsB≠1/RRBvsA) and may appear over-optimistic
Odds ratiosOdds ratios• Odds ratios (OR) are defined as the
ratio of the odds (P/[1-P]) and also used for dichotomic variables
• When prevalences are low, they are a good approximation of RR
• They behave similarly to RR (OR=1 means no difference in risk, …)
• ORs are less easy to interpret but more userfriendly from a statistical point of view (ORAvsB=1/ORBvsA), yet also overoptimistic
Risk differences and number Risk differences and number needed to treat/harmneeded to treat/harm
• The risk difference (RD), ie absolute risk difference, is the difference between the incidence of events in the experimental vs control groups
• The RD is theoretically the most clinically relevant statistics, but changes too much with disease prevalence
• The number to treat (NNT), defined as 1/RD, identifies the number of patients that we need to treat with the experimental therapy to avoid one event*
• The NNT is the most clinically meaningful parameter to express the impact of a treatment on a dichotomic outcome (eg death), but has the same limits of RD
*Numbers needed to harm (NNH) similarly express the number of patients that we have to treat with the experimental therapy to cause one adverse event
RR, OR or RD/NNT?RR, OR or RD/NNT?
OR RR RD/NNT
Communication - + ++
Consistency + ++ -
Mathematics ++ - -
Fixed vs random effectsFixed vs random effects• Statistical pooling may be based on:
– Fixed effect methods (eg Mantel-Haenszel or Peto), if we
can hypothesize studies were are estimating the same
population risk estimate (eg with RR, OR, or RD)
– Random effect methods (eg DerSimonian-Laird),
hypothesize individual studies are estimating different
treatment effects (eg with RR, OR, or RD)
• Additionally, inverse variance weighting (either based on
random or fixed effects) may be used to pool individual point
estimates with pertinent standard errors (even with HR)
Our adviceOur advice
• Both RR and OR can be your first choice statistics for uncommon events
• For common events, the OR is clearly less informative than the RR for the busy reader
• Complete your analyses by reporting RD and/or NNT for the sake of clarity
• Fixed effect methods are quite fine for homogeneous/ consistent data
• Random effect methods may be more appropriate for heterogeneous/inconsistent data, but often meta-regression (or even refraining from meta-analysis at all) might be the best option
Continous variablesContinous variables
• Continous variables can be pooled with
– Weighted mean differences (WMD), if the
same variable is used across studies
– Standardized mean differences (SMD), if
similar but not identical variables are used
– Inverse variance weighting, if only point
estimates and standard errors are available
Small study biasSmall study bias• Publication bias (eg the lower likelihood of
being published for studies with negative findings, or those originating in non-English speaking countries) may bias the results of a meta-analysis
• Other types of small study bias may undermine the validity of a meta-analysis
• A number of tests, analogical (eg the funnel plot) or analytical (eg Egger’s or Peter’s) have been proposed to appraise the likelihood of such small study bias
Peters et al, JAMA 2006
Statistical heterogeneityStatistical heterogeneity
• Statistical heterogeneity may be suspected
by inspecting tables (summary estimates/SE)
and forest plots, or analytically
• Chi-square, Breslow, or Cochran tests are
most commonly used
• While a 2-tailed p=0.05 is used for cut-off for
hypothesis testing of effect, a 2-tailed p=0.10
is conventionally chosen for heterogeneity
Statistical inconsistencyStatistical inconsistency
• Statistical inconsistency (I2) has been recently introduced to overcome the risk of alpha and beta error of standard tests for statistical heterogeneity
• It is computed as [(Q – df)/Q] x 100%, where Q is the chi-squared statistic and df is its degrees of freedom
• I2 values of 25% suggest low inconsistency, 50% moderate inconsistency, and 75% severe inconsistency
Higgins et al, BMJ 2003
IndexIndex
• Introduction & definitions
• Scientific hierarchy
• The Cochrane Collaboration
• Structured approach to systematic reviews
• Additional topics
• Statistical packages
• Further examples
QUOROMQUOROM
• xxx
QUADASQUADAS
Whithing et al, BMC Med Res Method 2003
Oxman and Guyatt indexOxman and Guyatt indexEvaluates the internal validity of a review on 9 separate questions for which 3
distinct anwers are eligible (“yes”, “partially/can’t tell”, “no”): 1. Where the search methods used to find evidence stated?2. Was the search for evidence reasonably comprehensive?3. Were the criteria for deciding which studies to include in the overview reported4. Was bias in the selection of studies avoided5. Were the criteria used for assessing the validity of the included studies reported?6. Was the validity of all studies referred to in the text assessed using appropriate
criteria7. Were the methods used to combine the findings of the relevant studies reported?8. Were the findings of the relevant studies combined appropriately relative to the
primary question the overview addresses?9. Were the conclusions made by the author(s) supported by the data and/or
analysis reported in the overview?Question 10 summarizes the previous ones and, specifically, asks to rate the
scientific quality of the review from 1 (being extensively flawed) to 3 (carrying major flaws) to 5 (carrying minor flaws) to 7 (minimally flawed). The developers of the index specify that if the “partially/can’t tell” answer is used one or more times in questions 2, 4, 6, or 8, a review is likely to have minor flaws at best and is difficult to rule out major flaws (ie a score≤4). If the “no” option is used on question 2, 4, 6 or 8, the review is likely to have major flaws (ie a score≤3).
Oxman et al, J Clin Epidemiol 1991
IndexIndex
• Introduction & definitions
• Scientific hierarchy
• The Cochrane Collaboration
• Structured approach to systematic reviews
• Additional topics
• Statistical packages
• Further examples
Statistical packagesStatistical packages
• EasyMA (http://www.spc.univ-lyon1.fr/easyma.net/)
• RevMan (http://www.cochrane.org)
» For meta-analyses of medical interventions
• Meta-Test ([email protected])
• Meta-DiSc (http://www.hrc.es/investigacion/metadisc.html)
» For meta-analyses of diagnostic tests
• FastPro• NCSS• SAS• SPSS• Stata• WEasyMA
FR
EE
WA
RE
S!
Not
for
fre
e
Revman Revman Review: Late percutaneous coronary intervention for infarct-related artery occlusionComparison: 01 Late percutaneous coronary intervention vs best medical therapy for infarct-related artery occlusion Outcome: 01 Death
Study PCI Medical Rx OR (random) OR (random)or sub-category n/N n/N 95% CI 95% CI O - E Variance
TOPS 0/42 0/45 Not estimable 0.00 0.00 TOMIIS 1/25 1/19 0.75 [0.04, 12.82] 0.00 2.10 Horie 1/44 5/39 0.16 [0.02, 1.42] 0.00 1.25 TOAT 2/32 1/34 2.20 [0.19, 25.52] 0.00 1.56 Zeymer et al 6/145 17/151 0.34 [0.13, 0.89] 0.00 0.24 DECOPI 6/109 7/103 0.80 [0.26, 2.46] 0.00 0.33 BRAVE-2 4/182 8/183 0.49 [0.15, 1.66] 0.00 0.39 Silva et al 0/18 2/18 0.18 [0.01, 3.99] 0.00 2.51
Total (95% CI) 597 592 0.48 [0.28, 0.85]Total events: 20 (PCI), 41 (Medical Rx)Test for heterogeneity: Chi² = 4.25, df = 6 (P = 0.64), I² = 0%Test for overall effect: Z = 2.53 (P = 0.01)
0.1 0.2 0.5 1 2 5 10
Favours PCI Favours medical Rx
Review: Late percutaneous coronary intervention for infarct-related artery occlusionComparison: 01 Late percutaneous coronary intervention vs best medical therapy for infarct-related artery occlusion Outcome: 01 Death
Study PCI Medical Rx OR (fixed) OR (fixed)or sub-category n/N n/N 95% CI 95% CI O - E Variance
TOPS 0/42 0/45 Not estimable 0.00 0.00 TOMIIS 1/25 1/19 0.75 [0.04, 12.82] 0.00 2.10 Horie 1/44 5/39 0.16 [0.02, 1.42] 0.00 1.25 TOAT 2/32 1/34 2.20 [0.19, 25.52] 0.00 1.56 Zeymer et al 6/145 17/151 0.34 [0.13, 0.89] 0.00 0.24 DECOPI 6/109 7/103 0.80 [0.26, 2.46] 0.00 0.33 BRAVE-2 4/182 8/183 0.49 [0.15, 1.66] 0.00 0.39 Silva et al 0/18 2/18 0.18 [0.01, 3.99] 0.00 2.51
Total (95% CI) 597 592 0.47 [0.27, 0.80]Total events: 20 (PCI), 41 (Medical Rx)Test for heterogeneity: Chi² = 4.25, df = 6 (P = 0.64), I² = 0%Test for overall effect: Z = 2.75 (P = 0.006)
0.1 0.2 0.5 1 2 5 10
Favours PCI Favours medical Rx
Revman Revman Review: Late percutaneous coronary intervention for infarct-related artery occlusionComparison: 01 Late percutaneous coronary intervention vs best medical therapy for infarct-related artery occlusion Outcome: 01 Death
Study PCI Medical Rx RR (fixed) RR (fixed)or sub-category n/N n/N 95% CI 95% CI O - E Variance
TOPS 0/42 0/45 Not estimable 0.00 0.00 TOMIIS 1/25 1/19 0.76 [0.05, 11.39] 0.00 1.91 Horie 1/44 5/39 0.18 [0.02, 1.45] 0.00 1.15 TOAT 2/32 1/34 2.13 [0.20, 22.31] 0.00 1.44 Zeymer et al 6/145 17/151 0.37 [0.15, 0.91] 0.00 0.21 DECOPI 6/109 7/103 0.81 [0.28, 2.33] 0.00 0.29 BRAVE-2 4/182 8/183 0.50 [0.15, 1.64] 0.00 0.36 Silva et al 0/18 2/18 0.20 [0.01, 3.89] 0.00 2.29
Total (95% CI) 597 592 0.49 [0.29, 0.82]Total events: 20 (PCI), 41 (Medical Rx)Test for heterogeneity: Chi² = 4.11, df = 6 (P = 0.66), I² = 0%Test for overall effect: Z = 2.73 (P = 0.006)
0.1 0.2 0.5 1 2 5 10
Favours PCI Favours medical Rx
Review: Late percutaneous coronary intervention for infarct-related artery occlusionComparison: 01 Late percutaneous coronary intervention vs best medical therapy for infarct-related artery occlusion Outcome: 01 Death
Study PCI Medical Rx RD (fixed) RD (fixed)or sub-category n/N n/N 95% CI 95% CI O - E Variance
TOPS 0/42 0/45 0.00 [-0.04, 0.04] 0.00 0.00 TOMIIS 1/25 1/19 -0.01 [-0.14, 0.11] 0.00 0.00 Horie 1/44 5/39 -0.11 [-0.22, 0.01] 0.00 0.00 TOAT 2/32 1/34 0.03 [-0.07, 0.13] 0.00 0.00 Zeymer et al 6/145 17/151 -0.07 [-0.13, -0.01] 0.00 0.00 DECOPI 6/109 7/103 -0.01 [-0.08, 0.05] 0.00 0.00 BRAVE-2 4/182 8/183 -0.02 [-0.06, 0.01] 0.00 0.00 Silva et al 0/18 2/18 -0.11 [-0.28, 0.06] 0.00 0.01
Total (95% CI) 597 592 -0.04 [-0.06, -0.01]Total events: 20 (PCI), 41 (Medical Rx)Test for heterogeneity: Chi² = 9.12, df = 7 (P = 0.24), I² = 23.2%Test for overall effect: Z = 2.80 (P = 0.005)
-0.5 -0.25 0 0.25 0.5
Favours PCI Favours medical Rx
Funnel plotFunnel plot
Review: Late percutaneous coronary intervention for infarct-related artery occlusionComparison: 01 Late percutaneous coronary intervention vs best medical therapy for infarct-related artery occlusion Outcome: 01 Death
0.1 0.2 0.5 1 2 5 10
0.0
0.4
0.8
1.2
1.6
SE(log OR)
OR (fixed)
IndexIndex
• Introduction & definitions
• Scientific hierarchy
• The Cochrane Collaboration
• Structured approach to systematic reviews
• Additional topics
• Statistical packages
• Further examples
Meta-analysis of intervention studiesMeta-analysis of intervention studies
Meta-analysis of intervention studiesMeta-analysis of intervention studies
Meta-analysis of intervention studiesMeta-analysis of intervention studies
Meta-analysis of intervention studiesMeta-analysis of intervention studies
Meta-analysis of intervention studiesMeta-analysis of intervention studies
Meta-analysis of prognostic studiesMeta-analysis of prognostic studies
Troponin
Meta-analysis of diagnostic studiesMeta-analysis of diagnostic studies
Louvard et al, JACC 2006
Take home messageTake home message• The validity of a meta-analysis refers to the
soundness of the original studies and the
procedures used to combine them
• Several dozens of potential validity leaks have
been identified in these procedures
• Given possible weaknesses, the enterprise may
seem hopeless, yet it’s no worse than that of a
pilot reading a preflight checklist and testing
against possibly disastrous conditions
Take home messageTake home message
• However, the checking and testing makes it possible for airplanes to fly even long distances with only minimal risk of equipment failure
• Systematic reviews and meta-analyses, similarly, succeed when researchers enforce sound validity checklists
A few referencesA few references• Biondi-Zoccai GGL et al. Parallel hierarchy of scientific studies in cardiovascular medicine. Ital Heart J 2003; 4: 819-20• Biondi-Zoccai GGL et al. Compliance with QUOROM and quality of reporting of overlapping meta-analyses on the role of
acetylcysteine in the prevention of contrast associated nephropathy: case study. BMJ 2006;332:202-209• Biondi-Zoccai GGL et al. A practical algorithm for systematic reviews in cardiovascular medicine. Ital Heart J 2004;5:486 -7• Bucher HC et al. The results of direct and indirect treatment comparisons in meta-analysis of randomized controlled trials. J
Clin Epidemiol 1997;50:683– 9• Cappelleri JC et al. Large trials vs meta-analysis of smaller trials: how do their results compare? JAMA 1996; 276: 1332-8• Clarke M et al, eds. Cochrane reviewers’ handbook 4.2.0. (www.cochrane.org/resources/handbook/handbook.pdf)• Cooper H et al, eds. The handbook of research synthesis. New York, NY: Russell Sage Foundation, 1994• Cucherat M et al. EasyMA: a program for the meta-analysis of clinical trials. Comput Methods Programs Biomed
1997;53:187- 90• Egger M et al, eds. Systematic reviews in health care: meta-analysis in context. 2nd ed. London: BMJ Publishing Group,
2001• Glass G. Primary, secondary and meta-analysis of research. Educ Res 1976;5:3-8• Glasziou P et al. Systematic reviews in health care. A practical guide. Cambridge: Cambridge University Press, 2001• Guyatt G et al, eds. Users’ guides to the medical literature. A manual for evidence-based clinical practice. Chicago, IL: AMA
Press, 2002• Higgins JPT et al. Measuring inconsistency in meta-analyses. BMJ 2003;327:557 – 60• Lau J et al. Summing up evidence: one answer is not always enough. Lancet 1998;351:123 -7• Moher D et al. Improving the quality of reports of meta-analyses of randomised controlled trials: the QUORUM statement.
Lancet 1999; 354: 1896-900• Petitti DB. Meta-analysis, decision analysis, and cost-effectiveness analysis: methods for quantitative synthesis in medicine.
New York, NY: Oxford University Press, 2000• Song F et al. Validity of indirect comparison for estimating efficacy of competing interventions: empirical evidence from published meta-analysis. BMJ 2003;326:472• Thompson SG et al. How should meta-regression analyses undertaken and interpreted? Stat Med 2002;21:1559-73