systematic review: analytical methods of meta-analysis stephen bent, m.d. assistant professor of...
TRANSCRIPT
Systematic Review:Analytical Methods of
Meta-analysis
Stephen Bent, M.D.Assistant Professor of Medicine, Epidemiology
and BiostatisticsUCSF
8 Steps to Systematic Review
1. Research Question 2. Protocol 3. Search 4. Study selection (inclusion/exclusion) 5. Quality assessment 6. Data abstraction 7. Analysis
– A) Create summary measure– B) Assess for heterogeneity– C) Assess for publication bias– D) Conduct sensitivity/subgroup analyses– E) Advanced issues/techniques
8. Interpretation
Juni et al, Hazards of scoring the quality of
clinical trials. JAMA. 1999;282:1054-60.
Why conduct a systematic review?
The best way to summarize evidence on a scientific topic
Concisely communicates findings to others in the field
Identifies author(s) as expertsIdentifies areas for future studyPerfect for background of grantsDon’t need to do primary data collection (so
can be done while waiting for data from other projects)
You have to do the work anyway, so might as well get a publication!
You can effect change in clinical management
Cumulative Meta-analysis
Antman EM et al: JAMA. 1992;268:240-248
Systematic Review: Clinical Implications (Antiarrhythmic Drugs for
Acute MI)
Teo KK et al. JAMA. 1993;270:1589-1595
Sample Systematic Reviews Kangelaris KN, Bent S, Nussbaum RL, Garcia DA, Tice JA.
Genetic testing before anticoagulation? A systematic review of the safety and efficacy of pharmacogenetic dosing of warfarin. Journal of General Internal Medicine (in press).
Nguyen SP, Bent S, Chen Y, Terdiman JP. Gender as a Risk Factor for Advanced Neoplasia and Colorectal Cancer: A Systematic Review and Meta-analysis. Clinical Gastroenterology. 2009;7:676-81.
Simon J, Chen Y, Bent S. The relation of alpha-linoleic acid to the risk of prostate cancer: a systematic review. Am J Clin Nutr. 2009;89:1-7S.
Li J, Winston LG, Moore DH, Bent S. Efficacy of short-course antibiotic regimens for community-acquired pneumonia: a meta-analysis. American Journal of Medicine. 2007;120(9):783-90.
Margaretten M, Kohlwes J, Moore D, Bent S. The rational clinical examination: does this patient have septic arthritis. JAMA. 2007;297:1478-1488.
Sample Systematic Reviews
Hsu J, Kohlwes J, Bent S. Efficacy of antifungal therapy in chronic rhinosinusitis: A systematic review. J Allergy Clin
Immunol. 2010 125:2
Guarnieri M, Bent S. Death from coronary artery disease in patients with systemic lupus erythematosus: a systematic
review and meta-analysis of mortality cohort studies. (submitted to Arthritis Care and Research 1/2012).
Lee JK, Liles EG, Bent S, Levin TR, Corley DA. Diagnostic Accuracy of Fecal Immunochemical Tests for Colorectal
Cancer: Systematic Review and Meta-analysis (submitted to JAMA 4/2013).
8 Steps of Systematic Review
1. Research Question 2. Protocol 3. Search 4. Study selection (inclusion/exclusion) 5. Quality assessment 6. Data abstraction 7. Analysis
– A) Create summary measure– B) Assess for heterogeneity– C) Assess for publication bias– D) Conduct sensitivity/subgroup analyses– E) Advanced issues/techniques
8. Interpretation
Create a Summary Measure
Before we get to the mechanics of a summary measure….
Be sure to provide your audience with a concise summary table
A “visual meta-analysis”Readers should be able to examine
Table 1 and reach their own conclusions about the data
Example
Antibiotics for acute bronchitis.After search and application of
inclusion/exclusion criteria, 8 studies were included.
RCTs in Acute Bronchitis
Study, yr N Abx Outcome Result*Stott, 76 207 Doxy Days of Yellow Spit 0.6 (-0.2 to 1.4)
Franks, 84 54 TMP/S Cough Amount Score 0.2 (-0.2 to 0.6)
Williamson, 84 69 Doxy Days of Purulent Sputum -0.2 (-1.2 to 0.8)
Dunlay, 87 45 Erythro Sputum production score 0.5 (0.1 to 0.9)
Scherl, 87 31 Doxy Days of sputum 1.9 (-0.2 to 4.0)
Verheij, 94 140 Doxy Days of productive cough 0.5 (-0.4 to 1.4)
Hueston, 94 23 Erythro Days of productive cough -0.4 (-2.4 to 1.6)
King, 96 91 Erythro Days of sputum production 0.7 (-1.3 to 2.7)
* Positive numbers indicate antibiotics are superior to placebo
How do you create a summary measure?
Clinical example: 5 year old girl presents with ear pain and is found to have an acute otitis media.
Should she get antibiotics?
Research Questions:1.In children with OM, are antibiotics
effective for pain relief?2.In children with OM, do antibiotics
reduce the rate of complications (mastoiditis, hearing problems)?
3 studies are identified (examining effect of Abx on Pain)
Study 1: N = 100RR=1.41Study 2: N=200 RR=0.98Study 3: N=300 RR=1.01
You could take the average effect: (1.41 + 0.98 + 1.01) / 3 = 1.13
Is this a good summary measure?
Summary measure weighted by sample size
Provide “weight” for studies based on their sample size
600Total1.013003
0.9820021.411001RRNStudy
summary effect estimate= Σ (Ni x effect estimatei) = 640 =1.07 Σ(Ni) 600
More refined: Provide “weight” by using inverse of
variance
Summary = Σ (weighti x effect estimatei) = 30.5 = 1.00effect estimate Σ(weighti) 30.3
Study N RR Var RR Weight
1 100 1.41 3.0 0.33
2 200 0.98 0.1 10
3 300 1.01 0.05 20
Total 700
Does the largest study always have the lowest variance and
therefore the greatest “weight”?
Dichotomous outcomes
Continuous outcomes
Confidence Intervals Around Summary Effect
Calculate variance of summary effect estimate, or the 95% CI around the summary estimate
Variance of summary estimate = 1 Σ(weightsi)
Variance of summary estimate = _1_ = .03 30.5
95% CI = + 1.96 √0.03 = + 0.34
Summary OR and 95% CI = 1.00 (0.65 - 1.33)
Type of Model?
Variance RRs = 1/wiVariance RRs = 1/wi
Weighti = 1
variance RRi + D
Weighti = 1
variance RRi
Variance of individual studies + variance of differences between studies
Weights: variance of individual studies
Existing studies are a random sample
Existing studies are the entire population
Goal: estimate the “true” effect
Goal: weighted average of risk from existing studies
Random EffectsFixed Effects
Formulas for D
Fixed Effects
Model:
Random Effects
Model:
Summary RRb
Summary RRa
Summary RRb
Summary RRa
Random VS. Fixed Effects Model Practical Implications of the Choice
Confidence intervals: RE model produces wider confidence intervals
Statistical significance: less likely with RE model
BOTTOM LINE: If the individual study findings are similar, the model
makes little difference in estimate or statistical significance.
If the individual study findings are heterogeneous, the model can affect the statistical significance.
Mantel-Haenszel Method (Fixed Effects Model)
Diseased Not diseasedTreated (exposed) ai ci
Not treated (unexposed) bi di
ORi = ai/ ci = ai x di lnORmh = Σ (wi x lnORi )
bi/ di bix ci Σwi
variance lnORi = 1 + 1 + 1+ 1 variance ORmh = 1 ai bi ci di Σ wi
weighti = (wi) = 1 variance lnORi
95% CI = elnORmh (1.96 x √variance lnORmh)
Randomized Trials of Antibiotic Rx for acute OM to prevent TM
perforationStudy 1 Perforation No PerforationAntibiotic 1 114Placebo 3 116
Study 2 Perforation No PerforationAntibiotic 7 65Placebo 12 65
1. Calculate OR and lnOR for each study:OR1= 1 x 116 = 0.34 lnOR1 = -1.08
3 x 114
OR2 = 7 x 65 = 0.58 lnOR2 = -0.54 12 x 65
Randomized Trials of Antibiotic Rx for acute OM to prevent TM
perforation
2. Calculate variance lnORi for each study:
Var ln OR1 = 1 + 1 + 1 + 1 = 1.35 1 3 114 116
Var ln OR2 = 1 + 1 + 1 + 1 = 0.26 7 12 65 65
3. Calculate wi for each study:
w1 = 1 = 0.74 1.35
w2 = 1 = 3.85
0.26
Study 1 Perforation No PerforationAntibiotic 1 114Placebo 3 116
Study 2 Perforation No PerforationAntibiotic 7 65Placebo 12 65
4.Calculate the wi x ln ORi for each study: w1 x lnOR1 = 0.74 x -1.08 = -0.80
w2 x lnOR2= 3.85 x -0.54 = -2.08
Randomized Trials of Antibiotic Rx for acute OM to prevent TM
perforation
5. Calculate the sum of the wi
w1 + w2 = 0.74 + 3.85 = 4.59
6. Summary lnORmh = Σ (wi x lnORi) = -0.80 + -2.08 = -0.63
Σ wi 4.59= ORmh = 0.53
7. Calculate variance ORmh = 1 = 1 = 0.22
Σ wi 4.59
8. Calculate 95% CI = elnORmh + (1.96 x √ variance lnORmh)
= e-.63 + (1.96 x √ 0.22) = 0.21 - 1.34
Summary OR = 0.53 (95% CI 0.21 – 1.34)
Randomized Trials of Antibiotic Rx for acute OM to prevent TM
perforation
Dersimonian and Laird Method (Random Effects Model)
Similar formula to Mantel-Haenszel:ln ORdl = Σ (wi x ln ORi) wi = 1
Σwi variancei + D
Where D gets larger as the OR (or effect estimate) of the individual studies vary from the summary estimate
But…All you need to know is:
When combined, individual study effect estimates are weighted by their inverse variance
Variance is related to sample size AND # of events (dichotomous) and precision (continuous)
Fixed effects just combines all weighted estimates, while random effects “penalizes” estimates for variation between studies
8 Steps to Systematic Review
1. Research Question 2. Protocol 3. Search 4. Study selection (inclusion/exclusion) 5. Quality assessment 6. Data abstraction 7. Analysis
– A) Create summary measure– B) Assess for heterogeneity– C) Assess for publication bias– D) Conduct sensitivity/subgroup analyses– E) Advanced issues/techniques
8. Interpretation
HeterogeneityAre you comparing apples and oranges?Clinical heterogeneity: are studies asking same
question?Statistical heterogeneity: is the variation likely
to have occurred by chance?
Measures how far each individual OR/RR is from the summary OR/RR.
Studies whose OR/RRs are very different from the summary OR/RRs contribute greatly to the heterogeneity, especially if they are weighted heavily.
Heterogeneity
Refers to the degree that the study results differ
Visual ApproachStatistical Approach
Q = sum [weighti x (ESs – ESi)]
p < 0.05 indicates heterogeneity
Summary RR = 0.93 (0.87-0.99)
Problem of Heterogeneity
Study findings are different – should they be combined?
Study OR1 0.012 1.03 10.0
Study OR1 0.352 0.563 0.974 1.155 1.756 1.95
Statistical tests of Heterogeneity
Is the variation in the individual study findings likely due to chance?
Ho: Effect estimate in each study is the same (or homogeneous)
Ha: Effect estimate in each study is not the same (or heterogeneous)
Q = Σ(wi x (ln ORmh – ln ORi )2) df = (N studies -1)
p < 0.05 or 0.10 = reject null, i.e., studies are heterogeneous
Heterogeneity – Interpret Findings (Example: RR of Colon CA, Men vs.
Women)
8 Steps to Sytematic Review
1. Research Question 2. Protocol 3. Search 4. Study selection (inclusion/exclusion) 5. Quality assessment 6. Data abstraction 7. Analysis
– A) Create summary measure– B) Assess for heterogeneity– C) Assess for publication bias– D) Conduct sensitivity/subgroup analyses– E) Advanced issues/techniques
8. Interpretation
Assessing for Publication Bias
Publication Bias – the publication or “non-publication” of research findings, depending on the nature and direction of the results.
Rosenthal, 1979 – published an article describing the “file-drawer problem” that journals publish only 5% of all negative studies, while the file drawers in the back of the lab contain the other 95%.
Methods for Assessing Publication Bias
Funnel plots – simple scatter plots of treatment effects (horizontal axis) vs. some measure of study size (vertical axis).
Choice of axes– Log scale for treatment effects (to ensure that
treatment effects in opposite directions are the same distance from 1.0 – e.g., 0.5 and 2.0)
– Standard error for measure of sample size• Power depends on both sample size and #
events• Standard error is consistent with the statistical
tests
Funnel Plot of Log Relative Risk vs Standard Error
Log Relative Risk
Sta
nd
ard
err
or
5
4
3
2
1
0
Example: ALA and Prostate Cancer Risk
RR=1.2 (1.01 to 1.43), Test for heterogeneity, p=0.00
ALA – Funnel Plot
Funnel Plot with Imputed Values for Publication Bias
RR=0.94, 95% CI: 0.79-1.17
Publication bias caveatFunnel plot asymmetry does not always
indicate bias– It is possible that smaller studies enrolled
higher risk patients, for example, and therefore found a greater effect.
– Small studies are often conducted before larger studies. In the intervening years, other interventions may have improved, thus reducing the relative efficacy of the treatment.
Statistical methods to assess publication bias
Examine associations between study size and treatment effect.– Sensitivity is poor when < 20 studies
Begg’s test: an adjusted rank correlation
Egger’s test: a weighted regression of effect size vs. standard error.– Basically asks if the regression line has a non-zero
slope– More sensitive than Begg’s test, but more false
positives, especially when 1) large treatment effects, 2) few events per trial, 3) all trials of similar size. (In these cases, one may decide a priori to use Begg’s test).
Begg's funnel plot with pseudo 95% confidence limits
RR
s.e. of: RR0 .2 .4 .6 .8
-1
0
1
2
Begg's Test adj. Kendall's Score (P-Q) = -30 Std. Dev. of Score = 14.58 Number of Studies = 12 z = -2.06 Pr > |z| = 0.040 z = 1.99 (continuity corrected) Pr > |z| = 0.047 (continuity corrected)
Egger's test------------------------------------------------------------------------------ Std_Eff | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- slope | .9810716 .1103858 8.89 0.000 .7351168 1.227026 bias | -.9911295 .3236382 -3.06 0.012 -1.71224 -.2700187------------------------------------------------------------------------------
8 Steps to Systematic Review
1. Research Question 2. Protocol 3. Search 4. Study selection (inclusion/exclusion) 5. Quality assessment 6. Data abstraction 7. Analysis
– A) Create summary measure– B) Assess for heterogeneity– C) Assess for publication bias– D) Conduct sensitivity/subgroup analyses– E) Advanced issues/techniques
8. Interpretation
Subgroup & Sensitivity Analysis
Subgroup Analysis – MA of a subgroup of eligible studies
age
ethnicity
risk factors
treatment
Sensitivity Analysis – add or delete questionable studies
eligibility
treatment
Subgroup Analysis
OR 95% CI N Ever user
Of estrogen:
All eligible studies
Cohort studies
Case-Control studies
2.3*
1.7*
2.4*
2.1 - 2.5
1.3 - 2.1
2.2 - 2.6
29
4
25
Dose of
estrogen:
0.3 mg
0.625 mg
1.25 mg
3.9
3.4
5.8
1.6 - 9.5
2.0 - 5.6
4.5 - 7.5
3
4
9
Duration of
use:
< 1 year
1-5 years
5-10 years
10 years
1.4
2.8
5.9
9.5*
1.0 - 1.8
2.3 - 3.5
4.7 - 7.5
7.4 - 12.3
9
12
10
10
Regimen: Cyclic
Daily
3.0*
2.9*
2.4 - 3.8
2.2 - 3.8
8
8
* p for heterogeneity < 0.05
Analytical Methods: Summary Points
Always start the meta-analysis with a “visual meta-analysis” (i.e., a great table 1). – A clinician should be able to interpret the results
Step 1: Calculate a summary measure = “weighted mean effect estimate”– You can combine anything, but use judgment
Step 2: Assess for heterogeneity– Heterogeneity is not always a problem
Step 3: Assess for publication bias– Both visual and statistical methods
Step 4: Perform subgroup/sensitivity analyses– Ideally specify these a priori
8 Steps to Systematic Review
1. Research Question 2. Protocol 3. Search 4. Study selection (inclusion/exclusion) 5. Quality assessment 6. Data abstraction 7. Analysis
– A) Create summary measure– B) Assess for heterogeneity– C) Assess for publication bias– D) Conduct sensitivity/subgroup analyses– E) Advanced issues/techniques
8. Interpretation
Can you conduct a systematic review when there are only a few studies?
Parachute use to prevent death and major trauma related to gravitational challenge: systematic review of
randomised controlled trials Objectives To determine whether parachutes are effective in preventing
major trauma related to gravitational challenge. Design Systematic review of randomised controlled trials. Data sources: Medline,Web of Science, Embase, and the Cochrane
Library databases; appropriate internet sites and citation lists. Study selection: Studies showing the effects of using a parachute during
free fall. Main outcome measure Death or major trauma. Results We were unable to identify any randomised controlled trials of
parachute intervention. Conclusions As with many interventions intended to prevent ill health,
the effectiveness of parachutes has not been subjected to rigorous evaluation by using randomised controlled trials. Advocates of evidence based medicine have criticised the adoption of interventions evaluated by using only observational data. We think that everyone might benefit if the most radical protagonists of evidence based medicine organised and participated in a double blind, randomised, placebo controlled, crossover trial of the parachute.
Smith GCS and Pill JP. BMJ 2003;327:1459–61
Advanced Topics
Individual participant dataMissing dataDifferent types of dataObservational studiesGeneralized synthesis of evidenceMeta-regressionCritique of a systematic review
Different types of data
Different scales (example)Ordinal dataBinary dataContinuous outcomesDiagnostic tests (example)
RCTs in Acute Bronchitis: Different Scales
Study, Study, yryr
NN AbxAbx OutcomeOutcome ResultResult
Stott, 76Stott, 76 202077
DoxyDoxy Days of Yellow SpitDays of Yellow Spit 0.6 (-0.2 to 0.6 (-0.2 to 1.4)1.4)
Franks, 84Franks, 84 5454 TMP/TMP/SS
Cough Amount ScoreCough Amount Score 0.2 (-0.2 to 0.2 (-0.2 to 0.6)0.6)
Williamson, Williamson, 8484
6969 DoxyDoxy Days of Purulent Days of Purulent SputumSputum
-0.2 (-1.2 to -0.2 (-1.2 to 0.8)0.8)
Dunlay, 87Dunlay, 87 4545 ErythErythroro
Sputum production Sputum production scorescore
0.5 (0.1 to 0.5 (0.1 to 0.9)0.9)
Scherl, 87Scherl, 87 3131 DoxyDoxy Days of sputumDays of sputum 1.9 (-0.2 to 1.9 (-0.2 to 4.0)4.0)
Verheij, 94Verheij, 94 141400
DoxyDoxy Days of productive Days of productive coughcough
0.5 (-0.4 to 0.5 (-0.4 to 1.4)1.4)
Hueston, 94Hueston, 94 2323 ErythErythroro
Days of productive Days of productive coughcough
-0.4 (-2.4 to -0.4 (-2.4 to 1.6)1.6)
King, 96King, 96 9191 ErythErythroro
Days of sputum Days of sputum productionproduction
0.7 (-1.3 to 0.7 (-1.3 to 2.7)2.7)
Problem
How do you combine studies with slightly different outcomes?
Option 1: - don’t do itOption 2: Transform all outcomes
to an effect size
What is an Effect Size?
Effect size – a way of expressing results in a common metric
Units – standard deviation
Effect Size
ES = X1 – X2
SDpooled
1. ES increases as difference between means increases
2. ES increases as SD decreases
3. ES is expressed in units of SD
4. Summary ES combines the weighted ES from each study.
Effect Size
Effect Size
Rough Estimates– SMALL 0.2
– MEDIUM 0.5
– LARGE >0.7
Context– Mean Duration of Cough = 8 days
– Standard Deviation = 3 days
Main Result
Summary ES = 0.21 (95% CI 0.05 to 0.36)
Summary Mean Differences
Outcome MeasureOutcome Measure Summary Mean Summary Mean Difference (95% CI)Difference (95% CI)
Days of Productive Days of Productive Cough (6 studies)Cough (6 studies)
0.4 days (-0.1 to 0.8)0.4 days (-0.1 to 0.8)
Days of cough (4 Days of cough (4 studies)studies)
0.5 days (-0.1 to 1.1)0.5 days (-0.1 to 1.1)
Time off work (6 Time off work (6 studies)studies)
0.3 days (-0.6 to 1.1)0.3 days (-0.6 to 1.1)
Different Types of Data: Diagnostic Tests
Sensitivity and Specificity
Sensitivity TP/(TP + FN)Positive in Disease
SpecificityTN/(TN + FP)Negative in Health
TNFNTest
-
FPTPTest
+
Disease
-
Disease
+
(+) Likelihood Ratio = Sensitivity
1-Specificity
(-) Likelihood Ratio = 1-Sensitivity1-Sensitivity Specificity Specificity
Does this patient have a specific disease?
What we thought before (pre-test probability)
+ Clinical information (diagnostic test, LR)
= What we think after (post-test probability)
Diagnostic OR = +LR/-LR
= TP x TN / FP x FNSensitivity Specificity Pos LR Neg LR Diag OR
0.5 0.5 1 1 10.6 0.6 1.5 0.67 2.30.7 0.7 2.3 0.43 5.40.8 0.8 4 0.25 160.9 0.9 9 0.11 810.95 0.95 19 0.05 3610.99 0.99 99 0.01 9801
Example: US and CT for Appendicitis
Goal: to determine whether US or CT is a “better” test for the evaluation of suspected appendicitis.
Diagnostic tests are complicated because there are 5 potential outcomes to summarize– LR+, LR-– Sensitivity, Specificity– Diagnostic OR– Assess heterogeneity, publication bias for
EACH outcome
Advanced Topics
Individual participant dataMissing dataDifferent types of dataObservational studiesGeneralized synthesis of evidenceMeta-regressionCritique of a systematic review
Meta-regression
Examines whether the study effects (outcomes) are related to one or more of the study characteristics.
Can be used to understand/explain heterogeneity.
Can be thought of as an epidemiological study of the trials or studies.
Clinical Questions: Meta-Regression
Are there certain situations in which BCG may be more effective for preventing TB?
Meta-regression: example
StudyStudy OROR 95% CI95% CI
11 0.3910.391 0.121, 1.2620.121, 1.262
22 0.1890.189 0.077, 0.4620.077, 0.462
33 0.2500.250 0.069, 0.9090.069, 0.909
44 0.2330.233 0.176, 0.3080.176, 0.308
55 0.8030.803 0.514, 1.2560.514, 1.256
66 0.3840.384 0.316, 0.4660.316, 0.466
77 0.1950.195 0.077, 0.4970.077, 0.497
88 1.0121.012 0.894, 1.1460.894, 1.146
99 0.6240.624 0.391, 0.9960.391, 0.996
1010 0.2460.246 0.144, 0.4220.144, 0.422
1111 0.7110.711 0.571, 0.8860.571, 0.886
1212 1.5631.563 0.373, 6.5480.373, 6.548
1313 0.9830.983 0.582, 1.6610.582, 1.661
BCG vaccine: used to prevent tuberculosis
Odds ratio estimates from 13 trials (right)
Scientists have suggested that effects may be related to geographic latitude
Funnel Plot
Funnel Plot – Organized by Latitude
Meta-regression: example, continued
Log odds ratio versus absolute latitude:
Meta-regression: example, cont
Same plot, showing precision:
Meta-regression: example, cont
Same plot, with fitted (meta-)regression line:
Meta-regression: example, cont
Is the slope of the line significantly different from 0?
If yes, we conclude that the study effects are in fact related to latitude
Meta-regression: details
In a regression model for the data: each study represents one observation
Weights equal to the study precision
Random effects model (recommended)
Built-in function in Stata: ‘metareg’
Critique of a Systematic Review
1. Research Question2. Protocol3. Search4. Study selection (inclusion/exclusion)5. Quality assessment6. Data abstraction7. Analysis
– A) Create summary measure– B) Assess for heterogeneity– C) Assess for publication bias– D) Conduct sensitivity/subgroup analyses– E) Advanced issues/techniques
8. Interpretation
Reviewing Journal Articles
Very little formal teaching“Because reviews are often highly
negative, the new researcher implicitly learns from the negative reviews received on his or her own submitted papers that reviews are supposed to be negative. It is as if the implicit message is: A reviewer’s job is to criticize the manuscript.”
12 Tips on Reviewing Articles
1. Know your mission 2. Be speedy 3. Read carefully 4. Say positive things in your review 5. Don’t exhibit hostility 6. Keep it brief 7. Don’t nitpick 8. Develop your own style 9. Be careful in recommending further experiments 10. Watch for egocentrism 11. Make a recommendation 12. Sign your review
http://www.psychologicalscience.org/observer/getArticle.cfm?id=2157
ConclusionsYou can combine almost anything
Use clinical judgment to guide you in deciding how and whether to combine studies.
Remember the main mission of a systematic review: to summarize a body of literature in a concise and clear way.
Get statistical input as needed.