Systematic reviews & meta-analyses

Download Systematic reviews & meta-analyses

Post on 05-Sep-2016




1 download

Embed Size (px)


  • Current Anaesthesia & Critical Care (2005) 16, 391394




    reve rerig

    Systematic review

    of idrimaraw c


    effect size with improved precision. Commonly it isinvoked as part of a systematic review of the

    was coined in 1976 by the psychologist, Gene Glass,

    conclusions to be reached had been proposed as

    statistical integration of separate studies. Asystematic review is an overview of primary


    Corresponding author.

    studies utilizing defined methods and criteria.2,7

    0953-7112/$ - see front matter & 2006 Elsevier Ltd. All rights reserved.doi:10.1016/j.cacc.2006.02.004

    E-mail address: (M.O. Columb).available literature.


    Modern medical practice demands the conscien-tious, explicit and judicious use of current best

    early as 1904 by Karl Pearson.4

    Meta-analysis has advantages over narrative re-views because it estimates the size of an effect,increases power and precision, and provides arigorous framework for the appraisal of research.5,6

    The difference between a systematic review and ameta-analysis is that the latter represents a

    7This is the statistical discipline of assimilating datafrom similar smaller studies to measure an overall

    to describe the statistical analysis of a largecollection of results from individual studies. Hedescribed meta-analysis as the analysis of ana-lyses.3 The concept of combining studies to enableThis is the formal processsal and evaluation of prelevant research to drspecific issue.entification, apprai-y studies and otheronclusions about a

    individual patients. This ideal can easily behampered in trying to keep pace with the expo-nential growth of research literature available.Traditionally the narrative review has been used asa means of providing a summary of availableevidence to guide clinical decision-making. Narra-tive reviews however are subjective and thereforeprone to bias and error.2 The term meta-analysisDefinitions evidence in making decisions about the care of1M.O. Columb, A.-G. Lalkhen

    Acute Intensive Care Unit, South Manchester UniverManchester M23 9LT, UK

    KEYWORDSSystematic review;Meta-analysis;Evidence-based

    Summary Systematicthe published literatuapproach to medicine.conducting systematicaffect the quality of th& 2006 Elsevier Ltd. AllSTATISTICS

    Systematic reviews & me a-analyses

    y Hospitals Trust, Wythenshawe,

    iews and meta-analyses are becoming more prevalent inand are routinely being used in the evidence-basedhis article describes the process and methodology ofiews and meta-analyses and discusses the factors thatsults and conclusions.hts reserved.

  • prospective double-blind randomized controlledtrials having the greatest weight.10,13,14 The sample

    plot. The results of individual trials are orderedand presented one below the other with their

    praised. In addition, the results of sensitivity

    Significancefollows from the greater propensity


    M.O. Columb, A.-G. Lalkhen392sizes, baseline patient characteristics, withdrawalrates and results of primary and secondary end-points of all the studies included must be tabu-lated. The more reliable meta-analyses includeonly properly randomized controlled trials that areappropriately blinded.10,1315

    The effect of interest may be either numerical,where the data are continuous, or categorical.Meta-analysis of continuous data involves the use aweighted average of the results or differences,which implies that larger studies are given moreweight or importance. The statistical methodscomprise fixed, random, mixed effects and Baye-Systematic reviews therefore may contain meta-analyses. The Cochrane Collaboration has as itsprimary purpose, the generation and disseminationof high-quality systematic reviews and meta-ana-lysis of medical interventions.8 The creation of thisorganization was precipitated by the exhortationsby Archie Cochrane, a physician and epidemiolo-gist. He encouraged the medical profession toprovide those interventions for which there wascollated evidence for their effectiveness.9 There-fore, systematic reviews and meta-analyses mustfollow the scientific process for any added value tobe gained.



    The research aim or question must be clearlydefined. This is usually to determine if a particulartreatment or intervention is beneficial. The clinicalquestion therefore needs to be precisely framedand focussed.


    The protocol must define the precise eligibility andinclusion criteria for the studies to be considered.2

    A rigorous and comprehensive search for allavailable primary studies must be conducted.Logical strategies include searches of electronicbibliographic databases such as Medline, PubMed,Embase and Cochrane Register of Controlled Trials(CENTRAL). Hand searching journals, proceedingsof scientific meetings, abstract publications, re-ference lists and contacting known researchers inthe field may be required.7,1012

    Quality assessment of the identified studiesshould be explicitly scored for reliability, withfor studies with positive or statistically significantresults to be published by scientific journalscompared to negative trials.11,14

    Replicationoccurs when the same data arepublished in multiple articles.Languageoccurs due to failure to search for

    articles other than in English, hence missingnegative trials or those with not achieving signifi-cance.analyses (vide infra) to test the strength of themain findings should be discussed for valid conclu-sions to be derived.


    Publication biasesrespective 95% confidence intervals. The pooledestimate is presented at the bottom with the 95%confidence interval.2


    The Quality of Reporting of Meta-analyses (QUOR-OM) statement describes the preferred method forreporting meta-analyses in order to prevent theproduction of flawed studies.13 These recommen-dations are based on evidence that points to theproduction of unreliable meta-analyses if certaincriteria are omitted, for example the quality ofrandomized controlled trials. Studies where alloca-tion concealment was poor can lead to a 30%overestimation of the effect of an intervention.The Discussion should cover these issues so that thestrengths and limitations are identified and ap-sian models, the differences between these modelsbeing the manner in which the variability in theresults of different studies is explained.2 Catego-rical data, such as binary (yes/no, dead/alive), arepresented as odds ratios, numbers needed to treat,absolute and relative risk ratios.2 The precisestatistical analysis of the data must be describedto allow any interested reader to replicate theprocess.


    The most common graphical technique used todisplay the results of a meta-analysis is a Forest


  • conclusions.18,19 A meta-analysis demonstrating the


    Systematic reviews & meta-analyses 393benefit of magnesium in acute myocardial infarc-tion, later discredited by the ISIS 4 trial, is oftencited as an example by detractors.20 It has beensuggested that the incorrect conclusion reached bythe magnesium meta-analysis was probably due toselective non-publication of negative trials.17 Theassertion by the Cochrane Injuries Group that theuse of intravenous albumin in critically ill patientsresults in one death for every 17 patients washeavily criticised.21 The authors were accused of aControversy

    Whilst the theoretical advantages of meta-analysesare clear, there are those who believe that thepractice is dubious. In a meta-analysis comparingthe effects of epidural analgesia with ropivacaineand bupivacaine on obstetric outcomes, theauthors were fortunate that basic errors in dataabstraction in five of 20 studies (with importanterrors in three) actually worked in favour of theirSensitivity analysis

    This involves checking to see whether alterations ofthe analyses by the omission of trials originallyincluded in the meta-analysis materially affect theoverall result. Studies may be excluded on the basisof methodological quality (excluding studies ofpoorer quality) or omitting studies stopped earlywhich are liable to bias towards rejecting the nullhypothesis.2Selectionoccurs when citations are specificallyderived from articles such as narrative reviews orexpert opinion.Funnel plots (where the magnitude of the

    treatment effect is plotted against the samplesize) may be used to detect publication biases. Asymmetrical inverted funnel implies that thestudies found are likely to be inclusive whilst anasymmetrical plot suggests that small negative orneutral studies have been omitted.7,17

    Statistical heterogeneity

    For studies to be combinable they should demon-strate homogeneity or similarity particularly withrespect to the subjects, pre-test variables andmethodology. Statistical tests for heterogeneityshould be performed prior to pooling data foranalysis of the effect of interest.16 Combiningheterogeneous studies may lead to irrelevant anderroneous conclusions.17References

    1. Sackett DL, Rosenberg WM, Gray JA, Haynes RB, RichardsonWS. Evidence based medicine: what it is and what it isnt. BrMed J 1996;312(7023):712.

    2. Egger M, Smith GD, Phillips AN. Meta-analysis: principles andprocedures. Br Med J 1997;315(7121):15337.

    3. Glass GV. Primary, secondary and meta-analysis of research.Educ Res 1976;5:35179.

    4. Pearson K. Report on certain enteric fever inoculationstatistics. Br Med J 1904;3:12436.

    5. Imperial TF. Meta-analysis: when and how. Hepatology1999;29:2631.

    6. Delahaye F, Landrivon G, Ecochard R, Colin C. Meta-analysis.Health Policy 1991;19(23):18596.

    7. Naylor CD. Meta-analysis and the meta-epidemiology ofclinical research. Br Med J 1997;315(7109):6179.

    8. Bero L, Rennie D. The Cochrane Collaboration. Preparing,maintaining, and disseminating systematic reviews of theeffects of health care. J Am Med Assoc1995;274(24):19358.

    9. Chalmers I, Dickersin K, Chalmers TC. Getting to grips withArchie Cochranes agenda. Br Med J 1992;305(6857):7868.Conclusion

    Meta-analyses that fail to produce a definitiveanswer to a clinical question can provide a potentstimulus for initiating research by highlighting theinadequacy of existing evidence.26 By indicatingthe heterogeneity of studies in terms of their end-points or outcomes, further studies may be morerobustly designed preventing potentially unethicaltrials. In the end a systematic review or meta-analysis can only be as reliable as the originalstudies.lack of clinical insight and of ignoring the hetero-geneity in the studies.22,23


    Meta-analyses however have also produced impor-tant contributions to clinical practice. The EarlyBreast Cancer Trialists Collaborative Group showedthat 20,000 lives per year could be saved bytamoxifen. This was accomplished by combiningdata from 55 trials.22,24 Cumulative meta-analysis,where the analysis is repeated when a new trial ispublished, may result in earlier recommendationsin terms of the clinical effectiveness of a particulartherapy. A retrospective cumulative meta-analysisfor the use of streptokinase in acute myocardialinfarction had demonstrated significant benefit by1973.10 However, a further 34,542 patients wereenrolled in studies before the treatment waslicensed after ISIS-2 in 1988.25

  • 10. Egger M, Smith GD. Meta-analysis. Potentials and promise.Br Med J 1997;315(7119):13714.

    11. Egger M, Smith GD. Bias in location and selection of studies.Br Med J 1998;316(7124):616.

    12. Smith GD, Egger M. Meta-analysis: unresolved issues andfurther developments. Br Med J 1996;316:22131.

    13. Moher D, Cook DJ, Eastwood S, Olkin I, Rennie D, Stroup DF.Improving the quality of reports of meta-analyses ofrandomised controlled trials: the QUOROM statement.Quality of Reporting of Meta-analyses. Lancet1999;354(9193):1896900.

    14. Bhandari M, Devereaux PJ, Montori V, Cina C, Tandan V,Guyatt GH, et al. Users guide to the surgical literature: howto use a systematic literature review and meta-analysis. CanJ Surg 2004;47(1):607.

    15. Smith GD, Egger M, Philips AN. Meta-analysis: beyond thegrand mean? Br Med J 1997;315:16109.

    16. Greenhalgh T. How to read a paper: the basics of evidencebased medicine, 2nd ed. London: BMJ Books; 2001.

    17. Egger M, Smith GD. Misleading meta-analysis. Br Med J1995;311(7007):7534.

    18. Halpern SH, Walsh V. Epidural ropivacaine versus bupiva-caine for labor: a meta-analysis. Anesth Analg 2003;96(5):14739.

    19. Polley LS, Columb MO. Ropivacaine and bupivacaine:concentrating on dosing!. Anesth Analg 2003;96(5):12513.

    20. ISIS-4 (Fourth International Study of Infarct Survival)Collaborative Group. ISIS-4: a randomised factorial trialassessing early oral captopril, oral mononitrate, andintravenous magnesium sulphate in 58,050 patients withsuspected acute myocardial infarction. Lancet1995;345(8951):66985.

    21. Cochrane Injuries Group Albumin Reviewers. Human albuminadministration in critically ill patients: systematic reviewof randomised controlled trials. Br Med J 1998;317(7153):23540.

    22. Horton R. The information wars. Lancet 1999;353(9148):1645.

    23. Petros A, Schindler M, Pierce C, Jacobe S, Mok Q. Humanalbumin administration in critically ill patients. Evidenceneeds to be shown in paediatrics. Br Med J1998;317(7162):8826.

    24. Early Breast Cancer Trialists Collaborative Group. Tamox-ifen for early breast cancer: an overview of the randomisedtrials. Lancet 1998;351(9114):145167.

    25. ISIS-2 (Second International Study of Infarct Survival)Collaborative Group. Randomised trial of intravenousstreptokinase, oral aspirin, both, or neither among 17,187cases of suspected acute myocardial infarction: ISIS-2.Lancet 1988;2(8607):34960.

    26. Naylor D. The case for failed meta-analyses. J Eval ClinPract 1995;1:12730.


    M.O. Columb, A.-G. Lalkhen394

    Systematic reviews & meta-analysesDefinitionsSystematic reviewMeta-analysis


    IssuesPublication biasesStatistical heterogeneitySensitivity analysisControversyBenefits



View more >