Cochrane Handbook for Systematic Reviews of Interventions Copyright © 2008 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd under “The Cochrane Book Series” Imprint. This extract is made available solely for use in the authoring, editing or refereeing of Cochrane reviews, or for training in these processes by representatives of formal entities of The Cochrane Collaboration. Other than for the purposes just stated, no part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning or otherwise, except under the terms of the Copyright, Designs and Patents Act 1988 or under the terms of a licence issued by the Copyright Licensing Agency Ltd, 90 Tottenham Court Road, London W1T 4LP, UK, without the permission in writing of the copyright holders. Permission to translate part or all of this document must be obtained from the publishers.

This extract is from Handbook version 5.0.1, which should be cited as follows: Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.0.1 [updated September 2008]. The Cochrane Collaboration, 2008. Available from www.cochrane-handbook.org. The material is also published in Higgins JPT, Green S (editors), Cochrane Handbook for Systematic Reviews of Interventions (ISBN 978-0470057964) by John Wiley & Sons Ltd, The Atrium, Southern Gate, Chichester, West Sussex PO19 8SQ, England, Telephone (+44) 1243 779777; Email (for orders and customer service enquiries): cs-books@wiley.co.uk. Visit their Home Page on www.wiley.com.

PREFACE The Cochrane Handbook for Systematic Reviews of Interventions (the Handbook) provides guidance to authors for the preparation of Cochrane Intervention reviews (including Cochrane Overviews of reviews).

Keeping up to date The Handbook is updated regularly to reflect advances in systematic review methodology and in response to feedback from users. Please refer to the following web site for the most recent version, for interim updates to the guidance and for details of previous versions of the Handbook. www.cochrane.org/resources/handbook Users of the Handbook are encouraged to send feedback and corrections to the Handbook editors; contact details are available on the web site.

Central sources of support

Present sources of support The Cochrane Collaboration Medical Research Council, United Kingdom Department of Health and Ageing, Australia

Monash University, Australia

Previous sources of support National Health Service Research and Development Programme, United Kingdom Health Research Board, Ireland National Institute of Public Health, Norway Copenhagen Hospital Corporation, Denmark Health Services Research and Development Service and the University of Texas Health Science Center, San Antonio, USA US Veterans Health Administration, USA Oxford Regional Health Authority, UK Nuffield Provincial Hospitals Trust, UK LW Frohlich Fund, USA Norwegian Ministry of Health and Social Affairs, Norway Norwegian Research Council, Norway Glaxo Wellcome, Norway

Acknowledgements We are grateful to all past and current members of the Handbook Advisory Group for discussions and feedback, and would particularly like to thank Doug Altman, Chris Cates, Mike Clarke, Jon Deeks, Donna Gillies, Andrew Herxheimer, Harriet MacLehose, Philippa Middleton, Ruth Mitchell, David Moher, Donald Patrick, Ian Shemilt, Lesley Stewart, Jessica Thomas, Jane Tierney and Danielle Wheeler. Many contributed constructive and timely peer review. We thank Phil Alderson, Claire Allen, Judith Anzures, Chris Cates, Jonathan Craig, Miranda Cumpston, Chris Del Mar, Kay Dickersin, Christian Gluud, Peter Gøtzsche, Frans Helmerhorst, Jini Hetherington, Sophie Hill, Sally Hopewell, Steve McDonald, David Moher, Ann Møller, Duncan Mortimer, Karen New, Denise O’Connor, Jordi Pardo, Rob Scholten, Simon Thompson, Jan Vandenbroucke, Janet Wale, Phil Wiffen, Hywel Williams, Paula Williamson, Jim Wright and Diana Wyatt. Specific administrative support for this version of the Handbook was provided by Jane Lane. In addition, skilled and generous administrative and technical support provided by Claire Allen, Dave Booker, Jini Hetherington, Monica Kjeldstrøm, Cindy Manukonga, Rasmus Moustgaard, Jane Predl, Jacob Riis and Verena Roloff has contributed greatly to the preparation and co-ordination of this Handbook. We would also like to thank Lucy Sayer, Fiona Woods and Laura Mellor at Wiley-Blackwell for their patience, support and advice. This major revision of the Handbook would not have been possible without the generous support provided to the editors by colleagues at the MRC Biostatistics Unit and the Institute of Public Health in Cambridge, UK and at the Australasian Cochrane Centre, Monash University, Australia.

The Handbook editors Julian Higgins is a Senior Statistician at the MRC Biostatistics Unit, Institute of Public Health, University of Cambridge and a Visiting Fellow at the UK Cochrane Centre, Oxford, UK. Sally Green is a Professorial Fellow at the Institute of Health Services Research at Monash University, Melbourne, Australia and Director of the Australasian Cochrane Centre.

Major contributors Acquadro, Catherine MAPI Research Institute Lyon France Alderson, Philip National Institute for Health and Clinical

Excellence London/Manchester United Kingdom Altman, Douglas G Centre for Statistics in Medicine University of Oxford Oxford United Kingdom Armstrong, Rebecca The McCaughey Centre: VicHealth Centre for

the Promotion of Mental Health and Community Wellbeing

University of Melbourne Melbourne Australia Askie, Lisa M NHMRC Clinical Trials Centre University of Sydney Camperdown Australia Becker, Lorne A Department of Family Medicine SUNY Upstate Medical University Syracuse, NY United States of America Berlin, Jesse A Pharmacoepidemiology Johnson & Johnson Pharmaceutical Research

and Development Titusville, NJ USA

Booth, Andrew School of Health and Related Research University of Sheffield Sheffield United Kingdom Byford, Sarah Centre for the Economics of Mental Health Institute of Psychiatry King’s College London London United Kingdom Clarke, Mike UK Cochrane Centre National Institute for Health Research Oxford United Kingdom Deeks, Jonathan J Department of Public Health and

Epidemiology University of Birmingham Birmingham United Kingdom Doyle, Jodie The McCaughey Centre: VicHealth Centre for

the Promotion of Mental Health and Community Wellbeing

University of Melbourne Melbourne Australia Drummond, Michael Centre for Health Economics University of York York United Kingdom Egger, Matthias Institute of Social and Preventive Medicine University of Bern

Bern Switzerland Eisenstein, Eric Duke Clinical Research Center Duke University Durham, NC United States of America Ghersi, Davina Department of Research Policy and

Cooperation World Health Organization Geneva Switzerland Glanville, Julie Centre for Reviews and Dissemination University of York York United Kingdom Glasziou, Paul P Department of Primary Health Care University of Oxford Oxford United Kingdom Green, Sally Australasian Cochrane Centre Monash University Melbourne Australia Guyatt, Gordon H Departments of Clinical Epidemiology and

Biostatics McMaster University Ontario Canada Hannes, Karin Belgian Centre for Evidence-Based Medicine Leuven Belgium Herxheimer, Andrew Co-founder, DIPEx; Emeritus Fellow, UK

Cochrane Centre London United Kingdom Higgins, Julian PT MRC Biostatistics Unit Cambridge United Kingdom

Knapp, Martin Institute of Psychiatry King’s College London and London School of Economics London United Kingdom Lefebvre, Carol UK Cochrane Centre National Institute for Health Research Oxford United Kingdom Loke, Yoon K School of Medicine, Health Policy and

Practice University of East Anglia Norwich United Kingdom Mallender, Jacqueline Matrix Knowledge Group Ltd. London United Kingdom Manheimer, Eric Center for Integrative Medicine University of Maryland School of Medicine Baltimore, MA United States of America McDaid, David Personal Social Services Research Unit London School of Economics and Political

Science London United Kingdom Moher, David Chalmers Research Group, Children’s Hospital

of Eastern Ontario Research Institute; Department of Epidemiology and Community

Medicine, University of Ottawa Ottawa Canada Mugford, Miranda Health Economics Group School of Medicine, Health Policy and

Practice University of East Anglia Norwich United Kingdom Noyes, Jane Centre for Health-Related Research

School of Healthcare Sciences Bangor University Bangor Wales United Kingdom O’Connor, Denise Australasian Cochrane Centre Monash University Melbourne Australia Oxman, Andrew D Preventive and International Health Care Unit Norwegian Knowledge Centre for the Health

Services Oslo Norway Patrick, Donald L Department of Health Services and Seattle

Quality of Life Group University of Washington Seattle, WA United States of America Pearson, Alan Joanna Briggs Institute University of Adelaide Adelaide Australia Popay, Jennie Institute for Health Research Lancaster University Lancaster United Kingdom Price, Deirdre Department of Clinical Pharmacology University of Oxford Oxford United Kingdom Reeves, Barnaby Bristol Heart Institute University of Bristol Bristol United Kingdom Scholten, Rob Dutch Cochrane Centre Academic Medical Center Amsterdam The Netherlands

Schünemann, Holger J INFORMA/CLARITY Research/Department

of Epidemiology National Cancer Institute Regina Elena Rome Italy Shemilt, Ian Health Economics Group School of Medicine, Health Policy and

Practice University of East Anglia Norwich United Kingdom Sterne, Jonathan AC Department of Social Medicine University of Bristol Bristol United Kingdom Stewart, Lesley A Centre for Reviews and Dissemination University of York York United Kingdom Tierney, Jayne F MRC Clinical Trials Unit London United Kingdom Vale, Luke Health Economics Research Unit University of Aberdeen Aberdeen United Kingdom Vist, Gunn E Preventive and International Health Care Unit Norwegian Knowledge Centre for the Health

Services Oslo Norway Walker, Damian Health Systems Program Department of International Health Johns Hopkins Bloomberg School of Public

Health Baltimore, MA United States of America Waters, Elizabeth The McCaughey Centre: VicHealth Centre for

the Promotion of Mental Health and Community Wellbeing

University of Melbourne Melbourne Australia Wells, George A Department of Epidemiology and Community

Medicine

University of Ottawa Ottawa Ontario Canada

Contents listing

1 Introduction 1.1 The Cochrane Collaboration

1.1.1 Introduction Box 1.1.a: The Principles of The Cochrane Collaboration

1.1.2 Structure of The Cochrane Collaboration 1.1.3 Publication of Cochrane reviews

Box 1.1.b: Databases published in The Cochrane Library 1.2 Systematic reviews

1.2.1 The need for systematic reviews 1.2.2 What is a systematic review?

1.3 About this Handbook 1.4 Contributors to the Handbook 1.5 Chapter information 1.6 References

2 Preparing a Cochrane review 2.1 Rationale for protocols 2.2 Format of a Cochrane review

2.2.1 Rationale for the format of a Cochrane review 2.2.2 Outline of a protocol for a Cochrane review

Box 2.2.a: Sections of a protocol for a Cochrane review 2.2.3 Detailed outline of a Cochrane review

Box 2.2.b: Sections of a Cochrane review 2.3 Logistics of doing a review

2.3.1 Motivation for undertaking a review 2.3.2 Planning the topic and scope of a review 2.3.3 Registering a protocol 2.3.4 The review team

2.3.4.1 The importance of a team 2.3.4.2 Consumer involvement 2.3.4.3 Advisory groups

Box 2.3.a: An example of the benefits of using an advisory group in the planning process 2.3.5 Cochrane software for review authors and editorial bases of Cochrane Review Groups 2.3.6 Training 2.3.7 Editorial procedures of a Cochrane Review Group 2.3.8 Resources for a systematic review

Box 2.3.b: Timeline for a Cochrane review 2.3.9 Seeking funding

2.4 Publication of Cochrane reviews in print journals and books 2.5 Publication of previously published reviews as Cochrane reviews 2.6 Declaration of interest and commercial sponsorship

Box 2.6.a The Cochrane Collaboration Code of Conduct for Avoiding Potential Financial Conflicts of Interest

2.7 Chapter information 2.8 References

3 Maintaining reviews: updates, amendments and feedback 3.1 Introduction

3.1.1 Why maintain a review? 3.1.2 How frequently should a review be revisited?

3.2 Some important definitions

3.2.1 Introduction 3.2.2 Updates and amendments: 3.2.3 Citation versions of Cochrane reviews and protocols

Box 3.2.a: Events leading to the creation of a Citation version of a Cochrane protocol or review Figure 3.2.a: Summary of changes to Cochrane reviews Figure 3.2.b: Summary of changes to Cochrane protocols

3.2.4 Application of terms to Cochrane protocols 3.2.4.1 Amendments to protocols 3.2.4.2 New citation versions of protocols

Box 3.2.b: Criteria for a new citation version of a Cochrane protocol 3.2.4.3 Examples of changes to protocols that do not indicate a new citation version

3.2.5 Application of terms to Cochrane reviews 3.2.5.1 Updates of reviews 3.2.5.2 Amendments to reviews 3.2.5.3 New citation versions of reviews

Box 3.2.c: Criteria for a new citation version of a Cochrane review 3.2.5.4 Examples of changes to reviews that do not indicate a new citation version

3.3 Important dates associated with Cochrane reviews 3.3.1 Introduction 3.3.2 Date review assessed as up to date

Box 3.3.a: Guidance for declaring a review as being up to date 3.3.3 Date of search 3.3.4 Date next stage expected 3.3.5 Date of last edit 3.3.6 Date declared review no longer needs to be updated

3.4 Considerations when updating a Cochrane review 3.4.1 Where to start 3.4.2 Updating a review with an unchanged review question

3.4.2.1 Re-executing the search 3.4.2.2 Updating reviews when no new studies are found 3.4.2.3 Updating reviews when new studies are found

3.4.3 Revising review questions and selection criteria 3.4.4 Splitting reviews 3.4.5 Amending the methodology of a review 3.4.6 Other changes to the review 3.4.7 Editorial process

3.5 ‘What’s new’ tables and History 3.5.1 What’s new events 3.5.2 Completing the ‘What’s new’ table

Table 3.5.a: Available ‘What’s new’ events for protocols Table 3.5.b: Available ‘What’s new’ events for reviews

3.5.3 History table 3.6 Incorporating and addressing feedback in a Cochrane review 3.7 Chapter information 3.8 References

4 Guide to the contents of a Cochrane protocol and review 4.1 Introduction 4.2 Title and review information (or protocol information)

4.2.1 Title Table 4.2.a: Structure for Cochrane review titles

4.2.2 Authors 4.2.3 Contact person 4.2.4 Dates 4.2.5 What’s New and History

4.3 Abstract 4.4 Plain language summary 4.5 Main text 4.6 Tables

4.6.1 Characteristics of included studies 4.6.2 Risk of bias 4.6.3 Characteristics of excluded studies 4.6.4 Characteristics of studies awaiting classification 4.6.5 Characteristics of ongoing studies 4.6.6 Summary of findings tables 4.6.7 Additional tables

4.7 Studies and references 4.7.1 References to studies 4.7.2 Other references

4.8 Data and analyses Figure 4.8.a: Illustration of the hierarchy of the ‘Data and analyses’ section.

4.9 Figures Table 4.9.a: Types of figures that can be included in a Cochrane review 4.9.1 RevMan plots and graphs 4.9.2 Other figures

4.10 Sources of support to the review 4.11 Feedback 4.12 Appendices 4.13 Chapter information 4.14 References

5 Defining the review question and developing criteria for including studies 5.1 Rationale for well-formulated questions

5.1.1 Eligibility criteria 5.2 Defining types of participants: which people and populations?

Box 5.2.a: Factors to consider when developing criteria for ‘Types of participants’ 5.3 Defining types of interventions: which comparisons to make?

Box 5.3.a: Factors to consider when developing criteria for ‘Types of interventions’ 5.4 Defining types of outcomes: which outcome measures are most important?

5.4.1 Listing relevant outcomes 5.4.2 Prioritizing outcomes: Main, primary and secondary outcomes

Box 5.4.a: Factors to consider when developing criteria for ‘Types of outcomes’ 5.4.3 Adverse outcomes 5.4.4 Economic data

5.5 Defining types of study 5.6 Defining the scope of a review question (broad versus narrow)

Table 5.6.a: Some advantages and disadvantages of broad versus narrow review questions 5.7 Changing review questions 5.8 Chapter information 5.9 References

6 Searching for studies 6.1 Introduction

6.1.1 General issues 6.1.1.1 Role of the Trials Search Co-ordinator 6.1.1.2 Minimizing bias 6.1.1.3 Studies versus reports of studies 6.1.1.4 Copyright

6.1.2 Summary points 6.2 Sources to search

6.2.1 Bibliographic databases 6.2.1.1 Bibliographic databases – general introduction 6.2.1.2 The Cochrane Central Register of Controlled Trials (CENTRAL) 6.2.1.3 MEDLINE and EMBASE 6.2.1.4 National and regional databases

Box 6.2.a: Examples of regional electronic bibliographic databases 6.2.1.5 Subject-specific databases

Box 6.2.b: Examples of subject-specific electronic bibliographic databases 6.2.1.6 Citation indexes 6.2.1.7 Dissertations and theses databases

Box 6.2.c: Examples of dissertations and theses databases 6.2.1.8 Grey literature databases

6.2.2 Journals and other non-bibliographic-database sources 6.2.2.1 Handsearching 6.2.2.2 Full-text journals available electronically

Box 6.2.d: Examples of full-text journal sources available worldwide without charge 6.2.2.3 Tables of contents

Box 6.2.e: Examples of organizations offering Table of Contents (TOC) services 6.2.2.4 Conference abstracts or proceedings

Box 6.2.f: Examples of specialist conference abstract sources 6.2.2.5 Other reviews, guidelines and reference lists as sources of studies

Box 6.2.g: Examples of evidence-based guidelines 6.2.2.6 Web searching

6.2.3 Unpublished and ongoing studies 6.2.3.1 National and international trials registers

Box 6.2.h: Examples of national and international trials registers 6.2.3.2 Subject-specific trials registers 6.2.3.3 Pharmaceutical industry trials registers

Box 6.2.i: Examples of pharmaceutical industry trials registers 6.2.3.4 Trials results registers and other sources

Box 6.2.j: Examples of trials results registers 6.2.4 Summary points

6.3 Planning the search process 6.3.1 Involving Trials Search Co-ordinators and healthcare librarians in the search process 6.3.2 Collaboration-wide search initiatives

Figure 6.3.a: Illustration of the contents of CENTRAL 6.3.2.1 What is in The Cochrane Central Register of Controlled Trials (CENTRAL) from

MEDLINE? Box 6.3.a: Cochrane definitions and criteria for randomized controlled trials (RCTs) and

controlled clinical trials (CCTs) Box 6.3.b: US National Library of Medicine 2008 definitions for the Publication Type terms

‘Randomized Controlled Trial’ and ‘Controlled Clinical Trial’ 6.3.2.2 What is in The Cochrane Central Register of Controlled Trials (CENTRAL) from

EMBASE? 6.3.2.3 What is in The Cochrane Central Register of Controlled Trials (CENTRAL) from other

databases and handsearching? 6.3.2.4 What is in The Cochrane Central Register of Controlled Trials (CENTRAL) from

Specialised Registers of Cochrane Review Groups and Fields? 6.3.3 Searching CENTRAL, MEDLINE and EMBASE: specific issues

6.3.3.1 Searching The Cochrane Central Register of Controlled Trials (CENTRAL): specific issues Box 6.3.c: Example of exclusion of MEDLINE and EMBASE records when searching

CENTRAL 6.3.3.2 Searching MEDLINE and EMBASE: specific issues

6.3.4 Summary points 6.4 Designing search strategies

6.4.1 Designing search strategies – an introduction 6.4.2 Structure of a search strategy 6.4.3 Service providers and search interfaces 6.4.4 Sensitivity versus precision 6.4.5 Controlled vocabulary and text words 6.4.6 Synonyms, related terms, variant spellings, truncation and wildcards 6.4.7 Boolean operators (AND, OR and NOT)

Figure 6.4.a: Combining concepts as search sets 6.4.8 Proximity operators (NEAR, NEXT and ADJ) 6.4.9 Language, date and document format restrictions 6.4.10 Identifying fraudulent studies, other retracted publications, errata and comments 6.4.11 Search filters

6.4.11.1 The Cochrane Highly Sensitive Search Strategies for identifying randomized trials in MEDLINE

6.4.11.2 Search filters for identifying randomized trials in EMBASE 6.4.12 Updating searches 6.4.13 Demonstration search strategies

Box 6.4.e: Demonstration search strategy for the CENTRAL, for the topic ‘Tamoxifen for breast cancer’

Box 6.4.f: Demonstration search strategy for the MEDLINE (Ovid format), for the topic ‘Tamoxifen for breast cancer’

6.4.14 Summary points 6.5 Managing references

6.5.1 Bibliographic software 6.5.2 Which fields to download

Box 6.5.a: Important field labels in PubMed 6.5.3 Summary points

6.6 Documenting and reporting the search process 6.6.1 Documenting the search process 6.6.2 Reporting the search process in the review

6.6.2.1 Reporting the search process in the protocol 6.6.2.2 Reporting the search process in the review

6.6.3 Summary points 6.7 Chapter information 6.8 References

7 Selecting studies and collecting data 7.1 Introduction 7.2 Selecting studies

7.2.1 Studies (not reports) as the unit of interest 7.2.2 Identifying multiple reports from the same study 7.2.3 A typical process for selecting studies 7.2.4 Implementation of the selection process 7.2.5 Selecting ‘excluded studies’ 7.2.6 Measuring agreement

7.2.6.1 Calculations for a simple kappa statistic 7.3 What data to collect

7.3.1 What are data? 7.3.2 Methods and potential sources of bias 7.3.3 Participants and setting 7.3.4 Interventions

7.3.4.1 Integrity of interventions 7.3.5 Outcome measures

7.3.5.1 Adverse outcomes 7.3.6 Results

7.3.7 Other information to collect 7.4 Sources of data

7.4.1 Reports 7.4.2 Correspondence with investigators 7.4.3 Individual patient data

7.5 Data collection forms 7.5.1 Rationale for data collection forms 7.5.2 Electronic versus paper data collection forms 7.5.3 Design of a data collection form 7.5.4 Coding and explanations

7.6 Extracting data from reports 7.6.1 Introduction 7.6.2 Who should extract data? 7.6.3 Preparing for data extraction 7.6.4 Extracting data from multiple reports of the same study 7.6.5 Reliability and reaching consensus 7.6.6 Summary

7.7 Extracting study results and converting to the desired format 7.7.1 Introduction 7.7.2 Data extraction for dichotomous outcomes 7.7.3 Data extraction for continuous outcomes

7.7.3.1 Post-intervention versus change from baseline 7.7.3.2 Obtaining standard deviations from standard errors and confidence intervals for group

means 7.7.3.3 Obtaining standard deviations from standard errors, confidence intervals, t values and P

values for differences in means 7.7.3.4 Transformations and skewed data 7.7.3.5 Medians and interquartile ranges 7.7.3.6 Ranges 7.7.3.7 No information on variability 7.7.3.8 Combining groups

7.7.4 Data extraction for ordinal outcomes 7.7.5 Data extraction for counts

7.7.5.1 Extracting counts as dichotomous data 7.7.5.2 Extracting counts as continuous data 7.7.5.3 Extracting counts as time-to-event data 7.7.5.4 Extracting counts as rate data

7.7.6 Data extraction for time-to-event outcomes 7.7.7 Data extraction for estimates of effects

7.7.7.1 Effect estimates and generic inverse variance meta-analysis 7.7.7.2 Obtaining standard errors from confidence intervals and P values: absolute (difference)

measures 7.7.7.3 Obtaining standard errors from confidence intervals and P values: ratio measures

7.8 Managing data 7.9 Chapter information 7.10 References

8 Assessing risk of bias in included studies 8.1 Introduction 8.2 What is bias?

8.2.1 ‘Bias’ and ‘risk of bias’ 8.2.2 ‘Risk of bias’ and ‘quality’ 8.2.3 Establishing empirical evidence of biases

8.3 Tools for assessing quality and risk of bias 8.3.1 Types of tools

8.3.2 Reporting versus conduct 8.3.3 Quality scales and Cochrane reviews 8.3.4 Collecting information for assessments of risk of bias

8.4 Introduction to sources of bias in clinical trials Table 8.4.a: A common classification scheme for bias

8.5 The Cochrane Collaboration’s tool for assessing risk of bias 8.5.1 Overview

Table 8.5.a: The Cochrane Collaboration’s tool for assessing risk of bias 8.5.2 The description

Table 8.5.b: Examples of summary descriptions for Sequence generation entry 8.5.3 The judgement

Table 8.5.c: Criteria for judging risk of bias in the ‘Risk of bias’ assessment tool 8.6 Presentation of assessments of risk of bias

Figure 8.6.a: Example of a ‘Risk of bias’ table for a single study (fictional) Figure 8.6.b: Example of a ‘Risk of bias graph’ Figure Figure 8.6.c: Example of a ‘Risk of bias summary’ Figure

8.7 Summary assessments of risk of bias Table 8.7.a: Possible approach for summary assessments of the risk of bias for each important

outcome (across domains) within and across studies 8.8 Incorporating assessments into analyses

8.8.1 Introduction 8.8.2 Exploring the impact of risk of bias

8.8.2.1 Graphing results according to risk of bias 8.8.2.2 Studies assessed as at unclear risk of bias 8.8.2.3 Meta-regression and comparisons of subgroups

8.8.3 Including ‘risk of bias’ assessments in analyses 8.8.3.1 Possible analysis strategies

8.8.4 Other methods for addressing risk of bias 8.8.4.1 Direct weighting 8.8.4.2 Bayesian approaches

8.9 Sequence generation 8.9.1 Rationale for concern about bias 8.9.2 Assessing risk of bias in relation to adequate or inadequate sequence generation

8.9.2.1 Adequate methods of sequence generation 8.9.2.2 Inadequate methods of sequence generation 8.9.2.3 Methods of sequence generation with unclear risk of bias

8.10 Allocation sequence concealment 8.10.1 Rationale for concern about bias 8.10.2 Assessing risk of bias in relation to adequate or inadequate allocation sequence concealment

8.10.2.1 Adequate methods of allocation sequence concealment Table 8.10.a: Minimal and extended criteria for judging concealment of allocation sequence to

be adequate (low risk of bias) 8.11 Blinding of participants, personnel and outcome assessors

8.11.1 Rationale for concern about bias 8.11.2 Assessing risk of bias in relation to adequate or inadequate blinding

8.12 Incomplete outcome data 8.12.1 Rationale for concern about bias 8.12.2 Assessing risk of bias from incomplete outcome data

8.12.2.1 Low risk of bias due to incomplete outcome data 8.12.2.2 High risk of bias due to incomplete outcome data 8.12.2.3 Attempts to address missing data in reports: imputation

8.13 Selective outcome reporting 8.13.1 Rationale for concern about bias 8.13.2 Assessing risk of bias from selective reporting of outcomes

8.14 Other potential threats to validity 8.14.1 Rationale for concern about bias

8.14.1.1 Design-specific risks of bias 8.14.1.2 Early stopping 8.14.1.3 Baseline imbalance 8.14.1.4 Blocked randomization in unblinded trials 8.14.1.5 Differential diagnostic activity 8.14.1.6 Further examples of potential biases

8.14.2 Assessing risk of bias from other sources 8.15 Chapter information 8.16 References

9 Analysing data and undertaking meta-analyses 9.1 Introduction

9.1.1 Do not start here! 9.1.2 Planning the analysis 9.1.3 Why perform a meta-analysis in a review? 9.1.4 When not to use meta-analysis in a review 9.1.5 What does a meta-analysis entail? 9.1.6 Which comparisons should be made? 9.1.7 Writing the analysis section of the protocol

9.2 Types of data and effect measures 9.2.1 Types of data 9.2.2 Effect measures for dichotomous outcomes

Box 9.2.a: Calculation of risk ratio (RR), odds ratio (OR) and risk difference (RD) from a 2×2 table

9.2.2.1 Risk and odds 9.2.2.2 Measures of relative effect: the risk ratio and odds ratio 9.2.2.3 Warning: OR and RR are not the same 9.2.2.4 Measure of absolute effect: the risk difference 9.2.2.5 What is the event?

9.2.3 Effect measures for continuous outcomes 9.2.3.1 The mean difference (or difference in means) 9.2.3.2 The standardized mean difference

9.2.4 Effect measures for ordinal outcomes and measurement scales 9.2.5 Effect measures for counts and rates

9.2.5.1 Warning: counting events or counting participants? 9.2.6 Effect measures for time-to-event (survival) outcomes 9.2.7 Expressing intervention effects on log scales

9.3 Study designs and identifying the unit of analysis 9.3.1 Unit-of-analysis issues 9.3.2 Cluster-randomized trials 9.3.3 Cross-over trials 9.3.4 Repeated observations on participants 9.3.5 Events that may re-occur 9.3.6 Multiple treatment attempts 9.3.7 Multiple body parts I: body parts receive the same intervention 9.3.8 Multiple body parts II: body parts receive different interventions 9.3.9 Multiple intervention groups

9.4 Summarizing effects across studies 9.4.1 Meta-analysis 9.4.2 Principles of meta-analysis 9.4.3 A generic inverse-variance approach to meta-analysis

9.4.3.1 Random-effects (DerSimonian and Laird) method for meta-analysis 9.4.3.2 The generic inverse variance outcome type in RevMan

9.4.4 Meta-analysis of dichotomous outcomes 9.4.4.1 Mantel-Haenszel methods

9.4.4.2 Peto odds ratio method 9.4.4.3 Random-effects method 9.4.4.4 Which measure for dichotomous outcomes?

9.4.5 Meta-analysis of continuous outcomes 9.4.5.1 Which measure for continuous outcomes? 9.4.5.2 Meta-analysis of change scores 9.4.5.3 Meta-analysis of skewed data

9.4.6 Combining dichotomous and continuous outcomes 9.4.7 Meta-analysis of ordinal outcomes and measurement scales 9.4.8 Meta-analysis of counts and rates 9.4.9 Meta-analysis of time-to-event outcomes 9.4.10 A summary of meta-analysis methods available in RevMan

Table 9.4.a: Summary of meta-analysis methods available in RevMan 9.4.11 Use of vote counting for meta-analysis

9.5 Heterogeneity 9.5.1 What is heterogeneity? 9.5.2 Identifying and measuring heterogeneity 9.5.3 Strategies for addressing heterogeneity 9.5.4 Incorporating heterogeneity into random-effects models

9.6 Investigating heterogeneity 9.6.1 Interaction and effect modification 9.6.2 What are subgroup analyses? 9.6.3 Undertaking subgroup analyses

9.6.3.1 Is the effect different in different subgroups? 9.6.4 Meta-regression 9.6.5 Selection of study characteristics for subgroup analyses and meta-regression

9.6.5.1 Ensure that there are adequate studies to justify subgroup analyses and meta-regressions 9.6.5.2 Specify characteristics in advance 9.6.5.3 Select a small number of characteristics 9.6.5.4 Ensure there is scientific rationale for investigating each characteristic 9.6.5.5 Be aware that the effect of a characteristic may not always be identified 9.6.5.6 Think about whether the characteristic is closely related to another characteristic

(confounded) 9.6.6 Interpretation of subgroup analyses and meta-regressions 9.6.7 Investigating the effect of baseline risk 9.6.8 Dose-response analyses

9.7 Sensitivity analyses 9.8 Chapter information

Box 9.8.a: The Cochrane Statistical Methods Group 9.9 References

10 Addressing reporting biases 10.1 Introduction

Table 10.1.a: Definitions of some types of reporting biases 10.2 Types of reporting biases and the supporting evidence

10.2.1 Publication bias Figure 10.2.a: Cumulative full publication of results initially presented as abstracts from 45 studies

reporting time to publication that followed up research presented at meetings and conferences Table 10.2.a: Publication status of five cohorts of research projects approved by ethics committees

or institutional review boards which had been completed and analysed at the time of follow up. (Adapted from Hopewell et al, (Hopewell 2008)).

Figure 10.2.b: Publication bias in clinical trials due to statistical significance or direction of trial results (adapted from Hopewell et al. (Hopewell 2008)).

10.2.1.1 Time lag bias 10.2.1.2 Who is responsible for publication bias?

10.2.1.3 The influence of external funding and commercial interests 10.2.2 Other reporting biases

10.2.2.1 Duplicate (multiple) publication bias 10.2.2.2 Location bias 10.2.2.3 Citation bias 10.2.2.4 Language bias 10.2.2.5 Outcome reporting bias

10.3 Avoiding reporting biases 10.3.1 Implications of the evidence concerning reporting biases 10.3.2 Including unpublished studies in systematic reviews 10.3.3 Trial registries and publication bias

10.4 Detecting reporting biases 10.4.1 Funnel plots

Figure 10.4.a: Hypothetical funnel plots 10.4.2 Different reasons for funnel plot asymmetry

Table 10.4.a: Possible sources of asymmetry in funnel plots Figure 10.4.b: Contour enhanced funnel plots

10.4.3 Tests for funnel plot asymmetry Table 10.4.b: Proposed tests for funnel plot asymmetry 10.4.3.1 Recommendations on testing for funnel plot asymmetry

10.4.4 Sensitivity analyses 10.4.4.1 Comparing fixed and random-effects estimates Figure 10.4.c: Comparison of fixed- and random-effects meta-analytic estimates of the effect of

intravenous magnesium on mortality following myocardial infarction 10.4.4.2 Trim and Fill 10.4.4.3 Fail-safe N 10.4.4.4 Other selection models 10.4.4.5 Sensitivity analyses based on selection models 10.4.4.6 Testing for excess of studies with significant results

10.4.5 Summary 10.5 Chapter information

Box 10.5.a: The Cochrane Bias Methods Group. 10.6 References

11 Presenting results and ‘Summary of findings’ tables 11.1 Introduction 11.2 Characteristics of included studies tables 11.3 Data and analyses

11.3.1 The ‘Data and analyses’ section of a review 11.3.2 Forest plots

11.3.2.1 Forest plots in RevMan Box 11.3.a: Details provided in a Cochrane forest plot Figure 11.3.a: Example of a RevMan forest plot

11.3.3 Other data tables 11.4 Figures

11.4.1 Types of figures 11.4.2 Selecting RevMan analyses as figures 11.4.3 Additional figures

11.5 ‘Summary of findings’ tables 11.5.1 Introduction to ‘Summary of findings’ tables 11.5.2 Selecting outcomes for ‘Summary of findings’ tables 11.5.3 General template for ‘Summary of findings’ tables

Figure 11.5.a: Example of a ‘Summary of findings’ table 11.5.4 Producing ‘Summary of findings’ tables 11.5.5 Statistical considerations in ‘Summary of findings’ tables

11.5.6 Detailed contents of a ‘Summary of findings’ table 11.5.6.1 Table title and header 11.5.6.2 Outcomes 11.5.6.3 Illustrative comparative risks 1: Assumed risk (with control intervention) 11.5.6.4 Illustrative comparative risks 2: Corresponding risk (with experimental intervention) 11.5.6.5 Relative effect (95% CI) 11.5.6.6 Number of participants (studies) 11.5.6.7 Quality of the evidence (GRADE) 11.5.6.8 Comments

11.6 Additional tables 11.7 Presenting results in the text

11.7.1 Results of meta-analyses 11.7.2 Results without meta-analyses

11.8 Writing an abstract Box 11.8.a: Hypothetical example of an abstract

11.9 Writing a plain language summary 11.9.1 About plain language summaries 11.9.2 Plain language title 11.9.3 Summary text

11.10 Chapter information 11.11 References

12 Interpreting results and drawing conclusions 12.1 Introduction 12.2 Assessing the quality of a body of evidence

12.2.1 The GRADE approach Table 12.2.a: Levels of quality of a body of evidence in the GRADE approach Table 12.2.b: Factors that may decrease the quality level of a body of evidence Table 12.2.c: Factors that may increase the quality level of a body of evidence

12.2.2 Factors that decrease the quality level of a body of evidence Table 12.2.d: Further guidelines for factor 1 (of 5) in a GRADE assessment: Going from

assessments of risk of bias to judgments about study limitations for main outcomes 12.2.3 Factors that increase the quality level of a body of evidence

12.3 Issues in applicability 12.3.1 The role of the review author 12.3.2 Biologic variation 12.3.3 Variation in context and culture 12.3.4 Variation in adherence 12.3.5 Variation in values and preferences

12.4 Interpreting results of statistical analyses 12.4.1 Confidence intervals 12.4.2 P values and statistical significance

12.5 Interpreting results from dichotomous outcomes (including numbers needed to treat) 12.5.1 Relative and absolute risk reductions 12.5.2 More about the number needed to treat (NNT) 12.5.3 Expressing absolute risk reductions 12.5.4 Computations

12.5.4.1 Computing NNT from a risk difference (RD) 12.5.4.2 Computing absolute risk reduction or NNT from a risk ratio (RR) 12.5.4.3 Computing absolute risk reduction or NNT from an odds ratio (OR) 12.5.4.4 Computing risk ratio from an odds ratio (OR) 12.5.4.5 Computing confidence limits

12.6 Interpreting results from continuous outcomes (including standardized mean differences) 12.6.1 Meta-analyses with continuous outcomes 12.6.2 Re-expressing SMDs using rules of thumb for effect sizes

12.6.3 Re-expressing SMDs by transformation to odds ratio Table 12.6.a: NNTs equivalant to specific SMDs for various given ‘proportions improved’ in the

control group 12.6.4 Re-expressing SMDs using a familiar instrument

12.7 Drawing conclusions 12.7.1 Conclusions sections of a Cochrane review 12.7.2 Implications for practice 12.7.3 Implications for research 12.7.4 Common errors in reaching conclusions

12.8 Chapter information Box 12.8.a: Cochrane Applicability and Recommendations Methods Group

12.9 References

13 Including non-randomized studies 13.1 Introduction

13.1.1 What this chapter is about Box 13.1.a: Some types of NRS design used for evaluating the effects of interventions

13.1.2 Why consider non-randomized studies? 13.1.3 Key issues about the inclusion of non-randomized studies in a Cochrane review 13.1.4 The importance of a protocol for a Cochrane review that includes non-randomized studies 13.1.5 Structure of subsequent sections in the chapter

13.2 Developing criteria for including non-randomized studies 13.2.1 What is different when including non-randomized studies?

13.2.1.1 Including both randomized and non-randomized studies 13.2.1.2 Evaluating benefits and harms 13.2.1.3 Determining which types of non-randomized study to include 13.2.1.4 Distinguishing between aetiology and effectiveness research questions

13.2.2 Guidance and resources available to support review authors 13.2.3 Summary

Table 13.2.a: List of study design features (studies with allocation to interventions at the individual level)

Table 13.2.b: List of study design features (studies with allocation to interventions at the group level)

13.3 Searching for non-randomized studies 13.3.1 What is different when including non-randomized studies?

13.3.1.1 Comprehensiveness of search strategy 13.3.1.2 Identifying NRS in searches 13.3.1.3 Reviewing citations / abstracts

13.3.2 Guidance and resources available to support review authors 13.3.3 Summary

13.4 Selecting studies and collecting data 13.4.1 What is different when including non-randomized studies? 13.4.2 Guidance and resources available to support review authors

Box 13.4.a: User guide for data collection/study assessment using checklist in Table 13.2.a or Table 13.2.a

13.4.3 Summary 13.5 Assessing risk of bias in non-randomized studies

13.5.1 What is different when including non-randomized studies? 13.5.1.1 Sources of bias in non-randomized studies 13.5.1.2 Evidence of risk of bias in non-randomized studies

13.5.2 Guidance and resources available to support review authors 13.5.2.1 General considerations in assessing risk of bias in non-randomized studies 13.5.2.2 Confounding and adjustment 13.5.2.3 Tools for assessing methodological quality or risk of bias in non-randomized studies 13.5.2.4 Practical limitations in assessing risk of bias in non-randomized studies

13.5.3 Summary 13.6 Synthesis of data from non-randomized studies

13.6.1 What is different when including non-randomized studies? 13.6.2 Guidance and resources available to support review authors

13.6.2.1 Controlling for confounding 13.6.2.2 Combining studies 13.6.2.3 Analysis of heterogeneity 13.6.2.4 When pooling is judged not to be appropriate

13.6.3 Summary 13.7 Interpretation and discussion

13.7.1 Challenges in interpreting Cochrane reviews of effectiveness that include non-randomized studies

13.7.1.1 Have all important and relevant studies been included? 13.7.1.2 Has the risk of bias to included studies been adequately assessed?

13.7.2 Evaluating the strength of evidence provided by reviews that include non-randomized studies 13.7.3 Guidance for potential review authors

13.8 Chapter information Box 13.8.a: The Cochrane Non-Randomised Studies Methods Group

13.9 References

14 Adverse effects 14.1 Introduction

14.1.1 The need to consider adverse effects 14.1.2 Concepts and terminology 14.1.3 When it is most important to consider adverse effects

Table 14.1.a: Contexts and examples warranting detailed examination of adverse effects 14.2 Scope of a review addressing adverse effects

14.2.1 Identical methods for beneficial and adverse effects 14.2.2 Different methods for beneficial and adverse effects 14.2.3 Separate review for adverse effects

14.3 Choosing which adverse effects to include 14.3.1 Narrow versus broad focus 14.3.2 Withdrawal or drop-out as an outcome measure for adverse effects

14.4 Types of studies 14.5 Search methods for adverse effects

14.5.1 Sources of information on adverse effects of drugs 14.5.2 Search strategy for adverse effects

14.5.2.1 Searching electronic databases for adverse effects using index terms 14.5.2.2 Searching electronic databases for adverse effects using free-text terms

14.6 Assessing risk of bias for adverse effects 14.6.1 Clinical trials 14.6.2 Case-control and cohort studies 14.6.3 Case reports

14.7 Chapter information Box 14.7.a: The Cochrane Adverse Effects Methods Group

14.8 References

15 Incorporating economics evidence 15.1 The role and relevance of economics evidence in Cochrane reviews

15.1.1 Introduction Box 15.1.a: Archie Cochrane on health economics (Cochrane 1972)

15.1.2 Economics and economic evaluation Box 15.1.b: Types of full economic evaluation

15.1.3 Coverage of economics issues in Cochrane reviews 15.2 Planning the economics component of a Cochrane review

15.2.1 Formulating an economics question Figure 15.2.a: Clinical event pathways Box 15.2.a: Background commentary highlighting economics aspects of interventions

15.2.2 Including measures of resource use, costs and cost-effectiveness as outcomes 15.2.3 Specifying types of health economics studies and the scope of the economics component of a

review 15.3 Locating studies

15.3.1 Use of electronic search filters 15.3.2 Use of specialist databases

15.4 Selecting studies and collecting data 15.4.1 Assessing relevance to the review topic 15.4.2 Collecting data

15.5 Addressing risk of bias 15.5.1 Classification of studies by study design 15.5.2 Critical appraisal of methodological quality

Figure 15.5.a: Drummond checklist (Drummond 1996) Figure 15.5.b: Evers checklist (Evers 2005)

15.6 Analysing and presenting results 15.6.1 Presenting results in tables 15.6.2 Narrative summary of results 15.6.3 Meta-analysis of resource use and cost data

Box 15.6.a: Statement of a decision not to conduct a meta-analysis of resource use or cost data 15.6.4 Developing an economic model

15.7 Addressing reporting biases 15.8 Interpreting results

Box 15.8.a: Highlighting a need for further economics studies in conclusions 15.9 Conclusions 15.10 Chapter information

Box 15.10.a: The Campbell and Cochrane Economics Methods Group 15.11 References

16 Special topics in statistics 16.1 Missing data

16.1.1 Types of missing data Table 16.1.a: Types of missing data in a meta-analysis

16.1.2 General principles for dealing with missing data 16.1.3 Missing standard deviations

16.1.3.1 Imputing standard deviations 16.1.3.2 Imputing standard deviations for changes from baseline

16.2 Intention-to-treat issues 16.2.1 Introduction 16.2.2 Intention-to-treat issues for dichotomous data 16.2.3 Intention-to-treat issues for continuous data 16.2.4 Conditional outcomes only available for subsets of participants

16.3 Cluster-randomized trials 16.3.1 Introduction 16.3.2 Assessing risk of bias in cluster-randomized trials 16.3.3 Methods of analysis for cluster-randomized trials 16.3.4 Approximate analyses of cluster-randomized trials for a meta-analysis: effective sample sizes 16.3.5 Example of incorporating a cluster-randomized trial 16.3.6 Approximate analyses of cluster-randomized trials for a meta-analysis: inflating standard

errors 16.3.7 Issues in the incorporation of cluster-randomized trials 16.3.8 Individually randomized trials with clustering

16.4 Cross-over trials

16.4.1 Introduction 16.4.2 Assessing suitability of cross-over trials 16.4.3 Assessing risk of bias in cross-over trials 16.4.4 Methods of analysis for cross-over trials 16.4.5 Methods for incorporating cross-over trials into a meta-analysis 16.4.6 Approximate analyses of cross-over trials for a meta-analysis

Table 16.4.a: Some possible data available from the report of a cross-over trial 16.4.6.1 Mean differences 16.4.6.2 Standardized mean difference 16.4.6.3 Imputing correlation coefficients 16.4.6.4 Example

16.4.7 Issues in the incorporation of cross-over trials 16.5 Studies with more than two intervention groups

16.5.1 Introduction 16.5.2 Determining which intervention groups are relevant 16.5.3 Assessing risk of bias in studies with more than two groups 16.5.4 How to include multiple groups from one study 16.5.5 Heterogeneity considerations with multiple-intervention studies 16.5.6 Factorial trials

16.6 Indirect comparisons and multiple-treatments meta-analysis 16.6.1 Introduction 16.6.2 Indirect comparisons 16.6.3 Multiple-treatments meta-analysis

16.7 Multiplicity and the play of chance 16.7.1 Introduction 16.7.2 Multiplicity in systematic reviews

16.8 Bayesian and hierarchical approaches to meta-analysis 16.8.1 Bayesian methods 16.8.2 Hierarchical models

16.9 Rare events (including zero frequencies) 16.9.1 Meta-analysis of rare events 16.9.2 Studies with zero-cell counts 16.9.3 Studies with no events 16.9.4 Confidence intervals when no events are observed 16.9.5 Validity of methods of meta-analysis for rare events

16.10 Contributions to this chapter 16.11 References

17 Patient-reported outcomes 17.1 What are patient-reported outcomes?

Figure 17.1.a: Classification of clinical trial outcomes with illustration of those most important to patients

17.2 Patient-reported outcomes and Cochrane reviews 17.3 Health status and quality of life as PRO outcomes

Table 17.3.a: Definitions of selected terms related to quality of life Table 17.3.b: A taxonomy of health status and quality of life measures adapted from Patrick and

Erickson (Patrick 1993). 17.4 Issues in the measurement of patient-reported outcomes

17.4.1 Validity of instruments 17.4.2 Ability of an instrument to measure change

17.5 Locating and selecting studies with patient-reported outcomes 17.6 Assessing and describing patient-reported outcomes

Table 17.6.a: A checklist for describing and assessing PROs in clinical trials 17.7 Comparability of different patient-reported outcome measures 17.8 Interpreting Results

17.8.1 Study summaries focusing on a single patient-reported outcome 17.8.2 Study summaries using more than one patient-reported outcome 17.8.3 When studies do not address patient-reported outcomes

17.9 Chapter information Box 17.9.a: The Cochrane Patient Reported Outcomes Methods Group

17.10 References

18 Reviews of individual patient data 18.1 Introduction

18.1.1 What is an IPD review? 18.1.2 When should an IPD review be done?

Box 18.1.a: Potential benefits of IPD 18.1.3 How are IPD review methods different? 18.1.4 How are IPD reviews organized? 18.1.5 What healthcare areas have used the IPD approach? 18.1.6 If I am thinking about doing an IPD review, what should I do first?

18.2 The collaborative nature of IPD meta-analyses 18.2.1 Collaborative groups 18.2.2 Negotiating collaboration 18.2.3 Confidentiality

18.3 Dealing with data 18.3.1 Deciding what data to collect 18.3.2 Data format 18.3.3 Re-coding and re-defining supplied variables 18.3.4 Checking data supplied

18.3.4.1 Checking for missing or duplicated data 18.3.4.2 Checking plausibility 18.3.4.3 Checking randomization 18.3.4.4 Checking information is up to date

18.4 Analysis 18.4.1 Analysis advantages 18.4.2 General approach 18.4.3 Time-to-event analyses 18.4.4 Bringing analyses up to date: long-term outcomes 18.4.5 Subgroup analysis 18.4.6 Additional analyses 18.4.7 Software

18.5 Limitations and Caveats 18.5.1 What an IPD review cannot fix 18.5.2 Unavailable studies 18.5.3 Deciding when an IPD review is appropriate

18.6 Chapter information Box 18.6.a: The Cochrane Individual Patient Data Meta-analysis Methods Group.

18.7 References

19 Prospective meta-analysis 19.1 Introduction

19.1.1 What is a prospective meta-analysis (PMA)? 19.1.2 What is the difference between a PMA and a large multi-centre trial? 19.1.3 What healthcare areas have used the PMA approach? 19.1.4 What resources do I need?

19.2 The collaborative nature of PMA 19.2.1 Collaborative groups 19.2.2 Negotiating collaboration 19.2.3 Confidentiality

19.3 The PMA protocol 19.3.1 What should the protocol contain?

Box 19.3.a: Elements of a PMA protocol 19.3.2 Publication of the protocol

19.4 PMA data collection 19.5 PMA analysis issues

19.5.1 General approach 19.5.2 Interim analysis and data monitoring

19.6 Chapter information Box 19.6.a: The Cochrane Prospective Meta-analysis Methods Group

19.7 References

20 Qualitative research and Cochrane reviews 20.1 Introduction 20.2 Incorporating evidence from qualitative research in Cochrane Intervention reviews: concepts and

issues 20.2.1 Definition of qualitative research 20.2.2 Using evidence from qualitative research in Cochrane reviews 20.2.3 Considering qualitative studies that are identified within, or alongside, randomized controlled

trials. 20.2.4 Resource considerations

20.3 Qualitative evidence synthesis 20.3.1 Exemplar of synthesizing qualitative evidence to supplement a Cochrane Intervention review:

Directly Observed Therapy and Tuberculosis (TB) Box 20.2.a: Directly observed therapy and tuberculosis: a synthesis of qualitative evidence-

summary 20.3.2 Methodological issues 20.3.2.1 Search strategies 20.3.2.2 Critical appraisal 20.3.2.3 Synthesizing evidence from qualitative research 20.3.2.4 Choosing an appropriate method 20.3.2.5 Approaches to integrating qualitative and quantitative evidence syntheses 20.3.2.6 Conclusion

20.4 Contributions to this Chapter Box 20.4.a: The Cochrane Qualitative Research Methods Group

20.5 References 20.6 Further Selected Reading

20.6.1 Qualitative Research – general 20.6.2 Qualitative Methods 20.6.3 Qualitative Literature Searching 20.6.4 Synthesizing Qualitative Evidence 20.6.5 Synthesizing qualitative and quantitative evidence 20.6.6 Critical appraisal of qualitative studies 20.6.7 Links

21 Reviews in health promotion and public health 21.1 Introduction 21.2 Study designs to include 21.3 Searching 21.4 Assessment of study quality and risk of bias 21.5 Ethics and inequalities 21.6 Context 21.7 Sustainability 21.8 Applicability and transferability 21.9 Contributions to this chapter

21.10 References

22 Overviews of reviews 22.1 Introduction

22.1.1 Definition of Cochrane Overviews of reviews 22.1.2 Rationale for Cochrane Overviews

Table 22.1.a: Reasons for overviewing reviews and their suitability for publication as a Cochrane Overview

22.2 Preparing a Cochrane Overview of reviews 22.2.1 Organizational issues 22.2.2 Methodological issues

Table 22.2.a: Comparison of methods between Cochrane Intervention reviews and Cochrane Overviews of reviews

22.2.3 Updating Cochrane Overviews 22.3 Format of a Cochrane Overview

22.3.1 Title and review information (or protocol information) 22.3.2 Abstract 22.3.3 Plain language summary 22.3.4 Text of a Cochrane Overview 22.3.5 Review and references

22.3.5.1 References to reviews 22.3.5.2 Other references

22.3.6 Tables 22.3.6.1 ‘Characteristics of included reviews’ table

Figure 22.3.a: Template for a ‘Characteristics of included reviews’ table 22.3.6.2 ‘Overview of reviews’ table

Figure 22.3.b: Template for an ‘Overview of reviews’ Table 22.3.6.3 Other tables

22.3.7 Figures Figure 22.3.c: Example of a ‘forest top plot’ comparing interventions for enuresis in children. This

example was prepared using Microsoft Excel. 22.4 Chapter information 22.5 References

Index absolute risk reduction (ARR) see risk difference abstracts 4:4, 11:15-11:17 adverse effects 14:8 non-randomized studies 13:13 Overviews of reviews 22:8-22:9 searching 6:3, 6:13, 6:40, 6:41, 13:4, 14:8 selecting studies 7:3 accumulation of changes 3:7 acknowledgements 4:15, 22:15 ACR see assumed control risks Additional tables 4:18, 8:43, 11:13, 15:16, 17:8 adherence 7:8, 12:9 adverse effects 14:1-14:13 data collection 7:9-7:10 definitions 14:2 drop-outs 14:5 eligibility criteria 14:3-14:5 narrow/broad focus 14:4-14:5 non-randomized studies 13:5-13:6, 13:7,

13:29, 14:5, 14:9-14:11 review questions 5:6-5:7 risk of bias/quality 14:8-14:11 searching 14:6-14:8 separate reviews 14:4 sources of information 14:6 strategies for addressing 14:3-14:4 Summary of findings tables 11:12 types of studies 14:6, 14:8-14:11 Adverse Effects Methods Group (AEMG) 14:11 advisory groups 2:7-2:8 AEMG see Adverse Effects Methods Group aetiological research questions 13:7 agreement measures 7:4-7:5, 7:15 allied health-specific databases 6:1-6:2 allocation concealment 8:6, 8:24, 8:27-8:29 results sections 4:12 Risk of bias tool 8:9, 8:12-8:13, 8:17-8:19 amendments 3:2, 3:5-3:6 analysing data see meta-analysis analysis of covariance (ANCOVA) 9:21 analysis plans 4:6, 4:9-4:11, 9:2-9:3 19:5, 19:7 prospective meta-analyses 19:5, 19:6-19:7 ANCOVA see analysis of covariance appendices 4:23 search strategies 6:41, 6:42 applicability discussion sections 4:13-4:14 interpreting results 12:8-12:9 Overviews of reviews 22:14 public health and health promotion 21:8-

21:9

Applicability and Recommendations Methods Group (ARMG) 12:20

ARR (absolute risk reduction) see risk difference as-treated analyses 8:36 assessing a review as up to date 3:8-3:9, 3:11,

4:3, 22:16 assumed control risks (ACR) 11:11, 11:12,

12:13, 12:14-12:15 attrition bias 8:8-8:9, 8:13-8:14, 8:31-8:37 authors’ conclusions see conclusions authorship 3:5, 3:7, 3:8, 4:2-4:3 available case analysis 16:7, 16:8, 16:9-16:10 background sections 4:5-4:6, 22:9-22:10 baseline imbalance 8:41 baseline risk see control group risk Bayesian methods 8:24, 16:29-16:31 before-and-after studies 13:2, 21:4 bias see publication bias; reporting bias; risk of bias Bias Methods Group (BMG) 10:24 bibliographic databases 6:3-6:10 binary outcome data see dichotomous outcome data biologic variation 12:9 biology-specific databases 6:7 blinding 8:6, 8:29-8:31 results sections 4:12 review authors 7:14, 8:5, 12:19 Risk of bias tool 8:7-8:9, 8:12, 8:17-8:19,

8:29-8:31 blocked randomization 8:7, 8:26, 8:41 BMG see Bias Methods Group Boolean operators 6:31, 6:39 broad questions 5:8-5:9 Campbell and Cochrane Economics Methods

Group (CCEMG) 15:4, 15:12, 15:17, 15:22, 17:12

Campbell Collaboration 8:43 Campbell Coordinating Groups 20:11 Campbell Equity Methods Group 21:5 carry-over effects 16:15-16:18 case reports 14:6, 14:10-14:11 case series 13:3, 14:6 case-control studies 13:3, 13:7, 13:20-13:21,

14:6, 14:9 CBA see cost-benefit analysis CBT see cognitive behavioural therapy CCEMG see Campbell and Cochrane Economics Methods Group CCTs see controlled clinical trials

CDSR see Cochrane Database of Systematic Reviews CEA see cost-effectiveness analysis CENTRAL see Cochrane Central Register of Controlled Trials central randomization 8:29 Centre for Reviews and Dissemination (CRD)

6:14, 15:9-15:11, 15:12 CGR see control group risk change scores 7:17, 9:21-9:22 standard deviations 16:4-16:7 changing review questions 5:9-5:10 Characteristics of studies tables 4:11-4:12, 4:17-

4:18 economics evidence 15:16 excluded studies 4:12, 4:17, 7:4 included studies 4:11-4:12, 4:17, 11:2,

15:26 ongoing studies 4:12, 4:18 Overviews of reviews 22:12, 22:16-22:17 studies awaiting classification 14:12, 14:18 checklists 8:4 chi-squared tests 9:28, 9:31, 9:33, 11:4-11:5 CINAHL database 6:8, 20:7 citation bias 10:2, 10:8-10:9 indexes 6:9 versions 3:3-3:5, 3:6-3:8, 4:4 citing protocols 4:6 reviews 2:11-2:12 clinical event pathways 15:6-15:7 clinical trials see randomized trials cluster-allocated studies 13:16-13:17 see also cluster-randomized trials clustering 16:14 cluster-randomized trials 9:14, 16:10-16:14 effective sample sizes 16:12-16:13 inflating standard errors 16:13-16:14 meta-analysis 16:12-16:14 methods of analysis 16:12 public health and health promotion 21:2-

21:3, 21:6 risk of bias 16:11-16:12 sensitivity analyses 9:38 CMR see Cochrane Methodology Register Cochrane, Archie 1:2, 15:2 Cochrane Central Register of Controlled Trials

(CENTRAL) 1:3, 6:3-6:5, 6:11, 6:19-6:28, 6:31, 6:35, 6:37-6:39,13:13

Cochrane Collaboration 1:1-1:3 maintenance of Cochrane reviews 3:2 preparation of Cochrane reviews 2:1, 2:6,

2:11-2:14 principles 1:2, 1:5 publication of Cochrane reviews 1:3

structure 1:2 Cochrane Database of Systematic Reviews

(CDSR) 1:3 citation versions 3:3-3:5 citing 2:11-2:13 dates 3:8, 4:3-4:4 Overviews of reviews 22:2 prospective meta-analyses 19:7 protocols 2:2-2:3, 2:5-2:6 searching see Cochrane Library withdrawn reviews 3:13 Cochrane Health Equity Field 21:5 Cochrane Library 1:3 Cochrane Review Groups 2:9-2:11 economics evidence 15:10 feedback 3:15 Overviews of reviews 22:2 searching 6:4-6:5, 6:21, 6:22, 6:25-6:26,

6:32 Cochrane Methodology Register (CMR) 1:3 Cochrane Review Groups (CRGs) 1:2, 1:4 economics evidence 15:4 guidance and support 2:5-2:6, 2:8-2:11,

2:14, 4:8, 4:15, 5:8 maintenance of Cochrane reviews 3:2, 3:5,

3:6-3:7,3:9-3:10,3:12-3:13, 3:15 non-randomized studies 13:6, 13:7-13:9,

13:13-13:14 Overviews of reviews 22:1-22:2, 22: plain language summaries 11:17-11:18 prospective meta-analyses 19:7 public health and health promotion 21:4 qualitative research 20:5, 20:10-20:11 searching 4:8-4:9, 6:2-6:3, 6:7, 6:11, 6:20,

6:23-6:25, 6:29, 6:41 statisticians 9:39-9:40 Cochrane reviews Diagnostic test accuracy 1:4, 2:8 Intervention 1:3-1:5, 2:2-2:4 Methodology 2:8 Overviews of reviews 2:8, 22:1-22:21 coding schemes 7:13 cognitive behavioural therapy (CBT) 17:4 cohort studies 13:2, 13:7, 13:20,14:6, 14:9 collaborations individual patient data 18:1, 18:2-18:3 prospective meta-analyses 19:4 see also Cochrane Collaboration combining studies see meta-analysis combining subgroups/intervention groups 7:21 comments and criticisms see feedback commercial interests 2:13-2:14, 10:6-10:7, 22:15 community healthcare-specific databases 6:8 comparisons 4:20-4:21, 5:4, 9:2-9:3, 9:4-9:5 completeness of evidence 4:13-4:14

concealment of allocation see allocation concealment conclusions 4:14, 12:18-12:20 abstract 11:16, 11:17 changed 3:4, 3:6-3:8 economics evidence 15:21-15:22 Overviews of reviews 22:9, 22:14-22:15 qualitative research 20:6-20:7 conditional outcomes 16:10 conference abstracts/proceedings 6:13 confidence intervals 12:9-12:11 continuous outcome data 7:18-7:20, 12:10 data extraction 7:18-7:20, 7:25-7:26 dichotomous outcome data 7:16, 12:10 forest plots 11:4 rare events 16:33 Summary of findings tables 11:13 confidentiality 18:3-18:4, 19:4 conflicts of interest see declarations of interest confounding GRADE/quality of evidence 12:4, 12:7 heterogeneity 9:35 indirect comparisons 16:26 non-randomized studies 13:4, 13:6, 13:18,

13:21, 13:24-13:25, 13:28-13:29 public health and health promotion 21:2 consensus 7:15 consistency see heterogeneity consumer involvement 2:6-2:7 contact persons 4:3 contents of Cochrane reviews 2:2-2:4, 4:1-4:24 abstracts 4:3 appendices 4:23 data and analyses 4:20-4:21 feedback 4:23 figures 4:21-4:22 plain language summaries 4:4-4:5 protocol information 4:1-4:5 review information 4:1-4:5 studies and references 4:19-4:20 tables 4:16-4:17 text 4:5-5:17 titles 4:2 context 12:9, 21:6-21:7 continuous outcome data 9:6, 9:10-9:122 confidence intervals 7:17-7:20, 12:10 counts 7:23 data extraction 7:16-7:22, 7:23-7:24 forest plots 11:3-11:5 funnel plots 10:19 individual patient data 18:6,18:7 intention to treat 16:9-16:10 interpreting results 12:16-12:17 meta-analysis 9:20-9:23, 9:38 non-randomized studies 13:25 Overviews of reviews 22:18

results sections 11:3-11:5, 11:12, 12:10, 12:16-12:17

risk of bias 8:14, 8:35, 8:38 Summary of findings tables 11:12 see also patient-reported outcomes contour-enhanced funnel plots 10:14-10:16 contributions of authors 4:15-4:16 control group risk (CGR) effect measures 9:6-9:9 meta-regression 9:36-9:37 interpreting results 11:10-11:11, 11:12-

11:13, 12:13, 12:15 see also assumed control risk controlled before-and-after studies 13:2 vocabulary 6:29-6:30, 14:7, 17:7-17:8 controlled clinical trial (CCT) 6:21-6:24, 6:26 copyright 6:3 correlation coefficients 16:5-16:7, 16:13-16:14,

16:20-16:21 correspondence with investigators 7:11, 7:13, 8:6 cost data 15:18-15:19 cost-benefit analysis (CBA) 15:2, 15:3, 15:7 cost-effectiveness analysis (CEA) 15:2, 15:3,

15:7,15:20 cost-utility analysis (CUA) 15:2, 15:3 count data 9:6, 9:12-9:13 data extraction 7:22-7:23 meta-analysis 9:24-9:25, 16:32 zero-cell 16:32 covariates 9:33-9:34, 13:26 Cox model 7:24, 9:25, 13:25 CRD see Centre for Reviews and Dissemination CRGs see Cochrane Review Groups critical appraisal qualitative research 20:8, 20:16-20:17 see also quality, risk of bias cross-over trials 9:14, 16:14-16:21 eligibility criteria 16:15 meta-analysis 16:17-16:21 methods of analysis 16:17 risk of bias 16:15-16:17 sensitivity analyses 9:38 unit of analysis errors 9:15, 16:17 cross-sectional studies 13:3, 13:20 CUA see cost-utility analysis cultural factors 12:2, 12:9 DARE see Database of Abstracts of Reviews of Effects data collection 7:5-7:13 abstracts 11:15, 11:16 economics evidence 15:11-15:12,

15:14-15:15 forms 7:11-7:13 individual patient data 18:4-18:5

interventions 7:-7:6, 7:8-7:9 methods of studies 7:5-7:6, 7:7 methods sections 4:9-4:11 non-randomized studies 13:14-14:17 outcomes 7:5-7:6, 7:9-7:10 Overviews of reviews 22:6-22:7, 22:11 participants 7:5-7:7 prospective meta-analyses 19:6-19:7 results 7:5-7:6, 7:10 risk of bias 8:5-8:6 sources of data 7:5-7:6, 7:11 sustainability 21:7 entry 7:16, 7:20, 7:24-7:26 extraction 7:13-7:26 consensus 7:15 continuous outcome data 7:16-7:22,

7:23 conversion to desired format 7:16-7:26 counts 7:22-7:23 dichotomous outcome data 17:16, 7:23 estimates of effects 7:24-7:26 multiple reports 7:14-7:15 ordinal outcome data 7:22 Overviews of reviews 22:11 results 7:16-7:26 time-to-event data 7:23-7:24 management 7:26 individual patient data 18:4-18:5 Overviews of reviews 22:11 prospective meta-analyses 19:6, 19:7,

19:8 synthesis see meta-analysis time to event see time-to-event data types 9:6-9:14 see also continuous outcome data; count

data; dichotomous outcome data; ordinal outcome data; rate data; time-to-event data

Data and analysis section 4:20-4:21, 11:3-11:5, 11:14

Database of Abstracts of Reviews of Effects (DARE) 1:3, 6:14

dates 3:8-3:10, 4:3-4:4, 6:32-6:33 declarations of interest 2:13-2:14, 4:16, 22:15 derivative products 2:13 DerSimonian and Laird method see random-effects methods descriptions of condition/intervention 4:6 design-specific risk of bias 8:40 detection bias 8:7 diagnostic test accuracy 1:4 dichotomous outcome data 9:6-9:10 confidence intervals 7:16, 12:9-12:10 counts 7:23 data extraction 7:16, 7:23 forest plots 11:4

funnel plots 10:12, 10:19 individual patient data 18:6, 18:7 intention to treat 16:8-16:9 interpreting results 12:12-12:16 meta-analysis 9:18-9:20, 9:22-9:23, 16:31-

16:34 non-randomized studies 13:25 patient-reported outcomes 17:10 risk of bias 8:14, 8:22, 8:34, 8:38 Summary of findings tables 11:10-11:11,

11:13 difference measures 7:25, 9:9, 9:10-9:11, 12:12-

12:16 difference in means 9:10-9:11, 12:16-12:17 cross-over trials 16:18-16:19 funnel plots 10:19-10:20 meta-analysis 9:20-9:23 differential diagnostic activity 8:42 directly observed therapy (DOT) 20:5-20:7 directness see indirect evidence discussion sections 4:13-4:15, 22:13-22:14 dissertations databases 6:6-6:10 dose-response analyses 9:37, 12:4, 12:7-12:8 DOT see directly observed therapy double-blinding see blinding Downs and Black instrument 13:22 drop-outs see incomplete outcome data Drummond checklist 15:13-15:15 duplicated data see multiple reports early stopping 8:40-8:41 economics evidence 15:1-15:25 clinical event pathways 15:6 data analysis 15:14-15:19 data collection 15:11-15:12, 15:14-15:15 eligibility criteria 15:8-15:9, 15:11, 15:4 evaluation types 15:2-15:4 meta-analysis 15:18-15:19 model development 15:19 outcomes 15:7 reporting bias 15:20 review questions 5:7, 15:5-15:7 risk of bias/quality 15:12-15:16, 15:17 role and relevance 15:1-15:4 searching 15:9-15:11 strategies for addressing 15:8-15:9 types of evidence 15:2-15:4, 15:8-15:9,

15:12-15:16 editorial procedures 2:9, 3:13 education-specific databases 6:8 effect estimates 7:24-7:26, 9:16-9:18 measures 4:9, 9:6-9:14, 9:29 modification 9:32 size 9:11

Effective Practice and Organisation of Care (EPOC) Group 21:4

effectiveness 13:7, 21:4-21:6 Effectiveness and Efficiency (Cochrane) 15:2 effects of interventions section 14:13 Electronic Information for Libraries (eIFL) 6:4,

6:12 eligibility criteria 5:1-5:9, 7:1-7:4 abstracts 11:15-11:16 adverse effects 14:3-14:5 cross-over trials 16:15 data collection 7:5, 7:11 economics evidence 15:8, 15:11, 15:13-

15:14 excluded studies 7:4 measures of agreement 7:4 methods sections 4:7 multiple reports 7:2 non-randomized studies 13:5-13:8, 13:14,

13:28 Overviews of reviews 22:3-22:5, 22:7,

22:10-22:11, 22:15 Cochrane Review Groups 2:10 prospective meta-analyses 19:1-19:2, 19:5,

19:6 qualitative research 20:6 references 4:19 searching 6:28 selection process 7:1-7:5 sensitivity analyses 9:38 studies versus reports 7:2 unpublished studies 10:10-10:11 updates and amendments 3:11 EMBASE 6:3-6:6, 6:2 adverse effects 14:7-14:8 CENTRAL 16:31 economics evidence 15:10 non-randomized studies 13:12 qualitative research 20:7 reporting bias 10:8 searching for reviews 6:14 searching for studies 6:25-6:30, 6:37 EPICOT acronym 12:18 EPOC see Effective Practice and Organisation of

Care EPPI see Evidence for Policy and Practice

Information errata 3:6-3:7, 6:33-6:34 estimates of effect see effect estimates ethics 8:43, 21:4-21:6 Evers checklist 15:14, 15:15 Evidence for Policy and Practice Information

(EPPI) Centre 20:8, 20:9, 20:17 excluded studies 7:4 Overviews of reviews 22:15 reporting bias 10:10-10:11

results sections 4:12 sensitivity analyses 9:29-9:30, 9:37 table 4:17, 4:17-4:18 see also eligibility criteria exclusions 8:6, 8:9, 8:13-8:14, 8:31-8:33 see also intention to treat explanatory variables 9:33-9:34 external funding 4:23, 10:6-10:7 factorial trials 16:21-16:22, 16:25-16:26 fail-safe N 10:22 false negatives 9:32 false positives 9:32-9:33, 10:19 feedback 3:15, 4:22 figures 4:21-4:22, 11:15-11:17 Overviews of reviews 22:20 risk of bias 8:19-8:21, 8:21 see also forest plots; funnel plots file drawer problem 10:3 first published dates 4:4 fixed-effect method 9:16, 9:29 confidence intervals 9:29, 12:9-12:11 forest plots 11:3-11:5 reporting bias 10:20-10:21 RevMan 9:17, 9:20-9:21, 9:25 sensitivity analyses 9:38, 10:20-10:21 forest plots 4:22, 11:3-11:5 figures 11:5-11:6 heterogeneity 9:28, 9:33 non-randomized studies 13:26-13:27 results sections 11:14 risk of bias 8:21 see also forest top plots forest top plots 11:6, 22:20 format of Cochrane reviews 2:2-2:4 see also contents of Cochrane reviews fraudulent studies 6:33 free-text terms 6:29-6:30, 14:8, 17:7 full economic evaluations 15:2-15:4 full-text electronic journals 6:11-6:12 full-text reports 7:2 functional status 17:3 funding preparation of Cochrane reviews 2:11 risk of bias 8:42 funnel plots 10:11-10:20 figures 4:22, 11:6 non-randomized studies 13:12 tests for asymmetry 10:16-10:20 GDP see gross domestic product generic inverse-variance outcome data 9:17-9:18 cluster-randomized trials 16:12, 16:13-

16:14 combining dichotomous and continuous outcome data 9:23

counts and rates 9:25 cross-over trials 16:18-16:20, 16:21 data extraction 7:24-7:25 forest plots 11:5 non-randomized studies 13:26-13:27 ordinal outcome data 9:24 RevMan 9:17, 9:25, 11:3 see also inverse-variance meta-analysis GRADE 11:13, 12:3-12:4, 12:20 non-randomized studies 13:29 Overviews of reviews 22:12, 22:13, 22:18-

22:19 GRADEprofiler software 11:9-11:10 graphs see figures grey literature databases 6:10-6:11, 6:19, 6:42 reporting bias 10:10-10:11 gross domestic product (GDP) 15:16 group-allocated studies see cluster-allocated

studies handsearching 6:11, 6:21 harms see adverse effects hazard ratio 9:13, 9:25, 13:26 headings 4:5 health equity 21:4-21:6 Health InterNetwork Access to Research Initiative (HINARI) 6:4, 6:12 health promotion-specific databases 6:7 see also public health and health promotion health status 17:3-17:4 Health Technology Assessment Database (HTA)

6:14 health-related quality of life (HRQOL) 17:3-

17:6, 17:7, 17:10-17:12 health-specific databases 6:6-6:9 healthcare librarians 6:20 heterogeneity 9:27-9:37 addressing 9:28-9:30 analysis plans 9:5, 9:34, 9:36 baseline risk 9:36-9:37 definitions 9:27 dichotomous outcome data 9:19-9:20 dose-response analyses 9:37 economics evidence 15:17-15:18 forest plots 11:4-11:5 GRADE/quality of evidence 12:4, 12:5 I-squared 9:28, 9:31 identification and measurement 9:28 interaction and effect modification 9:32 meta-regression 9:33-9:37 methods sections 4:10 multi-arm studies 16:24-16:25 non-randomized studies 13:18, 13:23,

13:26, 13:28 Overviews of reviews 22:18

rare events 16:14 reporting bias 10:15, 10:19 subgroup analyses 9:32-9:36 tau-squared 9:30, 10:19, 11:4-11:5 testing 9:28, 9:30-9:31 see also random-effects method hierarchical models 16:21 highly sensitive search strategies 6:33-6:37, 6:39 HINARI see Health InterNetwork Access to

Research Initiative historically controlled studies 13:2 History tables 3:13-3:15, 4:4 homogeneity see heterogeneity HRQOL see health-related quality of life HTA see Health Technology Assessment

Database I-squared 9:28, 9:31 ICC see intracluster correlation coefficients ICER see incremental cost-effectiveness ratio ICMJE see International Committee of Medical

Journal Editors identifiers 4:19, 6:40, 7:6, 7:13 implications for practice/research 4:14-4:15,

12:18-12:19 imputations change from baseline 9:22, 16:4-16:7 cluster-randomized trials 16:13 cross-over trials 16:18, 16:20-16:21 intention to treat 8:36-8:37, 16:3, 16:9-

16:10 standard deviations 16:4-16:7 IMRG see Information Retrieval Methods Group IMS see Information Management System incomplete outcome data 8:6, 8:31-8:37 attrition/drop-out 8:6, 8:32, 14:5 exclusions 8:6, 8:9, 8:13-8:14 results sections 4:13 Risk of bias tool 8:7, 8:9, 8:13-8:14, 8:31-

8:37 see also intention to treat; missing data;

selective outcome reporting incremental cost-effectiveness ratio (ICER) 15:7 index terms see controlled vocabulary indirect comparisons 12:5, 16:26-16:28, 22:12,

22:20 indirect evidence 12:4, 12:5 individual patient data (IPD) 7:11, 18:1-18:4 benefits and limitations 18:2, 18:6, 18:8 collaborations 18:3, 18:3-18:4 confidentiality 18:3-18:4 data collection 7:11-7:13, 8:4 meta-analysis 16:31, 18:6 prospective meta-analyses 19:1-19:4 individual patient data meta-analysis methods

group (IPD MA MG) 18:3, 18:7, 18:8-18:9

inequalities 21:4-21:5 Information Management System (IMS) 2:8 Information Retrieval Methods Group (IRMG)

6:43, 13:14 integrity of interventions 5:6, 7:8-7:9, 21:8-21:9 intention to treat (ITT) 8:32-8:33, 16:7-16:10 interaction 9:32 interim analyses 8:41, 19:8 International Committee of Medical Journal

Editors (ICMJE) 10:11 interpreting results 12:1-12:24 continuous outcome data 12:16-12:17 dichotomous outcome data 12:12-12:16 economics evidence 15:21 meta-regression 9:35-9:36 non-randomized studies 13:27-13:28 patient-reported outcomes 17:10-17:11 subgroup analyses 9:35-9:36 interquartile ranges 7:21 interrupted-time-series studies 13:2-13:3 interventions background sections 4:5 Characteristics of studies tables 11:2 data collection 7:6, 7:8-7:9 eligibility criteria 4:7, 5:2-5:3, 5:8 integrity 5:6, 7:8, 21:8-21:9 review questions 5:2-5:3, 5:8-5:9 intracluster correlation coefficient (ICC) 16:14-

16:14 inverse-variance meta-analysis 9:17-9:18 continuous outcome data 9:20-9:21 dichotomous outcome data 9:18-9:20 fixed-effect method 9:16 random-effects method 9:17, 9:23, 9:25 rare events 16:31-16:32, 16:33 RevMan 9:26 see also generic inverse-variance outcome

data IPD see individual patient data IPD MA MG see individual patient data meta-

analysis methods group IRMG see Information Retrieval Methods Group ITT see intention to treat Joanna Briggs Institute (JBI) 20:8, 20:9, 20:17 journals 6:11-6:13, 7:11 kappa statistics 7:4-7:5, 7:15 L’Abbé plots 11:6 language bias 6:2-6:3, 10:2, 10:9 restrictions 6:32-6:33 last edit date 3:9 last observation carried forward (LOCF) 8:36 licensing 6:3

location bias 10:2, 10:8 LOCF see last observation carried forward log scales 7:20-7:21, 7:25, 9:14, 9:16, 9:22,

10:12 log-rank statistics 7:24, 9:35 main outcomes 5:4-5:7, 11:7 main text sections see text sections maintenance of Cochrane reviews 3:1-3:15 amendments 3:2, 3:5-3:6 citation versions 3:3-3:5, 3:6-3:8 conclusions changed 3:4, 3:6-3:8 dates 3:8-3:10 definitions 3:2-3:8 editorial procedures 3:13 feedback 3:15 frequency 3:2 History tables 3:13-3:15 major changes 3:3, 3:4 methods 3:12-3:13 protocols 2:1-2:2, 3:3-3:5 review questions 3:10-3:12, 5:9 splitting reviews 3:12 updates 3:2, 3:5, 3:6-3:7, 3:9-3:13 What’s new events 3:10, 3:13-3:15 withdrawn reviews 2:6, 3:4, 3:13, 4:17 major changes to protocols 3:4, 3:5 Mantel-Haenszel methods 9:18, 9:33 forest plots 11:14 non-randomized studies 13:25 rare events 16:32, 16:34 RevMan 9:26, 11:3 test for subgroup differences 9:33 masking see blinding matching 13:24 mean difference see difference in means; standardized mean difference measurement scales see scales measures of agreement 7:4-7:5, 7:15 medians 7:21, 11:5 MEDLINE 6:1-6:9, 6:19 abstracts of Cochrane reviews 4:4 adverse effects 14:7 CENTRAL 6:21-6:22 citation versions 3:3 economics evidence 15:9 handsearching 6:11 identifiers 4:19 non-randomized studies 13:12, 13:13 patient-reported outcomes 17:4, 17:7 qualitative research 20:7 reporting bias 6:2, 10:2, 10:8 searching for reviews 6:6-6:7 searching for studies 6:25-6:27, 6:33-6:40 meta-analysis 1:4, 9:1-9:43, 16:2-16:34 benefits and limitations 9:3-9:4

baseline risk 9:36-9:37 Bayesian methods 8:24, 16:29-16:31 cluster-randomized trials 16:12-16:14 cross-over trials 16:17-16:21 data collection 7:10 data types continuous outcome data 9:20-9:23,

9:38 counts 9:24-9:25, 16:32 dichotomous outcome data 9:19-9:20,

9:22-9:25, 16:31-16:34 generic inverse-variance outcome data

9:17-9:18 O – E and Variance data 9:18, 9:25,

18:7 ordinal outcome data 9:23-9:24 rate data 9:24-9:25 time-to-event data 9:25 dose-response analyses 9:37 economics evidence 15:18-15:19 effect measures 9:6-9:7, 9:29 difference in means 9:20-9:23 odds ratio 9:17-9:20, 9:22-9:23 risk difference 9:18-9:20 risk ratio 9:18-9:20 standardized mean difference 9:11,

9:21, 9:23 fixed-effect method 9:16, 9:29 forest plots 4:21, 11:3-11:5 heterogeneity 9:27-9:32 hierarchical models 16:31 indirect comparisons 16:26-16:27 individual patient data 16:31, 18:6-18:7 interpreting results 12:9-12:12 inverse-variance 9:17 Mantel-Haenszel methods 9:18, 9:31 meta-regression 9:33-9:37 methods sections 4:10 missing data 16:2-16:7 multi-arm studies 16:21-16:25 multiple analyses 8:28, 8:37-8:42, 16:28-

16:29, 17:10 multiple-treatments 16:23, 16:26-16:27,

21:2 non-randomized studies 13:18, 13:23-13:27 patient-reported outcomes 17:10-17:11 Peto method 9:18, 9:25, 16:32, 16:33 planning 9:2-9:3 precision 9:3, 9:17 pre-specification 9:5, 9:34, 9:36, 16:28-

16:29 principles 9:2-9:3 prospective 19:4-19:7, 19:6 protocols 9:5 random-effects methods 9:16-9:18, 9:29-

9:32

rare events 16:31-16:34 results sections 14:11-14:13, 11:14 RevMan 9:2, 9:5, 9:17, 9:24-9:26, 9:29,

9:30-9:33 risk of bias 8:2-8:4, 8:21-8:24, 9:4 scales 9:21, 9:23-9:24 sensitivity analyses 9:37-9:39, 10:22-10:23 skewed data 9:22 standard errors 9:11, 9:20, 9:22-9:23 statistical test 9:16, 11:14-11:15 subgroup analyses 9:32-9:36 vote counting 9:26 weights 9:16-9:17, 9:19, 9:21, 9:31, 10:2 meta-epidemiology 8:3, 8:22 meta-regression 9:33-9:37 baseline risk 9:36-9:37 figures 11:16 interpreting results 9:35-9:36 risk of bias 8:23 study characteristics 9:34-9:35 methods sections 4:6-4:11, 22:10-22:13 MID see minimally important difference minimally important difference (MID) 12:11,

17:10 missing data 16:2-16:7 conditional outcomes 16:10 general principles 16:3-16:4 individual patient data 18:2, 18:8 intention to treat 16:7-16:10 methods sections 4:10 sensitivity analyses 9:38-9:39, 16:4, 16:9-

16:10 standard deviations 16:4-16:7 types 16:2-16:3 see also imputations; incomplete outcome

data; selective outcome reporting mixed methods approach 20:3, 20:4 MTM see multiple-treatments meta-analysis multi-arm studies 9:16, 16:21-16:25 factorial trials 16:22, 16:25-16:26 heterogeneity 16:24-16:25 meta-analysis 16:22-16:25 methods sections 4:10 risk of bias 16:23 multi-centre trials 19:2-19:3 multiple analyses 8:23, 8:37-8:40, 16:28-16:29,

17:10 body parts 9:15 intervention groups 4:10, 9:16, 16:21-16:26 publication 10:2, 10:7-10:8 reports 7:2, 7:14-7:15 treatment attempts 9:15 multiple-treatments meta-analysis (MTM) 16:23,

16:26, 22:12

narrative summaries 9:2-9:3, 15:17-15:18 narrow questions 5:8-5:9 National Library of Medicine (NLM) 6:5, 6:21-

6:23, 6:27 network meta-analysis see multiple-treatments

meta-analysis new citation versions 3:3-3:5, 3:6-3:8 Newcastle-Ottowa Scale 13:22, 13:27 next stage expected date 3:9, 4:4 NHS Economic Evaluation Database (NHS

EED) 15:10-15:11, 15:12, 15:14, 15:18 NLM see National Library of Medicine NNH see number needed to treat NNT see number needed to treat no longer updated date 41 Non-Randomized Studies Methods Group (NRS

MG) 13:1-13:3, 13:4, 13:7-13:8, 13:14, 13:22, 13:27,13:30-13:31

non-randomized studies (NRS) 13:1-13:34 adverse effects 13:5, 13:8, 13:29, 14:5,

14:9-14:10 adjusted estimates 13:18, 13:21 aetiological research questions 13:7 benefits and limitations 13:2-13:4, 13:5-

13:6 confounding 13:4, 13:6, 13:18, 13:21,

13:24-13:25 data collection 13:4-13:17 designs and design features 13:1-13:5, 13:6,

13:7-13:8, 13:10, 13:11 eligibility criteria 13:5-13:11, 13:14, 13:28 generic inverse-variance data 13:24 GRADE/quality of evidence 12:3, 12:4-

12:5, 12:7, 13:29-13:30 heterogeneity 13:18, 13:23, 13:26-13:27 inclusion with randomized trials 13:3, 13:5,

13:13-13:18 interpreting results 13:27-13:28 meta-analysis 13:18, 13:23-13:27 precision 13:5-13:6 protocols 13:4-13:5, 13:2 public health and health promotion 21:2,

21:4 reporting bias 5:7, 13:12 resources and guidance 13:7-13:8, 13:14,

13:15-13:17, 13:19-13:23, 13:24-13:27, 13:3

risk of bias/quality 13:3-13:4, 13:5-13:7, 13:17-13:23, 13:28-13:29

searching 13:12-13:14 selecting studies 13:14-13:17 selection bias 13:4, 13:21 strength of evidence 13:29-13:30 NRS see non-randomized studies NRS MG see Non-Randomized Studies Methods

Group

number needed to treat (NNT) 9:6, 9:9, 9:19-9:20, 12:12-12:16, 12:17

nursing-specific databases 6:8 O – E and Variance data 9:18, 9:25, 9:26, 18:7 see also time-to-event data objectives sections 4:6, 22:7, 22:10 observational studies see non-randomized studies odds ratio (OR) 9:6-9:10 cluster-randomized trials 16:14 confidence intervals 12:9-12:10 data extraction 7:24-7:26 forest plots 11:4-11:5 funnel plots 10:12, 10:18 interpreting results 12:12-12:14 log scales 7:25-7:26, 9:14, 9:16, 10:12 meta-analysis 9:18-9:20, 9:22-9:26 non-randomized studies 13:25, 13:26 ordinal outcome data 9:12, 9:23-9:24 rare events 16:32-16:34 RevMan 11:3-11:5 standardized mean difference 9:22-9:23,

12:17 Summary of findings tables 11:10-11:11,

11:13 ongoing studies 4:18, 4:19, 6:15-6:20 ordinal outcome data 9:6, 9:11-9:12 data extraction 7:22 meta-analysis 9:23-9:24 sensitivity analyses 9:38 OR see odds ratio other outcome data 4:20, 9:23, 9:26, 11:3, 11:5 outcomes adverse effects 5:6-5:7, 14:1-14:5 change from baseline 7:17-7:18, 9:21-9:22 Characteristics of studies tables 11:2 Data and analysis section 4:20-4:21 data collection 7:5-7:6, 7:9-7:10 economics evidence 15:7 eligibility criteria 5:2, 5:4-5:7 GRADE/quality of evidence 12:2-12:3 grouping in Risk of bias tool 8:8 main 4:8, 5:5-5:6, 11:7 methods sections 14:7-14:8 missing see incomplete outcome data multiple 16:28-16:29 patient-reported 17:1-17:14 primary 4:7, 5:6 reporting bias see selective outcome reporting review questions 5:4-5:7 risk of bias summarization 8:20, 12:6-12:7 secondary 4:8, 5:6 Summary of findings tables 4:8, 5:6, 11:7-

11:8, 11:12

see also continuous outcome data; count data; dichotomous outcome data; ordinal outcome data; other outcome data; rate data; time-to-event data

outcome reporting bias see selective outcome reporting

overview plots see forest top plots Overviews of reviews 1:4, 22:1-22:21 abstracts 222:8-22:9 acknowledgements 22:15 analyses 22:6-22:7, 22:11 comparison with intervention reviews 22:6-

22:7 conclusions 22:9, 22:14 data collection 22:7, 22:11 declarations of interest 22:15 definition 22:1-22:2 discussion sections 22:13-22:14 eligibility criteria 22:13-22:15, 22:7, 22:10-

22:11, 22:15 figures 22:20 format 22:8-22:20 GRADE/quality of evidence 22:11-22:12,

22:14, 22:18 implications for intervention reviews 5:5,

5:8 methods sections 22:10-22:12 new material and analyses 22:5, 22:8, 22:12 objectives sections 22:10 organizational issues 22:6, 22:6 plain language summary 22:9 protocols 22:6, 22:15 quality of included reviews 22:6, 22:7,

22:11-22:12, 22:13 rationale 22:2, 22:3-22:5 references 22:15 results sections 22:12-22:13 searching 22:6-22:7, 22:11 tables 22:16-22:20 text sections 22:9-22:13 titles 22:8 updates 22:8 P values 12:10-12:12 contour-enhanced funnel plot 10:14-10:15 data extraction 7:19-7:20, 7:25 funnel plot asymmetry test 10:16-10:17 heterogeneity test 9:28 meta-analysis test 9:16 multiple analyses 16:28-16:29 selection models 10:22 paired t-tests 16:17 pair-wise comparisons 9:2, 9:4, 16:22-16:24 parallel group trials 16:14 partial economic evaluations 15:2 participants

background sections 4:7 Characteristics of studies tables 11:2 data collection 7:5-7:6 eligibility criteria 4:7, 5:2-5:4, 5:8-5:9 responsiveness 7:8 review questions 5:2-5:3, 5:8-5:9 patient preferences 12:9 Patient-Reported Outcomes Methods Group

(PRO MG) 1:71, 17:3, 17:11-17:12 patient-reported outcomes (PROs) 17:1-17:14 assessment and description 17:8 comparability 17:9-17:10 definitions 17:1-17:5 importance 17:3 interpreting results 17:10-17:12 measurement of change 17:7 quality of life 17:3-17:6, 17:7, 17:9, 17:10 searching 17:11-17:12 validity of instruments 17:6-17:7 peer review 2:9, 10:6 performance bias 8:7 permission to publish 2:11-2:13 per-protocol analysis 8:33, 16:7-16:8s Peto method 9:18, 9:25, 16:32, 16:33 pharmaceutical industry trials registers 6:18-

6:19s pharmacology-specific databases 6:7 PHHP see public health and health promotion Phillips checklist 15:14 PICO acronym 5:1, 5:2, 6:28 placebo-controlled trials 8:30 plain language summaries 4:4-4:5, 11:18, 22:9 plots see figures; forest plots; funnel plots PMA MG see Prospective Meta-analysis

Methods Group PMAs see prospective meta-analyses pooled estimates see meta-analysis population average effect 13:26 post hoc analyses 2:2, 9:34, 9:36, 11:34 PPP see Purchasing Power Parities pre-specification meta-analysis methods 9:5, 9:34, 9:36,

16:28-16:29 non-randomized studies 13:4, 13:20 outcomes 7:9 review methods 1:3-1:4, 5:2 selective outcome reporting 8:14-8:15 precision confidence intervals 12:10 funnel plots 10:11 GRADE/quality of evidence 12:3, 12:5 meta-analysis 9:3, 9:16 non-randomized studies 13:5-13:6 risk of bias 8:3, 8:21, 8:42 searching 6:29, 6:34-6:37 preparation of Cochrane reviews 2:1-2:16

advisory groups 2:7-2:8 commercial sponsorship 2:13-2:14 consumer involvement 2:6-2:7 declarations of interest 2:13-2:14, 4:16,

22:15 editorial procedures 2:9, 3:13 format of Cochrane reviews 2:3-2:4 funding 2:11 logistics 2:5-2:11 motivation 2:5 permission to publish 2:11-2:12 protocols 2:1-2:2, 2:2-2:3, 2:5-2:6 publication in journals and books 2:11-2:13 resources 2:10-2:11, 13:30, 14:2, 14:4, 18:3,

19:3, 20:5 review teams 2:6-2:8 software 2:8, 4:1-4:2, 11:10 timeline 2:10, 2:11 topic and scope 2:5 training 2:9 presentation of results see results sections;

Summary of findings tables primary outcomes 4:7, 5:6, 8:15 PRO MG see Patient-Reported Outcomes

Methods Group PROGRESS acronym 21:4-21:6 proportional odds model 9:12, 9:23-9:24 PROs see patient-reported outcomes prospective meta-analyses (PMAs) 19:1-19:11 collaborations 19:4 data collection 19:6-19:7 definitions 19:1-19:4 eligibility criteria 19:1-19:7 meta-analysis 19:4-19:7, 19:8 multi-centre trials 19:2-19:3 protocols 19:4-19:7 Prospective Meta-analysis Methods Group (PMA MG) 19:1, 19:3 19:9 prospective trial registration 10:1 protocol information 4:2-4:4 protocols amendments 3:3-3:5 citation versions 3:3-3:4, 3:5 differences with reviews 4:16-4:17 economics evidence 15:4, 15:11 first published date 4:3 major change 3:5 meta-analysis 9:5-9:6 multiple analyses 16:1, 16:29 non-randomized studies 13:4-13:5, 13:19-

13:21 outline 2:2-2:3 Overviews of reviews 22:14 prospective meta-analyses 19:4-19:7 rationale 2:1-2:2 registration 2:5-2:6

proximity operators 6:32 psychiatry-specific databases 6:8-6:9 psychology-specific databases 6:8-6:9, 6:10 PsycINFO 6:9, 6:10, 20:7 public health and health promotion (PHHP)

21:1-21:2 applicability 21:8-21:9 context 21:6-21:7 ethics 21:4-21:6 inequalities 21:4-21:6 qualitative research 20:3-20:4 risk of bias/quality 21:4-21:6 searching 21:2, 21:3 study design 21:2 sustainability 21:7-21:8 transferability 21:8-21:9 publication bias 10:2-10:7 causes 10:6-10:7 direct and indirect evidence 10:3-10:5 GRADE/quality of evidence 12:4, 12:6 multiple 10:2, 10:7-10:8 review questions 5:7 time lag bias 10:2, 10:5-10:6 trial registries 10:11 see also reporting bias publication policies 2:11-2:13, 18:3, 19:7 published notes 4:17 PubMed 6:5, 6:40 see also MEDLINE Purchasing Power Parities (PPP) 15:16 QALY see quality-adjusted life years QOL see quality of life QRMG see Qualitative Research Methods Group qualitative research 20:1-20:18 critical appraisal/quality 20:8, 20:16-20:17 definitions 20:2-20:3 directly observed therapy 20:5-20:7 eligibility criteria 20:6 implementation of interventions 20:4-20:5,

20:6 integration with quantitative research 20:9-

20:10, 20:16 mixed methods approach 20:3, 20:4-20:5 public health and health promotion 21:2,

21:3 randomized trials 20:4-20:5 resources 20:5, 20:14-20:18 review questions 20:3 searching 20:4, 20:6, 20:7, 20:15 strategies for addressing 20:3-20:5 synthesis 20:4, 20:5-20:10, 20:15-20:16 Qualitative Research Methods Group (QRMG)

20:2, 20:10-20:11 quality adverse effects 14:8-14:11

economics evidence 15:12-15:16, 15:17 evidence see GRADE interventions 7:8 non-randomized studies 13:1-13:2, 13:3,

13:4, 13:5-13:7, 13:7-13:23, 13:28-13:29 Overviews of reviews 22:6, 22:7, 22:11-

22:13, 22:16 public health and health promotion 21:3-

21:4 qualitative research 20:8, 20:16-20:17 scales 8:4-8:5 weights 8:4, 8:24 see also risk of bias quality of life (QOL) 17:3-17:6, 17:7, 17:9-

17:10, 17:11 quality-adjusted life years (QALY) 15:3-15:4,

1:57 quantitative synthesis see meta-analysis quasi-randomized studies 6:22-6:23, 8:26, 13:2,

13:10, 13:16, 13:18 random-effects methods 9:16-9:18, 9:29-9:32 confidence intervals 12:9-12:11 dichotomous outcome data 9:18-9:20, 16:34 forest plots 11:3-11:5 multi-arm studies 16:24-16:25 reporting bias 10:10-10:21 RevMan 9:17, 9:19-9:20, 9:26, 9:31 sensitivity analyses 9:39, 10:20-10:21 tau-squared 9:30, 10:19, 11:4-11:5 randomization 8:6, 8:24-8:27, 18:5 randomized controlled trial (RCT) 6:21-6:22,

6:26, 6:34-6:37 randomized trials adverse effects 14:2 GRADE/quality of evidence 12:3-12:7 inclusion with non-randomized studies

13:3-13:4, 13:5 public health and health promotion 21:2,

21:4 qualitative research 20:4 review questions 5:7-5:10 risk of bias 8:6-8:7, 8:1-8:43 searching 6:36-6:38 ranges 7:21 rare events 16:31-16:34 rate data 9:6, 9:12-9:13 data extraction 7:22-7:23 meta-analysis 9:24-9:25 ratio measures 7:25-7:26, 9:6-9:8, 11:13, 12:12 RCT see randomized controlled trial RD see risk difference references 4:19-4:20 managing 6:39-6:40 Overviews of reviews 22:15 searching 6:14-6:16

regional databases 6:6-6:7 regression models 13:24, 13:25 re-inclusions 8:33 related terms 6:30-6:31 relative effects see ratio measures relative risk see risk ratio relative risk reduction (RRR) 12:12-12:13 see also risk ratio re-occurring events 9:15 repeated observations 9:15 reporting bias 10:2-10:33 avoidance 10:10-10:11 citation bias 10:2, 10:8-10:9 definitions 10:2 detection 10:11-10:23 economics evidence 15:20 funnel plots 10:11-10:20 grey literature 10:10-10:11 heterogeneity 10:14, 10:19 individual patient data 18:7-18:8 language bias 10:2, 10:9 location bias 10:2, 10:8-10:9 methods sections 4:10 non-randomized studies 5:8, 13:14 searching 6:2-6:3, 10:8, 10:10 selective outcome reporting 8:6, 8:37-8:40,

10:2, 10:9-10:11 sensitivity analyses 10:20-10:23 trial registries 10:11 unpublished studies 10:10-10:11 see also publication bias; risk of bias reports see study reports resources economics evidence 15:2-15:3, 15:19 information sources 6:3-6:27, 14:6, 15:2-

15:3, 21:3 preparation of Cochrane reviews 2:10-2:11,

13:30, 14:2, 14:4, 18:3, 19:3, 20:5 restricted randomization 8:26 results sections 4:11-4:13, 11:14, 21:12-21:13 retracted publications 6:33 review information 4:2-4:4 Review Manager (RevMan) software 2:8 cluster-randomized trials 16:12-16:14 cross-over trials 16:17 data entry 7:16, 7:20, 7:24-7:26 dates 3:8, 4:3-4:4 figures 4:21-4:22, 11:5-11:7 forest plots 11:3-11:5 format of Cochrane reviews 2:2-2:3, 4:1 headings 4:5 identifiers 7:12-7:13 indirect comparisons 16:26-16:27 individual patient data 18:7 meta-analysis 9:2, 9:5, 9:18, 9:24-9:26,

9:29, 9:31

non-randomized studies 13:24 funnel plots 10:12 P values 12:11-12:12 Risk of bias tool 8:17-8:19, 8:21-8:22 Summary of findings tables 11:8-11:11 review questions 5:1-5:11 broad versus narrow 5:8-5:9 changing 3:11-3:12, 5:9-5:10 eligibility criteria 5:2, 5:7 interventions 3:4, 5:8-5:9 outcomes 5:4-5:7 participants 5:2-5:4, 5:8-5:9 studies 5:7 Summary of findings tables 11:7-11:8 review teams 2:6-2:8 RevMan see Review Manager risk of bias 8:2-8:5 adverse effects 14:8-14:11 allocation concealment 8:6, 8:25, 8:27-8:29 assessment tools 8:4-8:6 see also Risk of bias tool baseline imbalance 8:41 blinding 8:6-8:7, 8:29-8:31 cluster-randomized trials 16:11-16:12 continuous outcome data 8:13-8:14, 8:35,

8:38 cross-over trials 16:15-16:17 data collection 7:7, 8:5-8:6 data extraction 7:15 definitions 8:2-8:4 design-specific 8:4 dichotomous outcome data 8:14, 8:22, 8:34,

8:38 differential diagnostic activity 8:42 direction and magnitude 8:19-8:21 early stopping 8:40-8:41 economics evidence 15:12-15:16, 15:18-

15:19 empirical evidence 8:3-8:4, 8:19, 8:24-8:25,

8:27, 8:29-8:30, 8:33, 8:33 figures/graphs 4:22, 8:17-8:18, 8:21 forest plots 8:21 funding 8:42 GRADE 12:3-12:8 incomplete outcome data 8:6, 8:31-8:37 meta-analysis 8:2-8:5, 8:21-8:24, 9:4 methods sections 4:9 multi-arm studies 16:23 non-randomized studies 13:3, 13:3, 13:4,

13:6-13:7, 13:17-13:23, 13:28-13:29 other sources 8:42-8:43 Overviews of reviews 22:12, 22:14, 22:17 precision 8:3, 8:21, 8:42-8:43 presentation of assessments 8:17 public health and health promotion 21:4 quality 8:3

randomized trials 8:6-8:7, 8:24-8:43 rare events 16:33-16:34 reported 8:4 results sections 4:12-4:14 scales 8:4-8:5, 8:19 sensitivity analyses 8:23 sequence generation 8:6-8:11, 8:18, 8:24-

8:27 summary assessments 8:19-8:21, Summary of findings tables 12:4, 12:6-12:7 tables 4:17, 8:19-8:21 time-to-event data 8:34 updates and amendments 3:10, 3:12-3:13 see also reporting bias risk difference (RD) 9:6-9:7, 9:7 funnel plots 10:19 interpreting results 12:12-12:15 meta-analysis 9:18-9:19 rare events 16:32-16:33 Summary of findings tables 11:10-11:11 Risk of bias tool 8:7-8:16 allocation concealment 8:9, 8:12, 8:17 blinding 8:8, 8:13, 8:18-8:19, 8:29-8:31 descriptions 8:10 incomplete outcome data 8:7-8:8, 8:13-

8:14, 8:18-8:19, 8:31-8:37 judgements 8:10-8:16 other sources 8:6, 8:42 RevMan 8:6, 8:21-8:22 selective outcome reporting 8:7-8:8, 8:13-

8:14, 8:18-8,19, 8:38-8:40 sequence generation 8:9, 8:10-8:12, 8:17-

8:18 unclear 8:10, 8:21-8:22, 8:27 risk ratio (RR) 9:6-9:10 funnel plots 10:19-10:20 interpreting results 12:12-12:13, 12:14-

12:15 meta-analysis 9:18-9:20 rare events 16:22-16:23 Summary of findings tables 11:10-11:11 RR see risk ratio RRR see relative risk reduction scales 9:6, 9:11-9:12 data extraction 7:6, 7:9, 7:22 meta-analysis 9:21, 9:23-9:24 risk of bias/quality 8:4-8:5, 8:17 see also continuous outcome data; log scales; ordinal outcome data SCHARP software 18:7 SCI see Science Citation Index Science Citation Index (SCI) 3:3, 3:5, 3:7, 4:4,

11:15 scope of review questions 5:8-5:9 searching 6:1-6:45

abstracts 6:3, 6:13, 6:40, 6:41, 13:14, 14:8 adverse effects 14:6-14:8 dates 3:9, 3:10, 4:3-4:4 documenting 4:8, 6:41 economics evidence 15:9-15:11 eligibility criteria 6:28 managing references 6:39-6:41 methods sections 4:6-4:11, 6:41-6:42 non-randomized studies 13:12-13:14 Overviews of reviews 22:7, 22:11 patient-reported outcomes 17:7-17:8 planning 6:20-6:27 public health and health promotion 21:2-

21:3 qualitative research 20:4-20:5, 20:6, 20:7,

20:15 references 6:14-6:15 reporting bias 6:3, 10:8, 10:10 results sections 4:11, 6:42 sources 6:3-6:20 bibliographic databases 6:3-6:10 Cochrane Library 6:4-6:5, 6:21, 6:22,

6:24, 6:25-6:26, 6:31-6:32 EMBASE 6:5, 6:14, 6:21, 6:25-6:30,

6:36-6:37 grey literature 6:10, 6:19, 6:42 journals 6:11-6:15 MEDLINE 6:5, 6:14, 6:21, 6:25-6:30,

6:36-6:37 ongoing/unpublished studies 6:15-6:19 strategies 6:28-6:29 Boolean operators 6:31, 6:39 examples 6:37-6:38 filters 6:33-6:37 proximity operators 6:32 restrictions 6:32-6:33 search filters 6:33-6:37 sensitivity and precision 6:29, 6:34-

6:37, 6:39 service providers/search interfaces

6:28-6:29 structure 6:28 subject terms 6:29-6:30 updating 6:37 titles 6:3, 13:14, 14:8 Trials Search Co-ordinators 6:2-6:3, 6:7,

6:11, 6:20, 6:24-6:25, 6:40, 6:42 secondary outcomes 4:8, 5:6 selection bias 8:7, 8:24, 8:28 funnel plots 10:14 non-randomized studies 13:4, 13:18, 13:31 see also selective outcome reporting selecting studies 7:2-7:5 selective outcome reporting 8:6, 8:37-8:40, 10:2,

10:9-10:10 results sections 4:13

Risk of bias tool 8:7-8:8, 8:11-8:16, 8:17-8:18, 8:19, 8:39-8:40

sensitivity analyses 9:37-9:39 cluster-randomized trials 16:14 cross-over trials 16:20-16:21 individual patient data 18:8 methods sections 4:11 missing data 16:4, 16:9-16:10 non-randomized studies 13:26 reporting bias 10:20-10:23 risk of bias 8:23 search strategy 6:29, 6:34-6:37, 6:39 sequence generation 8:6-8:12, 8:17-8:18, 8:24-

8:27 results sections 4:12 Risk of bias tool 8:9, 8:10-8:12, 8:17-8:19 service providers 6:28-6:29 simple randomization 8:25 skewed data 7:20-7:21, 9:20, 9:22 small study effects 10:14-10:16, 10:18, 10:20-10:23, 13:12 SMD see standardized mean difference SMG see Statistical Methods Group social healthcare-specific databases 6:8 social science-specific databases 6:8-6:9 socio-economic data 21:4-21:6 software 2:8, 4:1-4:2, 11:10 sources of support 2:13-2:14, 4:23 splitting reviews 3:12, 5:8 standard deviations cross-over trials 16:17-16:20 data extraction 7:16, 7:18-7:20, 7:22 missing 16:4-16:7 standardized mean differences 9:11, 9:21-

9:22, 12:17 standard errors cluster-randomized trials 16:13-16:14 cross-over trials 16:20-16:21 data extraction 7:16, 7:18-7:20, 7:25-7:26,

9:28 meta-analysis 9:17, 9:21, 9:25 standardized mean difference (SMD) 9:11,

12:16-12:17 change from baseline 9:21-9:22 cluster-randomized trials 16:11 cross-over trials 16:19 funnel plots 10:19 interpreting results 12:16-12:17 meta-analysis 9:11, 9:21, 9:22-9:23 patient-reported outcomes 17:9, 17:11 statistical heterogeneity see heterogeneity significance 12:11-12:12 tests funnel plot asymmetry 10:16-10:20

heterogeneity 9:28, 11:14-11:15 meta-analysis 9:16, 11:14-11:15 subgroup differences 9:32-9:33, 11:14-

11:15 Statistical Methods Group (SMG) 9:39-9:40,

10:24 Steering Group 2:14 stratification 8:26, 13:24, 13:25 stratified randomization 8:26 structure see format of Cochrane reviews studies awaiting classification 4:12, 4:18, 4:19 study reports data collection 7:11 data extraction 7:13-7:16 eligibility criteria 7:2 studies Characteristics of studies tables 11:2 data collection 7:16 eligibility criteria 4:7, 5:7 review questions 4:7-4:8, 5:8-5:10 selecting 7:2-7:5 subgroup analyses 9:32-9:35 Data and analysis section 4:20-4:21 individual patient data 18:6-18:7 interpreting results 9:35-9:36 methods sections 4:10 study characteristics 9:34-9:35 test for differences 9:33, 11:14-11:15 subheadings 4:5 subject-specific databases 6:7-6:9 subject-specific trials registers 6:18, 6:24-6:25 summary assessments 8:19-9:21 Summary of findings tables 4:18, 11:5-11:13 detailed contents 11:11-11:13 GRADE/quality of evidence 11:7-11:8,

11:13, 12:2-12:8 outcome selection 5:5-5:6, 11:7-11:8 Overviews of reviews 22:12, 22:17, 22:19 patient-reported outcomes 17:3 software 11:10 statistical considerations 11:10-11:11, 12:13 templates 11:8 survival data see time-to-event data sustainability 21:7-21:8 synonyms 4:30-4:31 synthesis qualitative 20:4-20:10, 20:15-20:16 quantitative see meta-analysis systematic review 1:3-1:4 t values 7:19-7:20 tables of contents (TOC) services 6:12-6:13 tables 4:17-4:18 Additional 4:18, 8:43, 11:13, 15:16-15:17,

17:8

Characteristics of excluded studies 4:12, 4:17, 7:4

Characteristics of included studies 4:11-4:12, 4:17, 11:2, 15:16

Characteristics of ongoing studies 4:12, 4:18

Characteristics of studies awaiting classification 4:12, 4:18

Overviews of reviews 22:12, 22:17-22:20 Risk of bias 4:17, 8:17-8:18 Summary of findings 4:18, 11:7-11:13 tau-squared 9:30, 10:19, 11:4-11:5 tests see statistical tests text sections 4:5-4:17, 22:9-22:15 authors’ conclusions sections 4:14 background sections 4:5-4:6 discussion sections 4:13-4:15 methods sections 4:6-4:11 objectives sections 4:6 Overviews of reviews 22:9-22:15 results sections 4:11-4:12, 11:14, 15:16-

15:17 text words 6:29-6:30, 14:8, 17:7 thesaurus terms see controlled vocabulary theses databases 6:9-6:10 time lag bias 10:2, 10:5-10:6 time-to-event data 9:6, 9:13 confidence intervals 12:9-12:11 counts 7:23 data extraction 7:23-7:24 individual patient data 18:2, 18:6, 18:7 meta-analysis 9:25 risk of bias 8:34 sensitivity analyses 9:38 timeline for Cochrane reviews 2:10, 2:11 titles 4:2 adverse effects 14:8 non-randomized studies 13:14 Overviews of reviews 22:8 results sections 11:11-11:12, 11:17 searching 6:3, 13:14, 14:8 selecting studies 7:2 TOCs see tables of contents training 2:9 transferability 21:8-21:9 transformations 7:20-7:21 treatment effects 61, 9:2 trials registers 6:15-6:19, 6:24-6:25, 10:11 trials results registers 6:19 Trials Search Co-ordinators (TSC) 6:2, 6:7, 6:11,

6:20, 6:24, 6:40, 6:42 trim and fill method 10:21-10:22 TRIP see Turning Research into Practice truncations 6:30-6:31 TSC see Trials Search Co-ordinators Turning Research into Practice (TRIP) 6:14

unclear risk of bias 8:10-8:16, 8:21-8:22, 8:27 unit-of-analysis errors 9:14-9:16, 16:10-16:11,

16:12, 16:17 unpublished studies 6:15-6:20, 10:10-10:11 unrestricted randomization 8:25 up-to-date reviews 3:8-3:9, 4:3 updates 3:2, 3:4, 3:5-3:6, 3:8-3:15 methods sections 4:11 Overviews of reviews 22:8 searching 6:37, 6:39 validation of instruments 17:6-17:8 validity see risk of bias variant spellings 6:30-6:31 vote counting 9:26

web searching 6:15 weighted mean difference (WMD) 9:10-9:11 weights meta-analysis 9:16-9:17, 9:18, 9:19, 9:21,

9:31, 10:20 quality 8:4, 8:24 well-being 17:2, 17:3 What’s new events 3:10, 3:13-3:15, 4:4 WHO see World Health Organization wildcards 6:30-6:31 WinBUGS software 16:29-16:31 withdrawals see incomplete outcome data WMD see weighted mean difference World Health Organization (WHO) 6:16 zero frequencies 16:32-16:33