systematic reviews overview
TRANSCRIPT
Systematic Reviews Overview
Karen Burstein, Senior Medical Librarian [email protected]
Rebecca Lloyd, Education Services [email protected]
Value of Systematic Reviews“Systematic reviews and meta-analyses are essential tools for summarizing evidence accurately and reliably.
-They help clinicians keep up-to-date
-Provide evidence for policy makers to judge risks, benefits, and harms of health care behaviors and interventions
-Gather together and summarize related research for patients and their caregivers
-Provide a starting point for clinical practice guideline developers
-Provide summaries of previous research for funders wishing to support new research”
Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JP, et al. The PRISMA Statement for Reporting Systematic Reviews and Meta-Analyses of Studies That Evaluate Health Care Interventions: Explanation and Elaboration. Ann Intern Med. 2009;151:W-65-W-94.
Growth of Systematic Reviews & Meta-Analyses in Medicine from 1995-2013
-Graph taken from Systematic Reviews workshop led by Susan Fowler, Medical Librarian,
Washington University in St. Louis
What is a Systematic Review ?Cochrane Collaboration Definition:
A Systematic Review:
-Attempts to collate all empirical evidence that fits pre-specified eligibility criteria in order to answer a specific research question
-Uses explicit, systematic methods that are selected with a view to minimizing bias
-Thus providing more reliable findings from which conclusions can be drawn and decisions made.
Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, 2011. Available from www.cochrane-handbook.org., Section 1.2.2
Systematic Reviews are Research Studies
• A Systematic Review is a research study, requiring a carefully thought out question, an investigative team, and a study protocol
• The “Subjects” of SRs are individual primary studies
• Each included study is considered a “unit of analysis” with eligibility criteria determining inclusion
Systematic Reviews versus Narrative Reviews
Narrative reviews are summaries of research- Lack explicit descriptions of systematic methods- Evidence is often incomplete- Relevance and validity of studies often not explicit- Tend to provide a wider view of a topic by a subject “expert”- Difficult or impossible to replicate
Content taken from Stanford’s Lane Medical Library -http://elane.stanford.edu/laneconnex/public/media/documents/SysReviews_LitSearch_CS_May_2014.pptx
Do I really want to do a Systematic Review?
1. Do I have a clearly defined clinical question with established inclusion and exclusion criteria?
2. Do I have a team of at least three people assembled?
3. Do I have time to go through as many search results as we might find?
4. Do I have resources to get foreign language articles appropriately translated?
5. Do I have the statistical resources to analyze and pool data?
• If you answered “No” to any of the first four questions, a traditional literature review will be more appropriate to do.
• If you answered “No” to the last question, a meta-analysis will not be an appropriate methodology for your review.
- http://liblog.mayo.edu/2013/05/01/i-want-to-do-a-systematic-review/
Average systematic review requires at
least 12 months of work
http://handbook.cochrane.org/chapter
_2/box_2_3_b_timeline_for_a_cochrane_review.htm
Timeline for a Cochrane Review
Month Activity
1 - 2 Preparation of protocol
3 - 8 Searches for published and unpublished studies
2 - 3 Pilot test of eligibility criteria
3 - 8 Inclusion assessments
3 Pilot test of 'Risk of bias' assessment
3 - 10 Validity assessments
3 Pilot test of data collection
3 - 10 Data collection
3 - 10 Data entry
5 - 11 Follow up of missing information
8 - 10 Analysis
1 - 11 Preparation of review report
12 - Keeping the review up-to-date
Concerns about standards and quality of systematic reviews
- The quality of systematic reviews varies; often the scientific rigor of the collected literature is not scrutinized or there are errors in data extraction and meta-analysis
- In 2008, Congress directed the Institute of Medicine (IOM) to develop standards for conducting systematic reviews
- The standards are meant to assure objective, transparent, and scientifically valid systematic reviews
http://www.iom.edu/Reports/2011/Finding-What-Works-in-Health-Care-Standards-for-Systematic-Reviews.aspx
Systematic Review Process1. Define a clinical question
2. Create a protocol
3. Define criteria for including and excluding the studies
4. Conduct an exhaustive literature search
5. Select the studies
6. Collect the data
7. Appraise each study
8. Synthesize the evidence
9. Draw conclusions
10. Publish the review
STANDARDS
Finding What Works in Healthcare: Standards for Systematic Reviews (IOM Report)
- The Institute of Medicine (IOM) is an independent, nonprofit organization that works outside of government to provide unbiased and authoritative advice to decision makers and the public. (http://iom.edu/About-IOM.aspx)
STANDARDS
Cochrane Handbook for Systematic Reviews of Interventions
• The Cochrane Collaboration is a global network of health practitioners, researchers, patient advocates and others that belong to various Cochrane groups (Acute Respiratory Infections Group, Hypertension Group, Consumers and Communication Group, etc.) that produce the highest quality systematic reviews.
• More information about the Cochrane organization at: http://www.cochrane.org/about-us
STANDARDS
PRISMA Statement (Preferred Reporting Items for Systematic Reviews and Meta-Analyses)
- Consists of:
the PRISMA Checklist (27 items)
the PRISMA Flow Diagram
- Endorsed by major journals such as Lancet, BMJ, and Neurology.
PICO- the framework used to formulate the research question that is answered by a systematic review
P = Patient or problemI = InterventionC = Comparison (if appropriate)O = Outcome
Create and Register a Plan (Protocol)
Creating a protocol guides the systematic review team
Registering the protocol: • Reduces duplication• Provides transparency• Counters publication bias
Examples of Registered Protocols
PROSPERO International prospective register of systematic reviews
Cochrane Database of Systematic Reviews- Via Ovid (select EBM Reviews – Cochrane Database of Systematic Review)- Via Wiley
Literature Search
- Exhaustive
- At least these databases: Pubmed, Embase, Cochrane Central Register of Clinical Trials
- Grey literature: conferences, drug trial registries, AHRQ reports, dissertations
- Search using both subject headings and title and abstract words
- Use truncation: *
- Use all possible synonyms
- Include British spelling
- Limiting to humans (if necessary) – check Cochrane Handbook or consult librarian for correct filter; do not use database limit
- the PRISMA Flow Diagram
Data ToolsCourtesy of Penn Biomedical Library: Systematic Reviews Tools for Data
Standards Recommending Librarian Involvement in Systematic Reviews
• The Institute of Medicine (2011) has two standards specifically referring to librarians:
3.1.1 Work with a librarian or other information specialist trained in performing systematic reviews to plan the search strategy 3.1.3 Use an independent librarian or other information specialist to peer review the search strategy http://www.iom.edu/Reports/2011/Finding-What-Works-in-Health-Care-Standards-for-Systematic-Reviews/Standards.aspx?page=2
How can the library help?HSL Systematic Review Service
- Conduct the literature search
- Provide the search results in RefWorks
- Provide the methods section on the literature search process
- Update the search as needed
OR
- Recommend databases to search
- Provide search tips
- RefWorks support