Literature Searchingfor Systematic Reviews

January 2009

Diane Lorenzetti MLS

Centre for Health & Policy Studies

Institute of Health Economics

dllorenz@ucalgary.ca

A systematic review is..

“a review of the evidence on a clearly formulated

question that uses systematic and explicit methods

to identify, select and critically appraise relevant

primary research, and to extract and analyse data

from the studies that are included in the review

Statistical methods (meta-analysis) may or may not

be used.”

(Undertaking systematic reviews of research on effectiveness: CRD's guidance for those carrying out or commissioning reviews (2001) http://www.york.ac.uk/inst/crd/report4.htm.)

Characteristics of systematic literature searches

• Meticulous planning

• Comprehensive method for searching the literature

• Inclusion of a variety of sources of information

• Identification of all relevant studies

• Unbiased searching (eg: inclusion of non-english language

research and both published and unpublished studies)

• Transparent and reproducible search strategy

Before beginning a systematic review…

Establish the need for a systematic review

Document decisions made by the team during the

question refinement and literature search process

Develop a timeline for completion of each stage of your

review

A timeline may include:

• Pre-searching and question refinement

• Development of inclusion/exclusion criteria

• Development of search strategy

• Literature searching (including searching of the grey literature and strategy refinement)

• Development of abstract inclusion/exclusion criteria

• Abstract review and selection

• Document retrieval

• Creation of data extraction form for full text article review

• Full text article review and selection

• Synthesis

• Preparation of the final reportMany of these activities may occur simultaneously……………

Pre-searching and Question Refinement

Develop a clear question

Conduct a pre-search prior to beginning your review

Be prepared to revise your question based on the results

of your pre-search

PICO

PICO is a model adopted by proponents of evidence-based medicine as an aid to question formulation

Developing a research question using PICOS

P = Population: the group under observation/study

I = Intervention: the intervention, exposure or action

of interest

C = Comparison an alternative action or intervention

against which the intervention of

interest will be compared

O = Outcome: a measure of the desired effect of

the Intervention/Exposure

S = Study design study designs appropriate for the

research question under consideration

Components of a research question (PICO format)

Q1: Is a pre-recorded telephone health information service

(Intervention) for mothers with toddlers (Population)

more effective than printed information (Comparison) in

increasing knowledge of accident prevention

techniques? (Outcome)

Q2: Is there a relationship between health care costs

(Intervention) and patients’ (Population) use of health

services (Outcome)?

Keywords & synonyms

Develop lists of keywords and synonyms for each

PICO component of your research question

before you begin searching for relevant literature.

Keywords are independent of any one

electronic database and will help you to be

consistent as you map your search to each

database/resource you use.

Keywords & synonyms

Eg: Is hypnotism an effective means of reducing

smoking in adolescents?

Concept/

Facet

Teenagers

P

Hypnotism

I

Smoking

O

Keywords/

Synonyms

teenager(s)

teen(s)

adolescent(s)

youth(s)

high school

student(s)

hypnotism

hypnosis

hypnotherapy

suggestion

smoke

smoking

cigarette(s)

tobacco

snuff

Boolean operators: OR

Used to connect synonyms, similar terms

Broadens / expands a search

Finds records with any of your search terms

Eg: glucosamine OR n-acetyl-d-glucosamine

Boolean operators: AND

Used to combine search concepts/components

Narrows / focuses a search

Finds records with all your search terms

Eg: osteoarthritis AND glucosamine

Is hypnotism an effective means of reducing smoking in adolescents?

PICO

Component

Teenagers

P

Hypnotism

I

Smoking

O

Keywords/

Synonyms

teenager(s)OR

teen(s)OR

adolescent(s)OR

high school

student(s)

hypnotismOR

hypnosisOR

hypnotherapyOR

suggestion

smokeOR

smokingOR

cigarette(s)OR

tobaccoOR

snuff

AND AND

Sensitivity and specificity in literature searching

Sensitivity (recall)

The proportion of relevant articles identified by a search strategy as a percentage of all relevant articles on a given topic. It is a measure of the ability of a search strategy to identify all relevant articles.

Specificity (precision)

The proportion of relevant articles identified by a search strategy as a percentage of all articles (relevant and irrelevant) identified by that search. It is a measure of the ability of a search strategy to exclude irrelevant articles.

Highly sensitive strategies tend to have low levels of precision and vice

versa. In a systematic review, the aim is to maximize sensitivity while also

attempting, insofar as it is possible, to maximize precision.

Study design filters

“a predefined search strategy designed to retrieve levels of evidence (RCTs, systematic reviews etc) or types of clinical queries (diagnosis, prognosis, etiology, treatment) when combined with the subject search terms of your choice. They are also referred to as hedges, Clinical Queries (USA), or optimal search strategies.” (BestBETS Search Strategies http://www.bestbets.org/links/strategies.html)

Used to limit search results by study design

Study design filters can increase the specificity of search results

Study design terms found in select filters

Qualitative research

(e.g. “findings”, “qualitative”, “interview$”)

Diagnosis

(e.g. “sensitivity”, “specificity”)

Prognosis

(e.g. “follow-up studies”, “incidence”)

Intervention/Therapy

(e.g. “randomised controlled trial.pt.”)

Causation

(e.g. “risk”, “risk factor” and “cohort studies”)

RCT Sensitive filter for OVID MEDLINE searches (Cochrane Collaboration)

1. (controlled clinical trial or randomized controlled trial or

meta analysis).pt.

2. clinical trials as topic/

3. (placebo$ or random$ or trial$).ti,ab.

4. 1 or 2 or 3

5. limit 4 to animals

6. limit 4 to (animals and humans)

7. 5 not 6

8. 4 not 7

RCT Sensitive filter for OVID EMBASE searches (Cochrane Collaboration)

1. cross-over procedure/ or double-blind procedure/ or

randomized controlled trial/ or single-blind procedure/

2. (allocat$ or assign$ or cross over$ or crossover$ or

(double ADJ blind$) or factorial or placebo$ or random$ or

(single ADJ blind$) or volunteer$).ti,ab.

3. 1 or 2

4. limit 3 to animals

5. limit 3 to (animals and humans)

6. 4 not 5

7. 3 not 6

Bibliographic database selection

No one bibliographic database is comprehensive enough to index all published literature on a topic. MEDLINE indexes approx 5000 of the 20,000 biomedical journals published.

Overlap between databases is common

Extent of overlap between MEDLINE and EMBASE is estimated to be 34%; ranging between 10% and 75%, depending on topic

Selection will vary with the subject. When choosing electronic databases, consider the multidisciplinary needs of your research question & select databases that reflect these needs

MEDLINE/PubMed

EMBASE

Cochrane Database of Systematic Reviews

Cochrane CENTRAL Register of Controlled Trials

CINAHL

PsycINFO

International Pharmaceutical Abstracts

Bibliographic databases for

clinical research topics

ABI Inform health policy, health services administration, economics of health care

ERIC health promotion, health education

Social Sciences Abstracts & Sociological Abstracts health policy, health promotion, patient access to and satisfaction with health care

Econlit economics of health care

Philosophers’ Index Ethical aspects of health care procedures and delivery

Bibliographic databases for non clinical

research topics

Database Thesauri/Subject Lists

• Search both general and specific thesaurus/subject terms as appropriate

MEDLINE examples:

Mental health[MeSH] plus MeSH terms for specific mental health conditions such as schizophrenia[MeSH], alcoholism[MeSH] etc…. as appropriate

Angiotensin-converting enzyme inhibitors[MeSH](ACE inhibitors used to treat high blood pressure and heart failure) plus captopril[MeSH] (a specific ACE inhibiting drug)

Broadening a search

Use broader, more general search terms

Include a variety of synonyms

Apply truncation symbols to word stems to retrieve alternate spellings, singular/plural word forms & word variants

Drop the least important concepts from your search – eg: outcomes

Apply fewer / no limits to your search

Truncation

Finds variations of a word stem

Eg: plan$ retrieves plan, plans, planner, planners, and planning

Truncation should only be used with keyword or title/abstract word searching - not with subject heading searching

Truncation symbol varies with database -

usually * but OVID databases use $

Narrowing a search

Use more specific search terms

Decrease # of synonyms included in the search

Eliminate or limit word truncation

Add concepts to your search

Apply limits

Searching Norms for Systematic Reviews

• Search both thesaurus/subject terms and text

(title/abstract) words

• Include both synonyms and truncated forms of text words

in your search

• Include CAS Registry numbers and generic/brand names

in drug searches

• Unless there is a compelling reason (eg: changes in

policies, procedures, interventions etc…) do not limit your

search by date or language

Pre-testing Your Search Strategy

Select one electronic database on which to craft, test and

finalize your search strategy

Be prepared to run multiple tests of your search strategy

with your research team before finalizing your approach

If you have pre-identified key studies, these studies can

act as a litmus test of the effectiveness of your search

strategy

Going beyond Electronic Databases

• Consider searching beyond electronic databases to locate

relevant studies:

a) Reference lists of key articles

b) Hand searching of key journals

c) Hand searching of key conference proceedings

d) Consultation with experts

e) Citation searching with Web of Science

f) Grey literature

Unpublished or ongoing studies

Conference and meeting proceedings and abstracts

Dissertations & theses

Newsletters

Reports of research organizations and government agencies

Informal communications

http://library.ucalgary.ca/branches/hsl/greylit

Grey literature may include…..

Key journals may not be indexed in mainstream electronic

databases

Examples: complementary and alternative medicine & health

promotion

Inaccurate indexing of studies in electronic databases

Incomplete search strategies may result in studies being

missed during database searching

Journals indexed in databases such as MEDLINE and

EMBASE may be indexed selectively

Abstracts, letters or conference reports may not be indexed

in electronic databases

Why hand search journals?

Cochrane collaborators around the world hand search health care journals and conference proceedings to locate randomized controlled trials of healthcare interventions.

RCTs found are entered into the Cochrane CENTRAL Register of Controlled Trials.

Currently, over 2300 journals are being searched.

The Master List of hand-searched journals can be viewed/searched online at http://apps1.jhsph.edu/cochrane/masterlist.asp

Cochrane Collaboration

hand searching initiative

Tracking your Literature Search

Record information on all resources searched (and when), search terms used, when the search was completed and how many records/results were found

Track studies retrieved, ordered and received

Consider using a bibliographic management software program to store and organize search results as well as track other activities associated with the systematic review process.

Eg: EndNote, RefWorks, Reference Manager etc..

RevMan

RevMan is the systematic reviews software used by

Cochrane to develop and publish Cochrane reviews

http://www.cc-ims.net/RevMan

• References retrieved from literature searching can be

imported into RevMan once they have been converted into

plain text records formatted to Vancouver Style

Most bibliographic software programs are capable of

converting references into a RevMan compatible format

Cochrane Collaboration Support

As part of the Cochrane Protocol approval process, a

Trials Coordinator from your Research Group will

advise/assist/review your search strategy

Literature searching support from Cochrane varies with

each Research Group

Researchers may consider approaching a librarian from

their own institution for assistance with their Cochrane

review

Questions? Comments?

Diane Lorenzetti MLS

Centre for Health & Policy Studies

Institute of Health Economics

dllorenz@ucalgary.ca / 210-9319