1

Compliance of Veterinary Medicine Systematic Reviews

with Literature Search

Reporting Standards

MLA 2013 One Health - Boston

Lorraine Toews MLISHealth Sciences Library, University of Calgary

Alberta, Canada

2

Introduction

“Poorly reported studies distort the available pool of research evidence and compromise its reliability.”

Simera I, Kirtley S, Altman DG. Maturitas 2012 May;72(1):84-7.

Reporting “deficiencies make it difficult … to evaluate the reliability of the presented findings”

Simera I, Moher D, Hirst A, Hoey J, Schulz KF, Altman DG. BMC Med. 2010 Apr 26;8:24

3

Introduction

“Searching is critical to conducting systematic

reviews. Searches for systematic reviews need to

be constructed to maximize recall and deal with …

potentially biasing factors. Errors made in the

search process may results in a biased or otherwise

incomplete evidence base for the review.”

Sampson M, McGowan J. Errors in search strategies were identified by type and frequency. J Clin Epidemiol. 2006 Oct;59(10):1057-63

4

Purpose

Purpose of my study … to examine

▪Reporting of literature searches in

systematic reviews in veterinary journals

▪ Compliance with PRISMA reporting

standard

▪ Assess conduct of review

5

Methods

▪ MEDLINE journal title search

▪ 14 high impact veterinary journal titles

▪ Limited to …

▪ Titles “meta-analysis”, “systematic review”

▪ Publication type “meta-analysis”

▪ Published mid-2009 to December 2012

6

Methods: Journals Searched

American Journal of Veterinary Research

Journal of Veterinary Pharmacology & Therapeutics

Australian Veterinary Journal New Zealand Veterinary Journal

Canadian Journal of Veterinary Research

Preventive Veterinary Medicine

Canadian Veterinary Journal Veterinary Journal

JAVMA Veterinary Record

Journal of Veterinary Diagnostic Investigation

Veterinary Research

Journal of Veterinary Internal Medicine

Veterinary Surgery

7

Methods: Systematic Reviews 2009-2012

Journals Number ofSystematic Reviews

JAVMA 4

Journal of Veterinary Diagnostic Investigation

1

Journal of Veterinary Internal Medicine

3

Preventive Veterinary Medicine 9

Veterinary Journal 7

Veterinary Research 1

TOTAL

25

8

Methods: PRISMA Elements

PRISMA reporting items examined …

▪ Information sources searched▪ Databases▪ Grey literature

▪ Full electronic search strategy▪ Limits ▪ Publication date▪ Language

▪ Article, appendix, supplementary data

reviewed for PRISMA reporting elements

9

PRISMA: Information Sources Searched

“Describe all information sources

… databases with dates of

coverage … in the search …”

10

Databases Searched

Why is this important?

Systematic review literature searches -

comprehensive

Incomplete or biased search can result in …

▪ Biased evidence base

▪ Inaccurate/over estimate of intervention

effect

11

Results: Databases Searched

Database %of Reviews Searched

CAB Abstracts 64%

PubMed 56%

Agricola 32%

MEDLINE 32%

Web of Science 20%

BIOSIS 12%

Google Scholar 12%

IVIS |Scopus | AGRIS | Current Contents |Global Health| Global Health | Google | Library catalogues |LILACS | PubMedCentral

8%

12

Results: Database Coverage

Databases Searched in Reviews

Database Coverage of Veterinary Journals *

CAB Abstracts CAB Abstracts

PubMed Scopus

Agricola Science Citation Index Expanded

MEDLINE Web of Science

Web of Science BIOSIS

BIOSIS EMBASE

Google Scholar MEDLINE

IVIS |Scopus | AGRIS | Current Contents |Global Health| Global Health | Google | Library catalogues |LILACS | PubMedCentral

* Grindlay, Brennan, Dean2012 JVME 39(4): 408

13

Results: Databases Searched

Number of Databases Searched

% of Reviews that Searched Databases

1 database 8%2 databases 20%3 databases 24%4 databases 12%5 databases 16%6 databases 4%10 databases 4%

14

Results: Grey Literature Searched

Only 32% of reviews searched for grey

literature

Only 24% reported searching for

conference proceedings

15

Grey Literature Searched

Why is this important?

Counteract publication bias

▪ Over half of trials in conference proceedings never published (Scherer et al. Cochrane Database Syst Rev 2007 2)

▪ Published trials showed greater treatment effect than unpublished trials (Hopewell et al. Cochrane Database Syst Rev 2007 2)

▪ Reviews based only on published literature risk exaggerating intervention effectiveness

(Relevo et al. J Clin Epidemiol. 2011 Nov;64(11):1169)

16

PRISMA Search Strategy

“Present full electronic search

strategy for at least one database …

… including any limits used …

… such that it could be repeated”

17

Search Strategy

Why is this important?

Systematic review literature search aims to …

▪ Be comprehensive

▪Minimize bias

▪ Be reproducible

18

Search Strategy Elements

High sensitivity strategies▪ Free-text & subject headings used▪ Subject headings exploded▪ Variant spelling, truncation

Strategies customized for each database▪ Subject headings vary between databases▪ Indexing varies between databases

Cochrane Handbook Pt 2 Ch 6

19

Results: Search Strategy

0% Line-by-line full search strategy

12% Used subject headings

0% Reported exploding subject headings

48% Limited by publication date

20% Limited by language

20

Conclusions

▪ Reporting often incomplete

▪Not comply fully with PRISMA

12% did not report databases searched

64% did not report if grey literature searched

0% reported full electronic search strategy

21

Conclusions

Gaps in reporting … unable to evaluate

▪Completeness & bias of search

▪Replicate search

22

Conclusions

Reporting gaps suggest …

▪Conduct of searches fell short of

best practices

▪ Validity of reviews questionable …

based on incomplete, biased search

23

Thank you for listening …

Questions?