Testing search strategies for systematic reviews in theMedline literature database through PubMedEnilze S. N. Volpato Postgraduate student1, Marluci Betini Postgraduate student1 and Regina El DibPost-doctoral fellow PhD2

1Librarian, Technical Division of Library and Documentation, UNESP – Univ Estadual Paulista, Botucatu, SP, Brazil2Assistant Professor, Anesthesiology Department, Botucatu Medical School, UNESP – Univ Estadual Paulista, Botucatu, SP, Brazil andResearch Collaborator, McMaster Institute of Urology, McMaster University, Hamilton, Ontario, Canada

Keywords

evidence-based medicine, informationstorage and retrieval, Medline, PubMed,systematic review

Correspondence

Mrs Enilze S. N. VolpatoTechnical Division of Library andDocumentationBotucatu Medical SchoolDistrito de Rubião Júniors/n – Botucatu – SP 18618-970BrazilE-mail: enilze@btu.unesp.br;enilze@gmail.com

Accepted for publication: 3 October 2013

doi:10.1111/jep.12094

AbstractBackground A high-quality electronic search is essential in ensuring accuracy and com-pleteness in retrieved records for the conducting of a systematic review.Objective We analysed the available sample of search strategies to identify the bestmethod for searching in Medline through PubMed, considering the use or not of paren-thesis, double quotation marks, truncation and use of a simple search or search history.Methods In our cross-sectional study of search strategies, we selected and analysed theavailable searches performed during evidence-based medicine classes and in systematicreviews conducted in the Botucatu Medical School, UNESP, Brazil.Results We analysed 120 search strategies. With regard to the use of phrase searches withparenthesis, there was no difference between the results with and without parenthesis andsimple searches or search history tools in 100% of the sample analysed (P = 1.0). Thenumber of results retrieved by the searches analysed was smaller using double quotationsmarks and using truncation compared with the standard strategy (P = 0.04 and P = 0.08,respectively).Conclusions There is no need to use phrase-searching parenthesis to retrieve studies;however, we recommend the use of double quotation marks when an investigator attemptsto retrieve articles in which a term appears to be exactly the same as what was proposed inthe search form. Furthermore, we do not recommend the use of truncation in searchstrategies in the Medline via PubMed. Although the results of simple searches or searchhistory tools were the same, we recommend using the latter.

IntroductionA high-quality electronic search is essential in ensuring the accu-racy and completeness of retrieved records for the conducting ofsystematic reviews [1]. The quality of retrieved informationdepends on the planning of specific search strategies for eachdatabase [2]. To achieve such quality, the researchers must knowthe controlled retrieval languages and available tools of each data-base, to adjust their search results according to their needs forspecificity or exhaustiveness in information retrieval [3].

PubMed is the Medline Web interface, with free online access tomedical information provided by the National Library of Medicine[4–6]. In the field of evidence-based medicine, Medline is one ofthe most important and consulted databases, along with EMBASEand the Cochrane Central Register of Controlled Trials.

We analysed the available sample of search strategies to identifythe best way to search Medline through PubMed, considering the

use of parentheses (Table 1), quotation marks (“ ”) (Table 2) andtruncation (*) (Table 3), and using simple searches or the searchhistory (Figs 1 and 2). We hypothesized that there would be adifference in the results retrieved by a search strategy usingphrase-searching parenthesis and double quotations marks, com-pared with the use without them, thereby reducing the uncertain-ties that still exist in the medical librarianship field.

Aimed to verify whether the use of truncation would increasethe number of results retrieved by the search strategies in Medlinethrough PubMed.

MethodsIn our cross-sectional study of search strategies, we selectedand analysed the available searches performed during Brazilianevidence-based medicine classes, as well as from systematicreviews conducted in our unit and from users of the Library

bs_bs_banner

Journal of Evaluation in Clinical Practice ISSN 1365-2753

Journal of Evaluation in Clinical Practice 20 (2014) 117–120 © 2013 John Wiley & Sons, Ltd. 117

Services at the Botucatu Medical School, Univ – Estadual Paulista(UNESP), São Paulo, Brazil.

The inclusion criteria for each search strategy were: (1) key-words consisting of at least two words to allow for the parenthesesand quotation marks tests; (2) at least two words with the samestem connected with the Boolean operator ‘OR’; and (3) at leastone Boolean operator ‘AND’ to allow for the testing of a simplesearch or search history.

We ran the different search strategies’ arrangements from thesame search on the same day, to avoid bias related to the addition

of new papers to the Medline database. All the searches weresubmitted to the Pubmed Advanced Search Builder, and we keptthe default configuration on the system. In our study, there was nolanguage, period or type of study restriction and no other filterswere applied.

Data extraction

We collected the original search strategies and applied the inclusioncriteria, and then we adapted each search to allow for the proposed

Table 1 The use or not of parentheses

With

( (obstructive sleep apneas) OR (obstructive sleep apnea) OR (obstructive breathing disorders) OR snoring) AND (adenotonsillectomy ORtonsillotomy OR (intracapsular partial tonsillectomy) OR uvulopharyngoplasty OR uvulopalatopharyngoplasty) AND (high blood pressure) OR(high blood pressures) OR hypertension OR (cardiac events) )

Without

(obstructive sleep apneas OR obstructive sleep apnea OR obstructive breathing disorders OR snoring) AND (adenotonsillectomy OR tonsillotomyOR intracapsular partial tonsillectomy OR uvulopharyngoplasty OR uvulopalatopharyngoplasty) AND (high blood pressure OR high bloodpressures OR hypertension OR cardiac events)

Table 2 The use or not of quotation marks

With

(“obstructive sleep apneas” OR “obstructive sleep apnea” OR “obstructive breathing disorders” OR snoring) AND (adenotonsillectomy ORtonsillotomy OR “intracapsular partial tonsillectomy” OR uvulopharyngoplasty OR uvulopalatopharyngoplasty) AND (“high blood pressure” OR“high blood pressures” OR hypertension OR “cardiac events”)

Without

(obstructive sleep apneas OR obstructive sleep apnea OR obstructive breathing disorders OR snoring) AND (adenotonsillectomy OR tonsillotomyOR intracapsular partial tonsillectomy OR uvulopharyngoplasty OR uvulopalatopharyngoplasty) AND (high blood pressure OR high bloodpressures OR hypertension OR cardiac events)

Table 3 The use or not of truncation

With

(obstructive sleep apnea* OR obstructive breathing disorder* OR snoring) AND (adenotonsillectom* OR tonsillotom* OR intracapsular partialtonsillectomy* OR uvulopharyngoplasty OR uvulopalatopharyngoplasty) AND (high blood pressure* OR hypertension OR cardiac event*)

Without

(obstructive sleep apneas OR obstructive sleep apnea OR obstructive breathing disorders OR snoring) AND (adenotonsillectomy OR tonsillotomyOR intracapsular partial tonsillectomy OR uvulopharyngoplasty OR uvulopalatopharyngoplasty) AND (high blood pressure OR high bloodpressures OR hypertension OR cardiac events)

Figure 1 Simple search, using obstructive sleep apnoea and adenotonsillectomy and high blood pressure as an example.

Testing search strategies E.S.N. Volpato et al.

© 2013 John Wiley & Sons, Ltd.118

tests. We ran the parenthesis strategy tests first, followed by thequotation marks and the truncation tests and finally the simple.

Simple size

To estimate the sample size, we assumed that, across all the ana-lysed search strategies related to truncation, 98% would show agreater number of retrieved references without truncation com-pared with those with truncation. An error of 3% within a 95%confidence interval was accepted. According to these assumptions,it was necessary to analyse approximately 84 search strategies,according to the following formula:

E Z pq n=

where E is the sample error (0.03); Z is the constant relative to a95% confidence interval (1.96); p corresponds to the expectedproportion of systematic reviews showing insufficient evidence;and q is the complementary of p regarding the total number ofsystematic reviews (1 – p).

Statistical analysis

We used the Kruskal–Wallis analysis of variance by ranks andSAS software (SAS 9.1.3 Help and Documentation, SAS InstituteInc., Cary, NC, USA) for statistical analysis. We expressed thenumber of searches as totals, means, medians, ranges and standarddeviations. We considered a P value of less than 0.05 to be statis-tically significant.

ResultsWe analysed 120 search strategies, which generated 600 arrange-ment tests of the systematic reviews’ search strategies conducted atour university.

The statistics relating to the numbers of results retrieved by the120 search strategies are shown in Table 4.

With regard to the use of phrase searching parentheses, therewas no difference between the results with and without paren-theses in 100% of the sample analysed (P = 1.0). The number ofresults retrieved by the searches analysed was smaller with the useof double quotation marks, as well as with the use of truncation,compared with the standard strategy (P = 0.04 and P = 0.08,respectively). The number of results retrieved by the search strat-egies was exactly the same when comparing a simple search withsearch history tools, with the data showing no statistically signifi-cant difference (P = 1.0).

DiscussionIn this study, we compared the numbers of records retrieved byfour different tools using identical search strategies. In otherwords, the same keywords and Boolean operators were used to testphrase-searching parenthesis, double quotations marks and trun-cation in Medline via Pubmed to identify the best way to developsearch strategies for systematic reviews.

The desired outcome was a greater number of results retrievedby the search strategy, to map all the existing studies evaluating thequestion under investigation. However, in this study, we did notconsider the relevance of the records for each clinical question. Ifthat had been the case, the study should have had the participationof two or more expert researchers for each strategy analysed,as well as the development of questionnaires to appraise therelevance.

Brazier & Begley [7] compared the usefulness of the Medlineand CINAHL databases for students in nursing courses, as well asthe relevance of the retrieved results. The two databases retrieveda total of 1162 references, of which 88% were in Medline, 33% in

Figure 2 Search history combining the results afterwards, using obstructive sleep apnoea and adenotonsillectomy and high blood pressure as anexample.

E.S.N. Volpato et al. Testing search strategies

© 2013 John Wiley & Sons, Ltd. 119

CINAHL and 20% in both sources. The positive predictive valueof CINAHL was greater than that of MEDLINE, but MEDLINEproduced more than twice as many relevant references asCINAHL. The authors recommended the use of MEDLINE as thefirst choice of bibliographic database for any subject strictlyrelated to nursing.

We chose Medline because this database is the most frequentlyaccessed source in the health field, and it contains more than 21.6million records. Furthermore, because PubMed is a free resourcethat is developed and maintained by the National Center for Bio-technology Information, we decided to use this interface to test ourhypothesis.

In several publications of systematic reviews, we have oftennoticed the use of truncation, which allows the computer to searchfor multiple forms of a word. The generally accepted symbol fortruncation in some databases is an asterisk (*). However, after weran 120 searches with or without the use of truncation, we found thatthe numbers of results retrieved by the analysed searches weresmaller with truncation than without (P = 0.08). The same resultwas obtained regarding double quotation marks, which are used tosearch for terms as a phrase and to narrow the results of searches inwhich the results would be more numerous without double quota-tion marks (P = 0.04). Therefore, if researchers want to find themaximum results with a search strategy for a particular topic, theyshould not use double quotation marks.

Some studies have also questioned whether the results would bethe same using a simple search (Fig. 1) or the search history tools(Fig. 2) when searching in Medline through PubMed, the latter ofwhich combines the results afterward. We found that the results ofboth tools were identical with no statistically significant difference(P = 1.0).

If systematic reviews use rigorous methods to identify, criticallyappraise and synthesize relevant research studies, we also shouldbe aware of the best tools to implement an adequate search strat-egy, depending on the clinical question, to ensure that the resultswill be as current as possible and not biased.

Overall, there is no need to use phrase-searching parenthesis toretrieve studies; however, we recommend the use of double quo-tations marks when the investigator wants to retrieve papers in

which the term appears exactly the same to what is proposed inthe search form. Furthermore, we do not recommend the use oftruncation in search strategies for systematic reviews in theMedline literature database through PubMed to find interventionalstudies in the health field. Although the results of simple searchesand search history tools were the same, we recommend using thelatter as it provided us with greater flexibility to manipulate thepartial results.

AcknowledgementThis study was supported by the State of São Paulo ResearchFoundation – FAPESP.

References1. Brettle, A. J. & Long, A. F. (2001) Comparison of bibliographic data-

bases for information on the rehabilitation of people with severemental illness. Bulletin of the Medical Library Assocociation, 89 (4),353–362.

2. Lopes, I. L. (2002) Estratégia de busca na recuperação da informação:revisão da literatura. Ciência da Informação, 31 (2), 60–71.

3. Aleixandre-Benavent, R., González Alcaide, G., González De Dios, J.& Alonso-Arroyo, A. (2011) Sources of bibliographic information.Rationale for conducting a literature search. Acta Pediatrica Espanõla,69 (3), 131–136.

4. Suarez-Almazor, M. E., Belseck, E., Homik, J., Dorgan, M. &Ramus-Remus, C. (2000) Identifying clinical trials in the medical lit-erature with electronic databases: MEDLINE alone is not enough. Con-trolled Clinical Trials, 21, 476–487.

5. Eveillard, P. & Hannedouche, T. (2007) Recherche bibliographiquemédicale avec Medline–Pubmed: une approche pratique basée surl’exemple. Néphrologie e Thérapeutique, 3, 475–485.

6. Younger, P. & Boddy, K. (2009) When is a search not a search? Acomparison of searching the AME complementary health database viaEBSCOhost, OVID and DIALOG. Health Information and LibrariesJournal, 26, 126–135.

7. Brazier, H. & Begley, C. M. (1996) Selecting a database for literaturesearches in nursing: MEDLINE or CINAHL? Journal of AdvancedNursing, 24 (4), 868–875.

Table 4 Comparison of results retrieved by the 120 search strategies, considering the use of phrase searching parenthesis, double quotationsmarks, truncation and search history in the Medline through PubMed

Tools

Equal numberof retrievedresults

Smaller numberof retrievedresults

Greater numberof retrievedresults Number P value Mean Median Range SD

Phrase searchingparentheses

120 N/A* N/A* With 38 997.5 1.0 2521.2 374.0 28 664 5198.1Without 38 997.5 2521.2 374.0 28 664 5198.1

Double quotationsmarks

10 106 4 With 31 370.0 0.04 1696.1 181.5 20 638 3939.2Without 38 997.5 2521.2 374.0 28 664 5198.1

Truncation 17 96 7 With 31 940.0 0.08 1774.9 149.0 27 702 4351.8Without 38 997.5 2521.2 374.0 28 664 5198.1

Search history 120 0 0 With 38 997.5 1.0 2521.2 374.0 28 664 5198.1Without 38 997.5 2521.2 374.0 28 664 5198.1

*N/A, not applied because the results were identical with and without phrase searching parentheses.

Testing search strategies E.S.N. Volpato et al.

© 2013 John Wiley & Sons, Ltd.120