searching for relevant studies interactive case study quiz: c. michael white, pharm.d., fcp, fccp...

Searching for Relevant Studies Searching for Relevant Studies Interactive Case Study Quiz: Interactive Case Study Quiz:

C. Michael White, Pharm.D., FCP, FCCPC. Michael White, Pharm.D., FCP, FCCP

Professor and DirectorProfessor and DirectorUniversity of Connecticut / Hartford HospitalUniversity of Connecticut / Hartford Hospital

Evidence-based Practice CenterEvidence-based Practice Center

Author has no actual or potential conflicts of interest in relation to this activity

Interactive Case Quiz: Instructions

Open this presentation as a slideshow. This will activate the hyperlinks.

When you come to a decision slide, choose the red box corresponding to the correct choice. If you are correct you will be directed forward in the case. If you are incorrect, you will be directed back to the decision slide to choose again.– Click on forward hyperlinks (Red Boxes) to follow

through case based on your responses – Click on home hyperlink (Blue House Icon) to go

back to the last correct step in the series

Case Study Quiz: Searching the Literature

You are interested in evaluating the benefits and harms associated with the use of thrombectomy devices in acute coronary syndromes. You need to determine what literature is available to conduct your review.

Participation Quiz: Question 1

You decide to conduct a systematic review (SR). What are two ways in which SRs differ from narrative reviews?

They are the same thingSystematic reviews use comprehensive

searches and explicit methodsSystematic reviews always employ

quantitative synthesis (meta-analysis

They are the Same

This is incorrect, narrative reviews do not, as a rule, employ comprehensive literature searches or explicit methods for conducting the review

[Click on Blue Box to go Back]

Comprehensive Search and Explicit Methods

You are correct! Systematic reviews employ comprehensive searches and use explicit methods for conducting the review.

[SELECT RED BOX]

Meta-Analysis

This is incorrect. While some systematic reviews employ quantitative synthesis such as meta-analysis, others do not.



So a comprehensive search is required. When conducting this search, what do you need to assure?

You balanced precision and recall

You have gotten every possible relevant citation regardless of the workload involved

That you limit the search so that only relevant citations are found

Precision and Recall

This is correct! Precision is the proportion of retrieved articles that are relevant while recall is the proportion of potentially relevant articles retrieved by the search. You want to employ methods to maximize precision and recall. Overly restrictive searches increase recall but compromise precision. Overly broad searches increase precision but compromise recall.

[SELECT RED BOX]

You’ve Gotten it All

This is incorrect. While it is laudable to want to acquire every possible relevant citation, this approach compromises precision and makes the SR unnecessarily onerous. You need to balance precision and recall.


You Only Have Relevant Citations in your Search

This is incorrect. Every citation found is relevant but it is likely that many relevant citations were missed by this overly restrictive search strategy. You need to balance precision and recall.



Given the need to balance precision and recall. What should you do before initiating a search?

Enter in the search as “thrombectomy AND acute coronary syndromes”

Enter the search as “thrombectomy OR acute coronary syndromes”

Understand the topic, devise an analytic framework, pose clearly defined key questions, and understand the scope of the review before devising your strategy

Thrombectomy AND ACS

This is incorrect. Searching based only on readily known disease and treatment information would be problematic. You need to understand the topic area before starting your search.


Thrombectomy OR ACS

This is incorrect. This would be a voluminous search for citations for thrombectomy anywhere in the body in addition to any citation published on acute coronary syndromes.


Due Diligence

This is correct. Understanding the topic and devising an analytic framework drive clearly defined key questions. Combining this with a understanding the scope of the project allows an optimal balance between precision and recall when sculpting the search.

[SELECT RED BOX]

The Power of Due Diligence

Your due diligence pays off, you now realize that your initial search “thrombectomy AND acute coronary syndrome” would have the compromised recall. ((Thrombectomy OR MerciClot.mp OR “other devices names”) AND (acute coronary syndromes OR myocardial infarction OR angina, unstable)) would assure better recall without appreciably impacting precision.


You try to decide whether to do one search or two searches (one for benefits and another for harms). Which of the following would cause you to perform two searches?

Performing two searches is redundant and should be avoided

You decide a priori to only allow RCT data when evaluating benefits but allowing RCTs and observational trials for harms

If the available literature base on the topic is extensive

Never Do Multiple Searches

This is incorrect. If the size of the available literature is small, it may be appropriate to conduct one single broad search. However, if the literature is extensive and observational trials are allowed in the harms but not the benefits evaluations, a single broad search would reduce precision for studies reviewing efficacy endpoints without appreciably improving the recall.


Separate Benefits and Harms Search

This is correct! Benefits searches is usually limited to randomized controlled trials since they have greater internal validity. Searches usually include the disease in question, the intervention, and a hedge for controlled trials. Harms searches are usually broader and include observational trials. Searches usually include the harm (or subheadings for general harms) and the intervention. When the available literature is extensive, two searches improves precision without appreciably impacting recall.

[SELECT RED BOX]


You need to decide which databases to use. The minimum requirement for a thorough search of the literature would include which two databases?

MEDLINE + TOXNET

MEDLINE + CINAHL

MEDLINE + Cochrane CENTRAL

MEDLINE + TOXNET

This is incorrect. Searching MEDLINE and the Cochrane CENTRAL databases are needed to fulfill the minimum broad search requirements. TOXNET is a database for adverse events or toxicology and can be used in addition to these others databases.


MEDLINE + CINAHL

This is incorrect. Searching MEDLINE and the Cochrane CENTRAL databases are needed to fulfill the minimum broad search requirements. CINAHL is a database that focuses on nursing literature and can be added to these other databases.


MEDLINE + Cochrane CENTRAL

This is correct. Combining MEDLINE with Cochrane CENTRAL would be a minimum requirement for a thorough search of the literature. Using MEDLINE alone may miss between 12% and 25% of relevant citations found in other databases and may produce a North American bias.

[SELECT RED BOX]


You decide to use hand searching of references from identified systematic reviews, studies, and abstract booklets from prominent meetings within the field of interest. Why would you do this?

Hand searching can capture citations that are not indexed or are improperly indexed.

Limiting hand searching to these sources can target this time intensive activity to areas where the yield will be the greatest.

Both are Correct

You are correct! Hand searching can yield up to 13-25% of citations that do not come up in database searches. Since hand searching is the most time intensive search strategy, it should be targeted to maximize the yield.

[SELECT RED BOX]


You decide to search the FDA website and ClinicalTrials.gov to identify studies that might be appropriate for your systematic review. These website are an example of:

Black literatureGrey literatureSIPs

Black Literature

This is incorrect. You chose the wrong color.


Grey Literature

This is correct! Grey literature can be important for identifying trials that were conducted but are, as of now, unpublished or incompletely published. Searching for grey literature is a useful way to minimize publication bias.

[SELECT RED BOX]

SIPs

This is incorrect. SIPs are Scientific Information Packets provided by the manufacturer of a drug or device.



You conduct a thorough search of MEDLINE and Cochrane CENTRAL, use two searches (one for benefits and another for harms, and employ targeted hand searching and grey literature evaluation. What is the last thing you need to remember to do germane to searching.

Only choose citations that agree with your preconceived notions

Selectively report aspects of your search strategyProvide transparent reporting of search

strategies and the citations you identified

Preconceived Notions

This is incorrect. This would be akin to a narrative review and should not be used in a systematic review.


Selective Reporting

This is incorrect. In order to build confidence in the systematic review results, transparent and defensible methods need to be employed. You need to report enough information about your search strategy so that a person, skilled in the art, could reproduce it.


Transparent Reporting

This is correct! Transparent reporting is important in supporting the credibility of systematic review results. Reporting of search strategies and disposition of the citations should allow a person, skilled in the art, to reproduce the search you conducted.

[SELECT RED BOX]

Congratulations!

You have successfully maneuvered through the pitfalls of literature searching. A comprehensive understanding of the topic, the use of multiple searches, multiple databases, use of hedges, use of hand searching and grey literature, and development of a transparent reporting structure will give readers confidence in your systematic review.

searching for relevant studies interactive case study quiz: c. michael white, pharm.d., fcp, fccp...

Documents