Literature review: How to search, evaluate, synthesize, and present the evidence

This workshop will focus on…

1. Searching through the literature for relevant sources;

2. Evaluating the findings from your search;

3. Synthesizing these findings; and

4. Presenting the results.

But WHY do we need to conduct a literature review?

A literature review can help us to…

• Address an information need• Find out whether the evaluation question has been

asked before• Find out what has been done in similar settings• Determine what best practices already exist• Discover what tools might be useful in our evaluation

What is the problem we need to solve?

What is the question we need answered?

Select first choice resource where an answer may be found

(e.g. the Cochrane library)

Design search strategy and carry out search

Appraise and summarize the evidence

Select second choice resource

Poor or insufficient evidence

Good Evidence

Assess the relevance of the evidence to your local situation

and target groups

Consider implications for your practice and resources, and

those of the organizations you work with

Apply the evidence to practice

Evaluate your practice No improvement

Improvement

© 2003, Evidence Network: What Works for Children?

The Evidence to Practice Process

Where do we start?

Searching through the LiteratureSearching through the Literature

Define the Problem1) The target population

E.g., Age, sex, ethnic group, diagnostic group

2) The intervention

The therapy that is of interest to your program

3) The outcome

The changes you would like to see in your target population

• Academic libraries• Online databases

• Primary source: The original publication of new data, results, and theories

Stice, E., Shaw, H., Burton, E., & Wade, E. (2006, April). Dissonance and healthy weight eating disorder prevention programs: a randomized efficacy trial. Journal of Consulting and Clinical Psychology, 74(2), 263-275. Retrieved September 18, 2008, from PubMed database.

• Secondary source: Summarizes or comments on primary sources in the context of the particular idea under study

Pratt, B.M. & Woolfenden, S.R. (2002, April). Interventions for preventing eating disorders in children and adolescents. Cochrane Database of Systematic Reviews, Issue 2. Retrieved September 18, 2008, from the Cochrane database.

Find the Information

Online databases with free content…• Centre for Reviews and Dissemination (CRD) http://www.crd.york.ac.uk/crdweb/

• Cochrane Database of Systematic Reviews (CDSR) http://www.cochrane.org/

• Turning Research Into Practice (TRIP) Database http://www.tripdatabase.com

• The International Network of Agencies for Health Technology Assessment http://www.inahta.org/

• British Medical Journal www.bmj.com/

• Directory of Open Access Journals (DOAJ) http://www.doaj.org/

• Substance Abuse and Mental Health Services Administration (SAMHSA) http://www.samhsa.gov/ebpWebguide/index.asp

• Centre of Excellence evidence database http://www.onthepoint.ca/EvidenceDB/

• PubMed http://www.ncbi.nlm.nih.gov/pubmed/

• York University Health Research Guide http://www.library.yorku.ca/ccm/rg/nk/health.jsp

• British Medical Association (BMA) http://www.bma.org.uk/ap.nsf/Content/Evidencebasedhealthinformationon

Online databases with limited free content…

• PsycINFO – American Psychological Association (APA) http://www.apa.org/psycinfo/

• Allied and Complementary Medicine Database (AMED) http://www.bl.uk/collections/health/amed.html

• EMBASE http://www.embase.com/

Databases available only by subscription…

Guidelines available online…• Guidelines are a rich source of evaluated evidence,

particularly if the question is about treatment or diagnosis of a relatively common medical condition

• Available for free:• UK National Electronic Library for Health Guidelines Finder

http://rms.nelh.nhs.uk/guidelinesfinder• US National Guideline Clearing House at

http://www.guidelines.gov• Guidelines International Network (GIN) at http://www.g-i-

n.net• BMJ Publishing Group’s Clinical Evidence

http://www.clinicalevidence.org/ceweb/conditions/index.jsp

Search through the Database

1. Use key words from your question as search terms

2. Use restrictions to refine the search • A clearly defined patient group and intervention are

the major parameters in most subject searches

3. Search by author, journal title etc.

Approaches to research…Qualitative

• In-depth information about people’s experiences

• Focus on subjective meanings

• Typically small samples• Emphasis on credibility

and trustworthiness

Quantitative

• Information about how different variables are related to one another

• Focus on objective measurement

• Typically large samples• Emphasis on causality,

reliability, validity

Types of studies you may find…

1. Systematic reviews2. Randomised controlled trials3. Quasi-experimental design4. Evaluation studies with non-experimental designs5. Case control studies6. Cohort studies7. Population surveys8. Qualitative research

Evaluating the ResultsEvaluating the Results

Initial questions to consider…

• Do the title and abstract suggest a fit between the source and your evaluation?

• Are the articles peer reviewed? • Is the study original?• Who is the study about? Is it reasonable to

expect that the results might apply to your target group?

• Are the claims made by the study plausible?

• Have the authors addressed all outcomes of interest?

• What does the study add to what we already know?

• The authors’ credentials: are the authors associated with the field of study? Do they have relevant clinical experience?

• Are there any issues related to “researcher bias” that are not addressed?

The Purpose and Method• Are the research questions clear, specific and

answerable?• Is the study design appropriate? Do the question,

method and analysis of results match up?• Is the sample appropriate?• If a comparison group was used, was it really

comparable to the group receiving the intervention?

• Did some people 'drop out' of the study, and if so, have the authors accounted for this in their conclusions?

Results and Conclusions• Is the material presented in a way that is

transparent and detailed in a way that can be easily examined and evaluated?

• How large is the effect of the intervention, if there is one?

• How precise is the estimate of the effect? How likely is it that the result was due to chance?

• Have the authors clearly shown how they came to their findings?

• Does the author explain study limitations?• Do the conclusions match the findings? Are the

conclusions supported by the analysis?

Synthesizing the FindingsSynthesizing the Findings

Initial questions to consider…

• What are the most relevant sources?• What are some common elements across sources?• What is unique about each source?• What are the key ideas/concepts being conveyed?• Why would this be important?• What are the limitations/gaps in the literature?

(Note: this is particularly important, because it speaks to your contribution to the literature – i.e., your work may help fill this gap)

Develop a Summary TableTitle and Publication Information

Type of Source and Approach

Overall Goal Main Ideas and Conclusion

Limitations

         

         

Presenting the FindingsPresenting the Findings

Planning the Literature Review

• What are the goals of the review?• Who is the audience?• What are your key messages?• How should the review be organized?

Tips for Writing the Literature Review• Use the right word in the right place• Avoid awkward, rambling and run-on

sentences• Avoid sentence fragments• Use clear, plain language• Always use examples or statements to

substantiate your point• Feel free to use an active voice

Next Steps

• Selecting measures?• Framework design?• Other ideas?

Visit our website for more information:

www.onthepoint.ca