systematic approaches to literature reviewing

Systematic Approaches to Literature Reviewing

Dr. Mark Matthews Student Learning

Development

mark.matthews@tcd.ie

Systematic Approaches to Literature Reviews

Dr. Mark Matthews

Student Learning Development

http://student-learning.tcd.ie

Systematic Approaches to Literature Reviewing

Dr. Mark Matthews Student Learning

Development

mark.matthews@tcd.ie

Finding your way in the woods

Systematic Approaches to Literature Reviewing

Dr. Mark Matthews Student Learning

Development

mark.matthews@tcd.ie

Finding your way in the woods

Systematic Approaches to Literature Reviewing

Dr. Mark Matthews Student Learning

Development

mark.matthews@tcd.ie

And making your

way back

Workshop Overview

explain elements of the systematic review process

explore how these might be used or adapted to support: - a thesis literature review; - approaches to keeping up-to-date with the literature through a PhD

contextualise this within other approaches to managing and working with the literature

1 Minute Big Picture Interviewer

Interview partner and take notes

The Literature Review ?

“Literature reviews …… introduce a topic, summarise the main issues and provide some illustrative examples.”

from www.eppi.ioe.ac.uk

Agree? Disagree?

If they are to be considered a reliable source of research evidence they should record how the primary studies were sought and selected and how they were analysed to produce their conclusions.

Readers need to be able to judge whether all of the relevant literature is likely to have been found, and how the quality of studies was assessed.

1 Agree? 5 Disagree? 10

The Literature Review ?

1. What is a Literature Review?2. Why do we do Literature Reviews?3. What are the main challenges?

Discussion

Literature Matters

From Holbrook et al (2007)

disciplinary perspective

connection to findings

coverage

working understanding

critical appraisal

scholarlinessliterature use

Means you need to be: critical = evaluate what you read analyse = extract differing information from what

you read synthesise = show relationships between

studies/sources; differing definitions, concepts, theories etc.

evaluate = methodological approaches used

(Hart, 2005)

Systematic Reviews

a review of a clearly formulated question that uses systematic and explicit methods to identify, select and critically appraise relevant research, and to collect and analyse data from the studies that are included within the review. Statistical methods (meta-analysis) may or may not be used to analyse and summarise the results of the included studies. 

from www.sebc.bangor.ac.uk

Student Learning Development, TCD 15

Systematic v.s. Narrative reviews

• Agreed standards• High degree of focus• Minimise bias

• Context and differences• Bias of empirical studies

Strengths of both approaches

“For some review topics, however, the strengths of the SR may turn into weaknesses…do not allow for comprehensive coverage” but “narrative reviews do not reveal how the decisions were made about relevance of studies”

‘Balancing the strengths of systematic and narrative reviews’ (Collins & Fauser, 2004) p. 103-104.

Key features of the systematic review process you might want to adopt or adapt–

1. Explicit and transparent methods2. a standard set of stages

3. Accountable, replicable and updateable

Seven steps (of a Cochrane Review)

1. Formulating a problem 2. Locating and selecting studies 3. Critical appraisal of studies 4. Collecting data 5. Analyzing and presenting results 6. Interpreting results 7. Improving and updating reviews

Workflow for Literature Reviews

1. Search 2. Assess3. Read4. Write

1. Formulating a Problem

1. Formulating a Problem

Example Questions?

Standard SR question containssubject- intervention- outcome- comparator

Does drinking coffee raise people’s blood pressure

(when compared to chocolate)?

Standard SR question containssubject- intervention- outcome- comparator

2. Locating and Finding Research

Where do I start?

Previous ThesesReview PapersRecent Conference PapersSubject librarianSupervisor

a broad but defined, systematic sweep

Defined search terms – record recall and precision

Defined search arena - e.g. databases, citation indices, reference lists from primary and review articles, grey literature, conference proceedings, research registers, the internet, individual researchers/practitioners

Other broad search limits, e.g. language, date,

TIPS! Document the search protocol and record what

research was found

Systematically manage the

search output, e.g. using

endnote

Phase 1- Identify the Research

Phase 2- Selection

select from research using criteria related to your research question

Develop inclusion or exclusion statements, these might relate to study outcomes, research design, methods used, population worked with etc.

e.g. studies with a mixed population of men and womene.g. random control trials onlye.g. maximum exposure time of 10mins

TIPS! Document the

statements and

their purpose (might be

pragmatic or research related)

Date Database Keywords Results

1/11/11 2pm

ACM Digital Library

Role play, role-play, role playing, role-playing, bodystorming, prototyping

10 articles

4/11/11 SpringerLink Role play,Role-play,Role-playing, Fish bowel

5 articles1 book

10/11/11 Scopus

Search Log

Running the search

1. How many titles and abstract can you check? 2. How easy will it be to decide to accept or reject a

record?3. Record the reason for rejection for “Excluded studies”

Don’t stop searching when you’ve stopped searching

3. Critical appraisal of studies

“Assessing the quality of methodology is a critical part of the systematic review process”

No standard approach but there are hierarchies in fields of study

Student Learning Development, TCD 32

3. Critical appraisal of studies

What would be appropriate to consider when critically

appraising research in your area?

3. Critical appraisal of studies

disciplinary perspective

connection to findings

coverage

working understanding

critical appraisal

scholarlinessliterature use

disciplinary perspective

connection to findings

coverage

working understanding

critical appraisal

scholarliness

disciplinary perspective

connection to findings

coverage

working understanding

critical appraisal

scholarlinessliterature use

No standard approach but there are hierarchies in fields of study

Student Learning Development, TCD 34

4. Collect data & analyse

• Evaluate• Synthesise results of literature review

– Tables to compare– Descriptive– Meta-analysis

Student Learning Development, TCD 35

Author Year Aim Methods Conclusions Critique

Student Learning Development, TCD 37

Write up literature review - Structure

BackgroundPurpose/Research questionMethodFindings DiscussionImplications/Recommendations

Voice

Writer as tour guide - voice

Writing Style

1. Assess the value of the literature2. Explain the context research takes place3. Emphasise limitations of existing

research4. Tell a story

Writing Style

IntroductionWhat I will show you?Why?

BodyWhy this area?Don’t leave reader to fill gaps

ConclusionWhat we have seen?How this is relevant to research?

Research evaluation

Evaluation and Literature Review

Comprehensive Literature Review

Coherent synthesis of past and present

research in the domain of study

Source: Dr Hazel Hall, Napier University

What are the main conclusions on

previous research in this area?

What are the key areas of debate in this

area?

Which aspects of this work are of most relevance to my

study?

What are the key concepts in this area?

What have been the main research

questions?

Where is existing knowledge “thin”?

How is this topic approached by

others?Where are the gaps in

literature?

What are the main

perspectives on this topic in

previous research?

Do parallel literatures exist for this topic?

Which discussions?

Which sub-themes? Which writers?

Which work is subject to

challenge?

Who are these

“others”?

Which existing work could be

extended?

In which subject areas has the topic

been studied?

Challenges in Conducting Literature Reviews

Other ideas and options

Speed reading

Endnote

Databases in my area

Google Alerts

SR websites

Centre for Evidence-based Conservation -http://www.cebc.bangor.ac.uk/

Centre for Reviews and Dissemination (medical) -http://www.york.ac.uk/inst/crd/index.htm

Cochrane Collaboration (international- medical) -http://www.cochrane.org/

EPPI-Centre, Institute of Education - http://eppi.ioe.ac.uk/cms/

Social Policy and Social Care -http://www.york.ac.uk/inst/chp/srspsc/index.htm

If you read one article, an example here of a review of the ways studies in reviews are appraised - http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=521688

If you prefer power-point, how about this one on mixed method reviews http://www.ccsr.ac.uk/methods/events/challenges/documents/JamesThomasESRCMethodologicalchallenges.ppt

Systematic Approaches to Literature Reviewing

Dr. Mark Matthews Student Learning

Development

mark.matthews@tcd.ie

Finding your way in the woods

Student Learning Development

Thank you for your time

Visit our website at: http://student-learning.tcd.ie