systematic literature review dr elena antonova. lecture aims types of literature reviews what is a...

53
Systematic Literature Review Dr Elena Antonova

Upload: jake-mclain

Post on 28-Mar-2015

270 views

Category:

Documents


6 download

TRANSCRIPT

Page 1: Systematic Literature Review Dr Elena Antonova. Lecture aims Types of literature reviews What is a systematic literature review Why do we need them How

Systematic Literature Review

Dr Elena Antonova

Page 2: Systematic Literature Review Dr Elena Antonova. Lecture aims Types of literature reviews What is a systematic literature review Why do we need them How

Lecture aims

• Types of literature reviews

• What is a systematic literature review

• Why do we need them

• How to be systematic about it

avoiding this state:

Page 3: Systematic Literature Review Dr Elena Antonova. Lecture aims Types of literature reviews What is a systematic literature review Why do we need them How

Learning outcomes

• Clarity about the purpose of lit review

• Understanding of the lit search process

• Appreciation of a systematic approach

• Ability to conduct a lit review

Page 4: Systematic Literature Review Dr Elena Antonova. Lecture aims Types of literature reviews What is a systematic literature review Why do we need them How

Publication food chain

•Anecdotal

•Expert opinion

•Case reports

•Case-control studies

•Cohort studies

•Randomised controlled trials

•Reviews and meta-analyses

Page 5: Systematic Literature Review Dr Elena Antonova. Lecture aims Types of literature reviews What is a systematic literature review Why do we need them How

Types of reviewsSystematic reviews:

Use explicit and rigorous methods to identify, critically appraise, and synthesize relevant studies

Narrative reviews:

Summaries of research that lack explicit descriptions of systematic methods

Meta-analysis: quantitative review using statistical analyses

Meta-ethnography: systematic review of qualitative data

Page 6: Systematic Literature Review Dr Elena Antonova. Lecture aims Types of literature reviews What is a systematic literature review Why do we need them How

Mulrow (1994) on systematic lit review:

“Through critical exploration, evaluation, and synthesis the systematic review separates the insignificant, unsound, or redundant deadwood in the medical literature from the salient and critical studies that are worthy of reflection”

Page 7: Systematic Literature Review Dr Elena Antonova. Lecture aims Types of literature reviews What is a systematic literature review Why do we need them How

What is systematic about it?

• Key search words

• Searching method

• Criteria for including/excluding references

• Clear structure in appraising the evidence

• Explicit discussion of limitations

Page 8: Systematic Literature Review Dr Elena Antonova. Lecture aims Types of literature reviews What is a systematic literature review Why do we need them How

Why a systematic literature review?

Ecclectism and the rise of BPS psychiatry:• proliferation of treatment approaches• growing number of RCTs published• limited NHS resources

-> need to identify effective evaluations to guide research investments and health care provision

Page 9: Systematic Literature Review Dr Elena Antonova. Lecture aims Types of literature reviews What is a systematic literature review Why do we need them How

Archie Cochrane 1909-1988

1972

Effectiveness and Efficiency: Random Reflections on Health Services

Emphasised the necessity of RCT studies

“It is surely a great criticism of our profession that we have not organised a critical summary, by specialty or subspecialty, adapted periodically, of all relevant randomised controlled trials”

The beginning…

Page 10: Systematic Literature Review Dr Elena Antonova. Lecture aims Types of literature reviews What is a systematic literature review Why do we need them How

Established in 1993 www.cochrane.org

• International, non-profit, independent organisation• Promotes evidence of the effects of therapies• Production and dissemination of systematic reviews

There are > 11,500 people working within The Cochrane Collaboration in over 90 countries.

50% are authors of Cochrane Reviews

Cochrane Collaboration

Page 11: Systematic Literature Review Dr Elena Antonova. Lecture aims Types of literature reviews What is a systematic literature review Why do we need them How

Another quote from Archie:

“I had considerable freedom of clinical choice of therapy: my trouble was that I did not know which to use and when. I would gladly have sacrificed my freedom for a little knowledge. I had never heard then of 'randomised controlled trials', but I knew there was no real evidence that anything we had to offer had any effect on tuberculosis, and I was afraid that I shortened the lives of some of my friends by unnecessary intervention."

Page 12: Systematic Literature Review Dr Elena Antonova. Lecture aims Types of literature reviews What is a systematic literature review Why do we need them How

Who needs literature reviews?

• Over two million!!! articles are published annually in the biomedical literature in over

20 000 journals.

• High quality information is not always available to everyone.

-> impossible for health care providers, researchers, and policy makers to keep up to date.

Page 13: Systematic Literature Review Dr Elena Antonova. Lecture aims Types of literature reviews What is a systematic literature review Why do we need them How

How is it helpful?

• Reduces large quantities of information to palatable pieces for digestion

• Minimises bias: systematic reviews and meta-analyses apply explicit scientific principles aimed at reducing random and systematic errors of bias

• Integrates up-to-date, reliable and critical biomedical information for use by:

• Practitioners: research-based practice • Decision and policy makers• Researchers (to avoid reinventing the wheel)

Page 14: Systematic Literature Review Dr Elena Antonova. Lecture aims Types of literature reviews What is a systematic literature review Why do we need them How

Further advantages of systematic reviews

• Assess consistency of relationships and effects• Among studies of the same or similar interventions • Across populations and settings• Across different diseases• Establish generalisability of scientific findings

• Identify variables affecting generalisability and consistency (e.g. sex, age, ethnicity)

• Facilitate the translation of medical research discoveries to clinical practice

Page 15: Systematic Literature Review Dr Elena Antonova. Lecture aims Types of literature reviews What is a systematic literature review Why do we need them How

How is it helpful to you?

• To locate previous studies in your subject area

• To find effective research methodologies

• To see if (and, if so, how well) your research topic has been tackled previously

• To put your research in the context of other work in this field

• To establish relevance

• To support bids for funding and sponsorship

Page 16: Systematic Literature Review Dr Elena Antonova. Lecture aims Types of literature reviews What is a systematic literature review Why do we need them How

The literature

Page 17: Systematic Literature Review Dr Elena Antonova. Lecture aims Types of literature reviews What is a systematic literature review Why do we need them How

Before entering the maze…

Formulate your question!

• Pragmatic, well-focused questions– Need to know what we are asking, in order to

know what studies we include in the review.

Page 18: Systematic Literature Review Dr Elena Antonova. Lecture aims Types of literature reviews What is a systematic literature review Why do we need them How

Stage 1: literature search

Page 19: Systematic Literature Review Dr Elena Antonova. Lecture aims Types of literature reviews What is a systematic literature review Why do we need them How

Key words

• Not too broad (‘scope creep’)

• Not too narrow (‘empty net’)

• Think of all possibly synonyms

• Check already retrieved papers for possible key words and their combinations

• Make sure to follow the search engines ‘rules’ in combining the key words

Page 20: Systematic Literature Review Dr Elena Antonova. Lecture aims Types of literature reviews What is a systematic literature review Why do we need them How

Databases

• PsychInfo (APA’s database)

• MedLine/PubMed

• Scopus: scientific, technical, medical and social sciences

• Google Scholar

• But Google definitely rocks!

Page 21: Systematic Literature Review Dr Elena Antonova. Lecture aims Types of literature reviews What is a systematic literature review Why do we need them How

Determining the year

• Generally, start with the year not included in the last literature review of the field

• If no previous reviews, go by when the first relevant study was published

• If overlapping, but not identical topic of review, ok to use the same studies

Page 22: Systematic Literature Review Dr Elena Antonova. Lecture aims Types of literature reviews What is a systematic literature review Why do we need them How

Literature Search Tree

Initial key word search

Retrieved papers cited by others

Retrieved papers references search

Unpublished data search ( literature, phd thesis, societal meetings)

Page 23: Systematic Literature Review Dr Elena Antonova. Lecture aims Types of literature reviews What is a systematic literature review Why do we need them How

Document you search in a detailed and explicit way

Atkins et al 2008

Page 24: Systematic Literature Review Dr Elena Antonova. Lecture aims Types of literature reviews What is a systematic literature review Why do we need them How

Stage 2: selecting papers

Page 25: Systematic Literature Review Dr Elena Antonova. Lecture aims Types of literature reviews What is a systematic literature review Why do we need them How

Method of selecting

• Decide on your inclusion/exclusion criteria

• This might depend on how advanced is the field (e.g. no-control studies, open-blind trials vs. RCTs)

• What demographics of participants might influence outcomes? (e.g. age, sex, ethnicity)

Page 26: Systematic Literature Review Dr Elena Antonova. Lecture aims Types of literature reviews What is a systematic literature review Why do we need them How

Details of all studies included in review

Page 27: Systematic Literature Review Dr Elena Antonova. Lecture aims Types of literature reviews What is a systematic literature review Why do we need them How
Page 28: Systematic Literature Review Dr Elena Antonova. Lecture aims Types of literature reviews What is a systematic literature review Why do we need them How

The write up

• Title

• Introduction

• Method

• Results

• Discussion

• Conclusion

• References

Page 29: Systematic Literature Review Dr Elena Antonova. Lecture aims Types of literature reviews What is a systematic literature review Why do we need them How

Introduction• Introduce the field of research

• State the question to be addressed

• Make reference to the previous reviews of the field

• Make clear why further review is need (i.e. number of new papers)

• Introduce any competing models/theories that the review aims to appraise

Page 30: Systematic Literature Review Dr Elena Antonova. Lecture aims Types of literature reviews What is a systematic literature review Why do we need them How

Introduction: specific points

– How important is the problem?

– Prevalence, incidence and impact of illness

– Is there any uncertainty about how to deal with the problem?

– What is the intervention supposed to achieve?

– What is the anticipated outcome of the intervention

– What are the models of efficacy of the intervention?

Page 31: Systematic Literature Review Dr Elena Antonova. Lecture aims Types of literature reviews What is a systematic literature review Why do we need them How

Method: design & procedures

Describe:

Search criteria and method

Selection criteria, including explicit justification for study exclusion

The scales or checklist by which studies are appraised

Any techniques of synthesis or analysis by which they are presented

Page 32: Systematic Literature Review Dr Elena Antonova. Lecture aims Types of literature reviews What is a systematic literature review Why do we need them How

Method: design & procedures

• Tabulate: - inclusion criteria

- sample size

- baseline patient characteristics

- withdrawal rate

- results of primary and secondary end points (for longitudinal studies)

Page 33: Systematic Literature Review Dr Elena Antonova. Lecture aims Types of literature reviews What is a systematic literature review Why do we need them How

Method

• Define participants

• Define interventions

• Define comparisons

• Define outcomes

Page 34: Systematic Literature Review Dr Elena Antonova. Lecture aims Types of literature reviews What is a systematic literature review Why do we need them How

Participants •How is the disease/condition defined?

•What are the most important characteristics that describe the participants relevant to your review?

•Are there any relevant demographic factors? (e.g. age, sex, ethnicity)

•What is the setting? (e.g. hospital, community etc)

•Who should make the diagnosis?

•Are there any co-morbidities to be excluded?

•Are there any other types of people who should be excluded from your review (because they are likely to react to the intervention in a different way)?

•How will studies involving only a subset of relevant participants be handled?

Page 35: Systematic Literature Review Dr Elena Antonova. Lecture aims Types of literature reviews What is a systematic literature review Why do we need them How

Interventions

•What are the experimental and control (comparator) interventions of interest?

•Does the intervention have variations (eg dosage, mode of delivery, personnel who deliver it, frequency of delivery, duration of delivery, timing of delivery)?

•Are all variations to be included (for example is there a critical dose below which the intervention may not be clinically appropriate)?

•How will trials including only part of the intervention be handled?

•How will trials including the intervention of interest combined with another intervention (co-intervention) be handled?

Page 36: Systematic Literature Review Dr Elena Antonova. Lecture aims Types of literature reviews What is a systematic literature review Why do we need them How

Comparisons

What are you interested in comparing the intervention to?

Depends on the primary question of the review:

Are you only interested in whether the intervention offers benefit over the natural course of the disorder (ie a comparison to placebo or no treatment)

or are you interested in whether the intervention offers benefit over other interventions.

Page 37: Systematic Literature Review Dr Elena Antonova. Lecture aims Types of literature reviews What is a systematic literature review Why do we need them How

Outcome measures

•How do you think it is important to measure change with respect to this intervention in this population?

•List all the outcomes you are including in your review. •Primary and Secondary

•Consider outcomes relevant to all potential decision-makers, including economic data.

•Should cover beneficial and adverse effects

Page 38: Systematic Literature Review Dr Elena Antonova. Lecture aims Types of literature reviews What is a systematic literature review Why do we need them How

Have a method to your method

• Reproducibility

– Ideally the transparency of methods used should mean that another researcher addressing the same question will identify the same set of studies and arrive at the same overall conclusion.

Page 39: Systematic Literature Review Dr Elena Antonova. Lecture aims Types of literature reviews What is a systematic literature review Why do we need them How

Results

• Normally the largest section of your write up

• Aim for a systematic grouping of the studies, e.g. results of controlled vs no-control trials, open-blind vs double-blind, or some other criteria

Page 40: Systematic Literature Review Dr Elena Antonova. Lecture aims Types of literature reviews What is a systematic literature review Why do we need them How

Practical tips

• Use a method which suits you best:– brief summary of each study for your notes– map the labs and their studies

!Make sure you know the labs and their ‘stories’!

Have a story to tell, not just summaries

Page 41: Systematic Literature Review Dr Elena Antonova. Lecture aims Types of literature reviews What is a systematic literature review Why do we need them How

Discussion & Conclusions

• Summaries the main results• Appraise the results (power, generasibility)• Clinical significance• Appraise in relation to the models/theories of

phenomenon (e.g. disease)• Discuss limitations and biases• Give guidance to the future research• Conclude with a succinct summary of main points

Page 42: Systematic Literature Review Dr Elena Antonova. Lecture aims Types of literature reviews What is a systematic literature review Why do we need them How
Page 43: Systematic Literature Review Dr Elena Antonova. Lecture aims Types of literature reviews What is a systematic literature review Why do we need them How

Major sources of bias in systematic reviews

1) Publication biasNegative findings may not get publishedPositive findings may get published more than once

2) Selection bias

Inclusion and exclusion criteria can create biasOverall quality of trialsCombinability of treatments, patients, outcomes, and lengths of follow up.

3) Language bias (i.e. English Language) Authors might be more likely to report positive findings in an international English language journal and negative findings in a local journal

Page 44: Systematic Literature Review Dr Elena Antonova. Lecture aims Types of literature reviews What is a systematic literature review Why do we need them How

Disadvantages of systematic reviews

• Synthesis may disguise or oversimplify important distinctions between primary studies with regard to inclusion/exclusion criteria or the nature of an intervention

• Reviews may make it difficult for practitioners to apply the results of studies to the specific characteristics of the situation in which they find themselves (over-generalisation)

• Reviews of similar topics may appear to reach different conclusions depending on the precise form of the "review question"

Page 45: Systematic Literature Review Dr Elena Antonova. Lecture aims Types of literature reviews What is a systematic literature review Why do we need them How

collecting,reviewing presenting

Statistical techniques used to extract, combine and summarise data

Systematic reviews vs. Meta-analysis

Different statistical methods exist for combining the data, but there is no single "correct" method.

Page 46: Systematic Literature Review Dr Elena Antonova. Lecture aims Types of literature reviews What is a systematic literature review Why do we need them How

Advantages of meta-analytic techniques

Statistical power Very large trials (maybe 10,000 subjects) needed to obtain sufficient power to "prove" the difference in a single study are slow and very expensive to run.

Several smaller studies might suggest improvement in clinical outcomes with moderate-poor statistical significance .

Combining smaller studies can offer the sufficient power needed to clearly interpret data.

Cumulative evidenceCan be statistically combined to facilitate translation of findings to clinical practice

Applicability By combining studies from different environments, meta-analysis can make generalisations to other populations more justifiable (i.e. if effectiveness is demonstrable consistently across a range of studies).

Page 47: Systematic Literature Review Dr Elena Antonova. Lecture aims Types of literature reviews What is a systematic literature review Why do we need them How

Advantages over a systematic literature review

• Precision of resultsResults can be presented not as subjective discussions of association, trend and relationship, but as precise numerical values, with confidence intervals and with precise estimates of the intervention's effect.

• ObjectivityMeta-analysis includes certain protections that reduce, (but not eliminate), opportunities for subjectivity.

• Quality control Bigger and more methodologically sound studies can be weighted to have more influence than smaller poorer studies.

Page 48: Systematic Literature Review Dr Elena Antonova. Lecture aims Types of literature reviews What is a systematic literature review Why do we need them How

Points of caution for meta-analysis

• Many treatments produce only modest, but useful, improvements in clinical outcomes.

• Cases in which the meta-analysis is dramatically better than individual trials or common clinical experience should be viewed with caution.

Page 49: Systematic Literature Review Dr Elena Antonova. Lecture aims Types of literature reviews What is a systematic literature review Why do we need them How

HJ Eysenck (1995):

"If a medical treatment has an effect so recondite and obscure as to require meta-analysis to establish it, I would not be happy to have it used on me. It would seem better to improve treatment, and the theory underlying the treatment“

Page 50: Systematic Literature Review Dr Elena Antonova. Lecture aims Types of literature reviews What is a systematic literature review Why do we need them How

Misguidance on magnesium

In 1993 a meta-analysis concluded that magnesium infusion is effective in reducing mortality from heart attacks, and that it should be adopted as good practice The meta-analysis encompassed 1300 patients and found a 55% risk reduction in the odds of death.

Two years later, a randomised clinical trial, which had recruited over 50,000 patients found against it. Magnesium infusions ceased to have any role on coronary care units.

Both sides raised issues: those favouring magnesium said it was given too late in the large trial; opponents said that the 1993 meta-analysis was biased.

In fact, the magnesium meta-analysis may have suffered from selection and publication bias, as later shown by hand searching and use of a funnel plot. Some trials may have been biased by faulty methodology. One trial which appeared to favour magnesium, when re-analysed using a more rigorous analysis, ceased to favour magnesium.

Page 51: Systematic Literature Review Dr Elena Antonova. Lecture aims Types of literature reviews What is a systematic literature review Why do we need them How

Summary

• A systematic review is an overview of primary studies that uses explicit and reproducible methods.

• A meta-analysis is a mathematical synthesis of the results of two or more primary studies that have addressed the same hypothesis in the same way.

Page 52: Systematic Literature Review Dr Elena Antonova. Lecture aims Types of literature reviews What is a systematic literature review Why do we need them How

Conclusion: its all in the method

• Two reviews could disagree

• Conflicting results of review/meta-analyses are often due to methodological differences

Judgement of quality of studies for inclusion

Summing-up evidence

-> be transparent about your method!

Page 53: Systematic Literature Review Dr Elena Antonova. Lecture aims Types of literature reviews What is a systematic literature review Why do we need them How

Useful references and sources• Greenhalgh T (1997) Papers that summarise other papers

(systematic reviews and meta-analyses). Br Med J 315: 672-5.

• Cook DJ, Mulrow CD, Haynes RB (1997) Systematic Reviews: Synthesis of Best Evidence for Clinical Decisions. Ann Intern Med 126:364-371

• Mulrow CD (1994) Rationale for systematic reviews BMJ 309:597-599 or

• Mulrow CD, Cook DJ, Davidoff F (1997) Systematic Reviews: Critical Links in the Great Chain of Evidence. Ann Intern Med, 126:389-391.

http://ssrc.tums.ac.ir/systematicreview/

www.shef.ac.uk/scharr/ir/units/systrev/index.htm

www.cochrane.org