quantitative research - london · pdf file · 2012-06-27quantitative research ziba...
TRANSCRIPT
Quantitative Research
Ziba Nadimi- Outreach Library Service Manager CNWL-NHS Camden Provider Services
June 2012
Learning Objectives
explain what is meant by the term critical appraisal
and why it is important
discuss the major features of quantitative research
and the key terminology associated with it
use a CASP checklist to critically appraise a
quantitative research article
What is critical appraisal?
“It usually comes as a surprise to students to learn
that some (the purists would say up to 99% of)
published articles belong in the bin and should not be
used to inform practice” (Greenhalgh 2006)
Not all papers are equal: some are good, some are
bad, most have strengths and weaknesses
“critical appraisal is concerned with the acquisition of necessary skills with which to discern clinical research papers accurately” (Ajetunmobi, 2002)
Understanding (in-depth reading)
CASP International-http://www.caspinternational.org/?o=1012
Dawes (2005) on critical appraisal
It may be easier to try to do it with someone else
It is hard to appraise a paper if you have not read it
Start with easy questions
Carry on even if you find flaws (there may still be useful information in the paper)
Look out for bias!
Review your appraisal – is it balanced, or have you “rubbished” the paper?
Can the results of the research be applied locally?
Quantitative vs. Qualitative Research?
“Both methods are valid if applied to appropriate
research questions, and they should complement
each other” (Bowling, 2002)
“The view that the two approaches are mutually
exclusive has itself become „unscientific‟”
(Greenhalgh, 2006)
Statistics
“…critical appraisal should not…be regarded as just another aspect of medical statistics. This misconception often leads to an overestimation of the level of statistical knowledge required for critical appraisal.” (Ajetunmobi, 2002)
However, for quantitative papers a little statistical knowledge can help a lot!
Definition
“…quantitative research should begin with an idea
(usually articulated as a hypothesis) which then,
through measurement, generates data and, by
deduction, allows a conclusion to be drawn.”
(Greenhalgh, 2006)
Quantitative Data Collection
Requires a specific protocol
Protocol is specified in advance of data collection.
Sample should be large. The larger the better.
Data Analysis
Statistical analysis
It should describe trends, compare groups &
relate variables
The analysis should also compare results with
past research
The Anatomy of a Research Paper
Introduction
Methods
Results and
Discussion
The Anatomy of a Research Paper
The introduction summarises the background to the
study - why
The methods describe how data was gathered and
analysed – vital for the critical appraisal of a paper
The results report findings objectively without
speculation or interpretation - what
In the discussion the authors interpret the findings
in light of the study design and other research. They
may also discuss limitations of their work – what it
means
Levels of Evidence – a general guide
Systematic Review
Identifies a health care intervention and examines the evidence as to whether or not this intervention works.
The authors locate, appraise and synthesise evidence from as many relevant scientific studies as possible.
They summarise conclusions about effectiveness.
They provide a collation of the known evidence on a given topic and identify gaps in research.
Statistical methods (meta-analysis) may or may not be used to analyse and summarise the results.
The Cochrane Library www.thecochranelibrary.com
Advantages of Systematic Reviews
Adhere to a strict design, therefore minimise the
chance of bias.
Provide a scientific rather than subjective
summarisation of literature.
Allow large amounts of information to be assimilated
quickly by healthcare providers, researchers, and
policymakers
Compare results of different studies to establish
generalisability of findings and consistency of results.
Disadvantages of Systematic Reviews
SRs are not the best way to address every question a
Cochrane reviews only include clinical trials
Papers with more interesting results are more likely
to be published (publication bias).
SRs may be biased due to the exclusion of relevant
studies (poor literature search) and the inclusion of
inadequate studies (no quality assessment).
SRs include an element of judgement
Randomised Controlled Trials- RCTs
Two or more interventions are compared by being
randomly allocated to participants.
Includes a control intervention or no intervention.
If possible should be single/double blinded.
Blinding in RCTs
Preventing those involved in a trial from knowing to
which comparison group, i.e. experimental or control,
a particular participant belongs.
The risk of bias is minimised.
Participants, caregivers, outcome assessors and
analysts can all be blinded.
Blinding of certain groups is not always possible. e.g.
surgeons in surgical trials.
Cohort Studies
An observational study
A defined group of people (the cohort) is followed
over time.
Outcomes are compared to examine people who
were exposed or not exposed to a particular
intervention.
A retrospective cohort study - identifies subjects from
past records and follows them to the present.
A prospective cohort study - assembles participants
and follows them into the future.
Case Control Studies
Compares people with a specific disease or outcome
of interest (cases) to people from the same
population without that disease or outcome (control).
Seeks associations between the outcome and prior
exposure to particular risk factors e.g. one group may
have been exposed to a particular substance that the
other was not.
They are usually concerned with causes of a disease.
They are usually retrospective.
Case Series
A study reporting observations on a series of
individuals, usually all receiving the same
intervention, with no control group.
Statistics
A small knowledge of statistical concepts will be
sufficient to appraise most quantitative papers
These include:
Risk (expressed as odds ratios or weighted mean
difference)
Confidence intervals
P values
This is a diagram from a Cochrane Review. It shows the risk ratios
(odds ratios) for a number of different clinical trails.
The vertical line has a ratio of one. This represents a point where
the treatment and comparison are the same. It is known as the
line of no effect.
Results to the left of the line of no effect = less of the outcome in the
experimental group. Odds ratio <1.
Results to the right of the line of no effect = more of the outcome in the
experimental group. Odds ratio >1.
It is important to note whether the outcome is good or bad.
The horizontal line is the confidence interval for each study. It usually represents 95% of the population sampled.
If CI crosses line of no effect = Inconclusive results.
Longer CI = Smaller study (less confident of results).
Shorter CI = Bigger study (more confident of results).
In a systematic review the diamond shows the meta-analysis: it
represents the statistical summary of the separate studies. This is
only possible if the studies are sufficiently similar (homogenous).
Weighted Mean Difference
The design of the diagram is very similar, but the “line of no effect” is set at zero. Results to the left of the line are expressed in negative numbers, for example -0.5
P Values
P for probability (ranging from zero to one).
The result could have occurred by chance if in reality
the null hypothesis was true.
The null hypothesis- the factor of interest (e.g.
treatment) has no impact on outcome (e.g. risk of
death).
P value of less than 0.05 means the likelihood of
results being due to chance is less than 1 in 20 =
“Statistically significant”.
References
•McGovern, D.P.B. etal, Evidence-based medicine, BIOS Scientific Publishers Ltd., 2001
•Greenhalgh, T., How to read a paper, 3rd ed., BMJ Publishing Group, 2006
•Kelsey, K.D., (Lecture 2) Quantitative and qualitative approaches to research - PowerPoint
Presentation
•Ward, L., Critical reading made easy:effectiveness and experience, University Hospitals of
Leicester NHS Trust- PowerPoint Presentation
•Jackson, N., Conducting systematic reviews of public health and health promotion interventions,
Cochrane Health Promotion and Public Health Field- PowerPoint Presentation
•The Cochrane Library‟s Glossary of Terms, Wiley InterScience
•Andrew Hayward, Critical Appraisal of Analytical Studies - PowerPoint Presentation
•Ajetunmobi, O. (2001) Making sense of critical appraisal London: Arnold
•Dawes, M. (2005) „Introduction to critical appraisal‟. in Dawes, M et al Evidence-based practice
(2nd ed) Edinburgh: Elsevier
•Sackett et al (1996) „Evidence based medicine: what it is and what it isn't‟ BMJ 312: 71-72
(13 January)
•Freshwater, D. (2005) Blackwell‟s Nursing Dictionary (2nd ed) Oxford: Blackwell
•Bowling, A. (1997) Research methods in health Buckingham: Open University Press