is it true: evaluating medical reviews. “the review article itself should be the product of...
TRANSCRIPT
![Page 1: Is it True: Evaluating Medical Reviews. “The review article itself should be the product of scientific investigation in which the participants are original](https://reader036.vdocuments.mx/reader036/viewer/2022062718/56649e805503460f94b8486b/html5/thumbnails/1.jpg)
Is it True: Evaluating Medical Reviews
![Page 2: Is it True: Evaluating Medical Reviews. “The review article itself should be the product of scientific investigation in which the participants are original](https://reader036.vdocuments.mx/reader036/viewer/2022062718/56649e805503460f94b8486b/html5/thumbnails/2.jpg)
“The review article itself should be the
product of scientific investigation in which
the participants are original investigations
(research) rather than patients”
![Page 3: Is it True: Evaluating Medical Reviews. “The review article itself should be the product of scientific investigation in which the participants are original](https://reader036.vdocuments.mx/reader036/viewer/2022062718/56649e805503460f94b8486b/html5/thumbnails/3.jpg)
Usefulness
Work: Low, good source for POEMs Relevance: If title and abstract or article
conclusion hold promise of POEM, continue Validity: Uncertain
![Page 4: Is it True: Evaluating Medical Reviews. “The review article itself should be the product of scientific investigation in which the participants are original](https://reader036.vdocuments.mx/reader036/viewer/2022062718/56649e805503460f94b8486b/html5/thumbnails/4.jpg)
Reviews- Three Basic Types
Textbooks
Academic Reviews
“Translation” Journals
![Page 5: Is it True: Evaluating Medical Reviews. “The review article itself should be the product of scientific investigation in which the participants are original](https://reader036.vdocuments.mx/reader036/viewer/2022062718/56649e805503460f94b8486b/html5/thumbnails/5.jpg)
Textbooks
Collection of review articles Minimal, if any, supporting evidence Questionable validity, long lag time to publish
• Average 1-2 years
Most useful for retracing, less hunting and foraging
![Page 6: Is it True: Evaluating Medical Reviews. “The review article itself should be the product of scientific investigation in which the participants are original](https://reader036.vdocuments.mx/reader036/viewer/2022062718/56649e805503460f94b8486b/html5/thumbnails/6.jpg)
Academic Reviews
Summary: “Broadly paint landscape” Validity uncertain- begin with conclusions and
find supporting references References often inaccurate and out of date **Expertise of author varies inversely with
quality of review- Oxman/Guyatt**• More later
Must confirm POEMs with original research, increasing work
![Page 7: Is it True: Evaluating Medical Reviews. “The review article itself should be the product of scientific investigation in which the participants are original](https://reader036.vdocuments.mx/reader036/viewer/2022062718/56649e805503460f94b8486b/html5/thumbnails/7.jpg)
Academic Reviews
Synthesis: Systematic reviews• Meta-analysis or overviews
Answer one or two specific questions Review primary literature with strict criteria Conclusions supported by available evidence Meta-analysis: Achieve power not possible by
single study
![Page 8: Is it True: Evaluating Medical Reviews. “The review article itself should be the product of scientific investigation in which the participants are original](https://reader036.vdocuments.mx/reader036/viewer/2022062718/56649e805503460f94b8486b/html5/thumbnails/8.jpg)
Academic Reviews
Excellent source for hunting and foraging
The Cochrane Library - “Database of Systematic Reviews”
Clinical Evidence (BMJ-BMA)
Clinical Inquiries (FPIN)- SORT
AFP EB Reviews- SORT
Dynamed - SORT
Essential Evidence Plus- SORT
![Page 9: Is it True: Evaluating Medical Reviews. “The review article itself should be the product of scientific investigation in which the participants are original](https://reader036.vdocuments.mx/reader036/viewer/2022062718/56649e805503460f94b8486b/html5/thumbnails/9.jpg)
Translation Journals
Quick reads for retracing and sporting Low work, but with low validity, may be zero
usefulness Hunting/foraging: Entering jungle on
starless night
![Page 10: Is it True: Evaluating Medical Reviews. “The review article itself should be the product of scientific investigation in which the participants are original](https://reader036.vdocuments.mx/reader036/viewer/2022062718/56649e805503460f94b8486b/html5/thumbnails/10.jpg)
Translation Journals
Common POEMs need original data for
verification, greatly increasing work
Watch for “weasel words”, based on DOEs
and anecdotes
• “it seems”, “may be effective”, “so one may assume”, “it appears”, “in my experience”
![Page 11: Is it True: Evaluating Medical Reviews. “The review article itself should be the product of scientific investigation in which the participants are original](https://reader036.vdocuments.mx/reader036/viewer/2022062718/56649e805503460f94b8486b/html5/thumbnails/11.jpg)
Weasel Words
![Page 12: Is it True: Evaluating Medical Reviews. “The review article itself should be the product of scientific investigation in which the participants are original](https://reader036.vdocuments.mx/reader036/viewer/2022062718/56649e805503460f94b8486b/html5/thumbnails/12.jpg)
Translation Journals
“Buyer Beware: Unsystematic reviews lead to unsystematic conclusions. Readers looking for a shortcut to understanding evidence about health problems and patient care should at least look for reviews by those who have not taken shortcuts”
![Page 13: Is it True: Evaluating Medical Reviews. “The review article itself should be the product of scientific investigation in which the participants are original](https://reader036.vdocuments.mx/reader036/viewer/2022062718/56649e805503460f94b8486b/html5/thumbnails/13.jpg)
Determining the usefulness of reviews
Onto the worksheets!
![Page 14: Is it True: Evaluating Medical Reviews. “The review article itself should be the product of scientific investigation in which the participants are original](https://reader036.vdocuments.mx/reader036/viewer/2022062718/56649e805503460f94b8486b/html5/thumbnails/14.jpg)
Reviews: Determining Relevance
A. Addressing specific clinical question?
A. Patient-oriented evidence?
B. Common problem?
C. Change your practice?
![Page 15: Is it True: Evaluating Medical Reviews. “The review article itself should be the product of scientific investigation in which the participants are original](https://reader036.vdocuments.mx/reader036/viewer/2022062718/56649e805503460f94b8486b/html5/thumbnails/15.jpg)
Reviews: Determining Validity
Answer ALL worksheet questions Stop = “fatal flaw” Notice how hard this is! Average time for a
good systematic review- 2 years!• Much different from your “usual review/CME talk”
![Page 16: Is it True: Evaluating Medical Reviews. “The review article itself should be the product of scientific investigation in which the participants are original](https://reader036.vdocuments.mx/reader036/viewer/2022062718/56649e805503460f94b8486b/html5/thumbnails/16.jpg)
Worksheet Qs: Finding the studies
Clearly stated?• Terms appropriate? MESH-linked? None missing?
Comprehensive?• Medline + another
• MEDLINE misses >50% of articles• Cochrane registry is especially good source • Science Citation Index
• Bibliographic review • Unpublished literature
• conference abstracts, personal correspondence with important investigators or pharmaceutical companies
Done by more than one person and compared
![Page 17: Is it True: Evaluating Medical Reviews. “The review article itself should be the product of scientific investigation in which the participants are original](https://reader036.vdocuments.mx/reader036/viewer/2022062718/56649e805503460f94b8486b/html5/thumbnails/17.jpg)
Worksheet Qs: Selecting the studies
Inclusion Criteria• Established a priori
• Minimum factors: Population/problem; intervention/comparison; outcomes; study design
• Prefer no language restriction
• Sometimes validity criteria incorporated (random, blinded, appropriate follow-up, gold standard, etc.)
Best if done independently by 2 investigators• Possibly blinded to author/journal/study results
![Page 18: Is it True: Evaluating Medical Reviews. “The review article itself should be the product of scientific investigation in which the participants are original](https://reader036.vdocuments.mx/reader036/viewer/2022062718/56649e805503460f94b8486b/html5/thumbnails/18.jpg)
Worksheet Qs:Validity of included studies
Appropriate criteria?• Assurance that criteria specific to type of article employed
(therapy, diagnosis, prognosis, etc)• If therapy: randomization, blinding, concealed allocation,
follow-up Process independent by > 2 authors?
• Surprising differences!• Why blinding may be important:
• 2 sample articles, same study methods• One finds benefit, other does not• “serious flaws” in article without benefit
![Page 19: Is it True: Evaluating Medical Reviews. “The review article itself should be the product of scientific investigation in which the participants are original](https://reader036.vdocuments.mx/reader036/viewer/2022062718/56649e805503460f94b8486b/html5/thumbnails/19.jpg)
Worksheet Qs: Validity
Were the included studies valid?• Garbage in = garbage out
• If yes, no problem
• If no, how did authors handle this?• Exclusion/inclusion criteria for quality of study
• Subanalysis with comparison of results
• Need to consider how these flaws affect results/conclusions
![Page 20: Is it True: Evaluating Medical Reviews. “The review article itself should be the product of scientific investigation in which the participants are original](https://reader036.vdocuments.mx/reader036/viewer/2022062718/56649e805503460f94b8486b/html5/thumbnails/20.jpg)
Worksheet Qs: Analyzing the data
Homogeneity vs Heterogeneity: just finding the words and an explanation most important
If NOT homogeneous?• Need qualitative explanation. Is it due to chance vs study
design, population, exposure, or outcome?
![Page 21: Is it True: Evaluating Medical Reviews. “The review article itself should be the product of scientific investigation in which the participants are original](https://reader036.vdocuments.mx/reader036/viewer/2022062718/56649e805503460f94b8486b/html5/thumbnails/21.jpg)
![Page 22: Is it True: Evaluating Medical Reviews. “The review article itself should be the product of scientific investigation in which the participants are original](https://reader036.vdocuments.mx/reader036/viewer/2022062718/56649e805503460f94b8486b/html5/thumbnails/22.jpg)
![Page 23: Is it True: Evaluating Medical Reviews. “The review article itself should be the product of scientific investigation in which the participants are original](https://reader036.vdocuments.mx/reader036/viewer/2022062718/56649e805503460f94b8486b/html5/thumbnails/23.jpg)
Worksheet Qs: Analyzing the data
Appropriateness of combining data:• “Vote” count not usually appropriate
• Important to include ‘magnitude’ of the overall effect
• Cannot be done without some common ground- outcome
Publication bias• Small, negative trials less likely to be published
• Examined by funnel plot
• Number needed to change results
![Page 24: Is it True: Evaluating Medical Reviews. “The review article itself should be the product of scientific investigation in which the participants are original](https://reader036.vdocuments.mx/reader036/viewer/2022062718/56649e805503460f94b8486b/html5/thumbnails/24.jpg)
From: Cooper & Hedges: The handbook of research synthesis. 1994
Funnel plot examples
![Page 25: Is it True: Evaluating Medical Reviews. “The review article itself should be the product of scientific investigation in which the participants are original](https://reader036.vdocuments.mx/reader036/viewer/2022062718/56649e805503460f94b8486b/html5/thumbnails/25.jpg)
From: Cooper & Hedges: The handbook of research synthesis. 1994
![Page 26: Is it True: Evaluating Medical Reviews. “The review article itself should be the product of scientific investigation in which the participants are original](https://reader036.vdocuments.mx/reader036/viewer/2022062718/56649e805503460f94b8486b/html5/thumbnails/26.jpg)
Reviews: Major Points Validity traps to avoid
• Assertions based on DOEs -- avoid perpetuating medical gossip
• Unassessed validity -- Personal experience unreliable as a basis for therapeutic interventions
Missing pieces -- **Quality of the review varies inversely with the expertness of the writer**
Failure to identify level of evidence – Look for LOE’s/ SORT
![Page 27: Is it True: Evaluating Medical Reviews. “The review article itself should be the product of scientific investigation in which the participants are original](https://reader036.vdocuments.mx/reader036/viewer/2022062718/56649e805503460f94b8486b/html5/thumbnails/27.jpg)
Reviews- Three Basic Types
![Page 28: Is it True: Evaluating Medical Reviews. “The review article itself should be the product of scientific investigation in which the participants are original](https://reader036.vdocuments.mx/reader036/viewer/2022062718/56649e805503460f94b8486b/html5/thumbnails/28.jpg)
![Page 29: Is it True: Evaluating Medical Reviews. “The review article itself should be the product of scientific investigation in which the participants are original](https://reader036.vdocuments.mx/reader036/viewer/2022062718/56649e805503460f94b8486b/html5/thumbnails/29.jpg)
Highly Controlled ResearchRandomized Controlled TrialsSystematic Reviews
Physiologic ResearchPreliminary Clinical ResearchCase reportsObservational studies
Uncontrolled Observations&
Conjecture
Effect on Patient-Oriented OutcomesSymptomsFunctioningQuality of LifeLifespan
Effect on Disease MarkersDiabetes (microalbuminuria, GFR, photocoagulation rates)Arthritis (ESR, x-rays)Peptic Ulcer (endoscopic ulcers)
Effect on Risk Factors for DiseaseImprovement in markers (blood pressure, HbA1C, cholesterol)
SORTA
Validity of Evidence
Re
leva
nce
of O
utc
om
eR
ele
vanc
e o
f Ou
tco
me
SORTB
SORTC