systematic review module 11: grading strength of evidence interactive quiz kathleen n. lohr, phd...
TRANSCRIPT
Systematic Review Module 11:Systematic Review Module 11:Grading Strength of EvidenceGrading Strength of Evidence
Interactive QuizInteractive Quiz
Kathleen N. Lohr, PhDKathleen N. Lohr, PhDDistinguished Fellow Distinguished Fellow
RTI InternationalRTI International
Interactive Case Quiz: Interactive Case Quiz: InstructionsInstructions
Open this presentation as a slideshow. This will activate the hyperlinks.
When you come to a decision slide, choose the red box corresponding to the correct choice. If you are correct, you will be directed forward in the case. If you are incorrect, you will be directed back to the decision slide to choose again.– Click on forward hyperlinks (Red Boxes) to follow
through case based on your responses – Click on home hyperlink (Blue House Icon) to go
back to the last correct step in the series
Case Study Quiz: Case Study Quiz: Grading Strength of Evidence (SOE)Grading Strength of Evidence (SOE)
You are near the end of the process of developing a CER of the benefits and harms of a therapeutic intervention. This step entails grading the strength of one or more bodies of evidence that pertain to the key questions in your CER.
Participation Quiz: Question 1Participation Quiz: Question 1
Grading SOE is the same as rating the quality of studies.
They are the same thing.
You can grade SOE at the same time as you rate the quality of individual studies.
You can grade SOE only after you have rated the quality of individual studies.
They Are the Same ThingThey Are the Same Thing
This is incorrect. Individual studies are rated for their
quality (i.e., internal validity or risk of bias).
Grading pertains to entire bodies of evidence about important key questions, outcomes, or comparisons, and the quality of individual studies is just one element in grading SOE.
[Click on Blue Box to Go Back]
You Can Grade SOE and Rate You Can Grade SOE and Rate Quality at the Same TimeQuality at the Same Time
This is incorrect. Grading pertains to entire bodies of
evidence about important key questions, outcomes, or comparisons.
Because domains other than the quality of individual studies must be used in grading SOE, evaluating study quality and SOE together is insufficient.
[Click on Blue Box to Go Back]
You Grade SOE Only After You Grade SOE Only After Rating Study QualityRating Study Quality
Yes, you are correct. Grading SOE is a late step in the review
process. It requires judgments about several key domains, only one of which is quality of individual studies.
[SELECT RED BOX]
Participation Quiz: Question 2Participation Quiz: Question 2
Grading SOE specifies required domains. What are they?
Risk of bias, consistency, precision, and directness
Risk of bias, consistency, precision, directness, and publication bias
Risk of bias, precision, strength of association, and directness
Four Required Domains Four Required Domains
Yes, You are correct! The four required domains are
– Risk of bias: quality ratings for individual studies
– Consistency: degree of similarity in the effect sizes of different studies within an evidence base
– Precision: degree of certainty for estimate of effect with respect to a specific outcome
– Directness: whether evidence reflects a single, direct link between the interventions of interest and the ultimate health outcome under consideration, or for comparisons, whether the evidence base has head-to-head studies [SELECT RED BOX]
Four Required Domains and Four Required Domains and Publication BiasPublication Bias
This is incorrect. The four required domains are
– Risk of bias
– Consistency
– Precision
– Directness
Publication bias is an additional domain that can and should be used only when relevant.
[Click on Blue Box to Go Back]
Three Required Domains and Three Required Domains and Strength of AssociationStrength of Association
This is incorrect. The four required domains are
– Risk of bias
– Consistency, which is the missing required domain in this answer
– Precision
– Directness
Strength of association is an additional domain referring to magnitude of effect that can be used when relevant.
[Click on Blue Box to Go Back]
Participation Quiz: Question 3Participation Quiz: Question 3
Strength of evidence is graded according to the following levels or grades:
High, moderate, low, very low
Very high, high, moderate, low,
very low
High, moderate, low, insufficient
High, Moderate, Low, Very LowHigh, Moderate, Low, Very Low
This is incorrect. Very low is not an option. Use four SOE grades:
– High confidence that the evidence reflects the true effect
– Moderate confidence that the evidence reflects the true effect
– Low confidence that the evidence reflects the true effect
– Insufficient: Evidence either is unavailable or does not permit a conclusion
[Click on Blue Box to Go Back]
High, Moderate Low, High, Moderate Low, InsufficientInsufficient
Yes, this is correct. Strength of evidence can have three
“strength” grades relating to the confidence one has that evidence reflects the true effect and that future research might confirm or overturn the results.
Insufficient implies either that one has no relevant evidence or that the evidence is so mixed and inadequate that one cannot definitely label it as high, moderate, or low.
[SELECT RED BOX]
Very High, High, Moderate, Very High, High, Moderate, Low, Very LowLow, Very Low
This is incorrect. Very high and very low are not used.
The four correct options for scores are – High
– Moderate
– Low
– Insufficient
[Click on Blue Box to Go Back]
Participation Quiz: Question 4Participation Quiz: Question 4
In combining scores for domains into an overall grade for strength of evidence, you can use the following approaches:
Only the GRADE algorithm
Only your own weighting system
Only your own qualitative approach
Any of the above
Only One Approach is Only One Approach is AllowedAllowed
This is incorrect. Any of the three approaches can be
used. The critical element in selecting any
option is careful documentation of the methods.
[Click on Blue Box to Go Back]
GRADE, Own Weighting, or GRADE, Own Weighting, or Own Qualitative ApproachOwn Qualitative Approach
Yes, this is correct. Any of the three approaches can be
used. The critical element in selecting any
option is careful documentation of the methods.
[SELECT RED BOX]
Participation Quiz: Question 5Participation Quiz: Question 5
In scoring required and additional domains and in arriving at an overall grade for strength of evidence, you
Use only one senior reviewer to do these tasks and report this score.
Use two or more reviewers, resolve differences by consensus or adjudication by a third party, and report all scores.
Use two or more reviewers, resolve differences by consensus or adjudication by a third party, and report a consensus score.
Only One ReviewerOnly One Reviewer
This is incorrect. You should use at least two reviewers
with appropriate clinical and methodological expertise.
[Click on Blue Box to Go Back]
Two or More Reviewers and Two or More Reviewers and Report All ScoreReport All Score
This is incorrect. You should use at least two reviewers
with appropriate clinical and methodological expertise.
But Differences should be mediated or
adjudicated by a third party. You should report only the consensus
grade. [Click on Blue Box to Go Back]
Two or More Reviewers and Two or More Reviewers and Consensus ScoreConsensus Score
Yes, this is correct. You should
– Use at least two reviewers with appropriate clinical and methodological expertise and
– Mediate or adjudicate differences by a third party
But You should report only the consensus
grade.