diagnostic accuracy part 1 - etusivu | · 2019-09-20 · diagnostic accuracy - part 2: predictive...
TRANSCRIPT
Diagnostic accuracy – part 1STUDY DESIGN
DEPARTMENT OF MEDICAL LABORATORY DIAGNOSTIC SUNIVERSITY HOSPITAL SVETI DUHZAGREB, CROATIA
Recommended literatureo Bossuyt PM et al. The STARD Statement for Reporting Studies of Diagnostic
Accuracy: Explanation and Elaboration. Clinical Chemistry 2015;61:1446–1452
o Simundic AM. Diagnostic Accuracy - Part 1: Basic Concepts: Sensitivity and Specificity, ROC Analysis, STARD Statement. Point of Care 2012;11:6-8
o Simundic AM. Diagnostic Accuracy - Part 2: Predictive Value and Likelihood Ratio. Point of Care 2012;11:9-11
o Raslich MA, Markert RJ, Stutes SA. Odabir i tumačenje dijagnostičkih pretraga. / Selecting and interpreting diagnostic tests. Biochemia Medica 2007;17(2):151-161.
o Habibzadeh F, Habibzadeh P, Yadollahie M. On determining the most appropriate test cut-off value: the case of tests with continuous results. Biochemia Medica2016;26(3):297-307.
basic
appliedunderstanding relationships
application in clinical practice
Study types (aim)
observational
experimental
Study types (role)
• no intervention
• low strenght of evidence
• no causal relationship • intervention
• high strenght of evidence
• causal relationship
Study types (time)
time
• study design• data collection• data analysis
prospectiveretrospective
cross-sectional
• study design• data analysis
• data collection (from the past)
• study design
• data collection (in the future)• data analysis
Diagnostic accuracy study
How accurately a diagnostic test/procedure can discriminate….
• health and disease• two conditions (viral vs. bacterial)• disease stage (acute vs. chronic)
Diagnostic processPatient historyClinical exam
Pre-test probability
Diagnostic test
Post-test probability
Does this patient have AMI?
cTn-I
AMI + AMI -
study participants
index test
reference test
disease positive
symptomatic patients
inclusion + exclusion criteria☺
indeterminate
not included
Study prototype
• cross-sectional study• consecutive patients
gate
disease negative
BiasAny trend in the collection, analysis, interpretation, publication, or review of data that can lead to conclusions that are systematically incorrect.McGraw-Hill Concise Dictionary ofModern Medicine. © 2002 by TheMcGraw-Hill Companies, Inc.
bias = random error
fx
X
BiasAny trend in the collection, analysis, interpretation, publication, or review of data that can lead to conclusions that are systematically incorrect.McGraw-Hill Concise Dictionary ofModern Medicine. © 2002 by TheMcGraw-Hill Companies, Inc.
bias = random error
fx
BMI (kg/m2)25 35
bias=10 kg/m2
Quality of studyWhy is it important?
Bad results
overestimated
biased
subjective
Clinical practice
medical errors
unnecessary costs
patient harm
Lijmer et al. JAMA, 1999.The effect of bias can be quantified
JG Lijmer, et al. Empirical Evidence of Design-Related Bias in Studies of Diagnostic Tests. JAMA. 1999;282(11):1061-66.
Empirical evidence of design-related bias in studies of diagnostic tests
Those who recruit patients for clinical trials should not be aware of treatment allocation.
blinded, N=57
unblinded, N=45
non-random, N=43
14%
>1 variable not distributed equally
among groups
26%
58%
Difference in case fatality rate between
treatment and control group
5 (8,8%)
11 (24,4%)
25 (58%)
Therapeutic RCT – acute myocardial infarction (N=145)
Chalmers TC, Celano P, Sacks HS, Smith H Jr. Bias in treatment assignment in controlled clinical trials. N Engl J Med. 1983;309(22):1358-61.
Kohn MA, et al. Understanding the direction of bias in studies of diagnostic test accuracy. Acad EmergMed. 2013;20(11):1194-206.
Possible sources of bias in diagnostic accuracy studies
Possible sources of bias in diagnostic accuracy studies
1/2
study participants
index test
reference test 1
disease positive
symptomatic patients
inclusion + exclusion criteria
Differential verification bias
gate
disease negative
reference test 2
study participants
index test
reference test
disease positive
symptomatic patients
inclusion + exclusion criteria
Partial verification bias
gate
disease negative
+
-
Possible sources of bias in diagnostic accuracy studies
2/2
study participants
index test
reference test
disease positive
symptomatic patients
inclusion + exclusion criteria
Treatment paradox
gate
disease negative
Review bias
study participants
index test
reference test
disease positive
symptomatic patients
inclusion + exclusion criteriagate
disease negative
index test
Subjects in whom reference standard was not done are excluded.
Do we really need such test?
Two gate design(case – control)
gate 1 gate 2
patients
index test
healthy controls
index test
This is just to remind you what kind of test we need…
Patient historyClinical exam
Pre-test probability
Diagnostic test
Post-test probability
Does this patient have AMI?
cTn-I
AMI + AMI -
Evidence based laboratory medicine
Case-control (two gate ) design
Example…
Staack A, Badendieck S, Schnorr D, Loening SA, Jung K. BMC Urol. 2006;6:19
BMC Urology has an Impact Factor of 1.606.
Staack A, Badendieck S, Schnorr D, Loening SA, Jung K. BMC Urol. 2006;6:19
Staack A, Badendieck S, Schnorr D, Loening SA, Jung K. BMC Urol. 2006;6:19
ne uključuju opće smjernice za pisanje znanstvenog rada
poboljšati kvalitetu istraživanja
olakšati čitanje radova
Reporting guidelines
• these guidelines refer to the:• study design,
• study methodology and reporting
• these guidelines ensure study: • comparability,
• transparency,
• reproducibility
www.equator-network.org/reporting-guidelines
STARD Statement
Published in 2003 and updated in 2015:◦ Clinical Chemistry
◦ Annals of Internal Medicine
◦ Radiology
◦ BMJ
◦ Lancet
◦ American Journal of Clinical Pathology
◦ Clinical Biochemistry
◦ CCLM
STARD Statement(Flow chart)
STARD Statement (Checklist) 1/2
STARD Statement (Checklist) 2/2
How to assess the quality of a diagnostic accuracy
study?
Whiting PF et al. Ann Intern Med. (2011)
From: QUADAS-2: A Revised Tool for the Quality Assessment of Diagnostic Accuracy Studies
Ann Intern Med. 2011;155(8):529-536. doi:10.7326/0003-4819-155-8-201110180-00009
Risk of Bias and Applicability Judgments in QUADAS-2
Copyright © American College of Physicians. All rights reserved.
From: QUADAS-2: A Revised Tool for the Quality Assessment of Diagnostic Accuracy Studies
Ann Intern Med. 2011;155(8):529-536. doi:10.7326/0003-4819-155-8-201110180-00009
Risk of Bias and Applicability Judgments in QUADAS-2
Copyright © American College of Physicians. All rights reserved.
From: QUADAS-2: A Revised Tool for the Quality Assessment of Diagnostic Accuracy Studies
Ann Intern Med. 2011;155(8):529-536. doi:10.7326/0003-4819-155-8-201110180-00009
Copyright © American College of Physicians. All rights reserved.
From: QUADAS-2: A Revised Tool for the Quality Assessment of Diagnostic Accuracy Studies
Ann Intern Med. 2011;155(8):529-536. doi:10.7326/0003-4819-155-8-201110180-00009
Risk of Bias and Applicability Judgments in QUADAS-2
This tool allows more transparent rating of bias and applicability of primary diagnostic accuracy studies.
Read carefully!Be skeptical!
Question everything!