sensitivity, specificity and likelihood ratios
TRANSCRIPT
Sensitivity, Specificity and Likelihood Ratios
K.S. Chew
Faculty of Medicine and Health Sciences
Universiti Malaysia Sarawak
Email: [email protected]/25/2016 1
Sensitivity
• Proportion of patients with disease who are tested positive with a test
• A 100% sensitive test will not have any false negative results (although it may have a high rate of false positive results)
• Therefore, a negative result of a highly sensitive test means it is likely to be a true negative (it rules out the disease)
“SN-OUT”
1/25/2016 2
Specificity
• Proportion of patients without disease who are tested negative with a test
• A 100% specific test will not have false positive results (although it may have high rate of false negative results)
• Therefore, a positive result of a highly specific test means it is likely to be true positive (it rules in the disease)
“SP-IN”
1/25/2016 3
Sensitivity
Disease +ve Disease -ve
Test +ve a (TP) b (FP)
Test –ve c (FN) d (TN)
1/25/2016 4
TP = True positiveFN = False negativeFP = False positiveTN = True negative
Sensitivity = (a)/(a+c)
Positive Predictive Value
Disease +ve Disease -ve
Test +ve a (TP) b (FP)
Test –ve c (FN) d (TN)
1/25/2016 5
TP = True positiveFN = False negativeFP = False positiveTN = True negative
Positive PV = (a)/(a+b)
Specificity
Disease +ve Disease -ve
Test +ve a (TP) b (FP)
Test –ve c (FN) d (TN)
1/25/2016 6
TP = True positiveFN = False negativeFP = False positiveTN = True negative
Specificity = (d)/(b+d)
Negative Predictive Value
Disease +ve Disease -ve
Test +ve a (TP) b (FP)
Test –ve c (FN) d (TN)
1/25/2016 7
TP = True positiveFN = False negativeFP = False positiveTN = True negative
Negative PV = (d)/(c+d)
Sensitivity and Specificity
Image taken from: http://library.med.utah.edu/WebPath/TUTORIAL/BIOSTATS/BIOSTATS.html
1/25/2016 8
To increase sensitivity, shift to the left (purple line)
But by shifting to the left, it increases proportion of false positive, which means reduced specificity
Sensitivity and Specificity
Image taken from: http://library.med.utah.edu/WebPath/TUTORIAL/BIOSTATS/BIOSTATS.html
1/25/2016 9
To increase specificity, shift to the right (purple line)
But by shifting to the right, it increases proportion of false negative, which means reduced sensitivity
Example: Troponin assays
• First generation assay: cut-off 0.5 microgm/l
• 3rd generation assay: 0.05 – 0.10 microgm/l
• High-sensitive troponin (hsTn): 0.0030 microgm/l
• High-sensitive Roche Elecsys: 0.0014 microgm/l
• The diagnostic sensitivity of hsTn assays (ability to rule-out MI) are of the order of 90–95% when tested at the point of admission (still misses 5 - 10% of cases)
1/25/2016 10
Ref: Gamble et al, Br J Cardiol. 2013;20(4)
Causes of elevated troponins
• Myocardial ischemic conditions
• ACS
• Myocardial ischemic conditions other than ACS
• Systemic conditions
• Myocardial injury without ischemic insults
• Systemic conditions – renal failure, sepsis
• Specific identifiable precipitants – cardiac contusion, burns >30% BSA
1/25/2016 11
Sensitivity and Specificity
• A trade-off
• When sensitivity increases, specificity decreases
• When specificity increases, sensitivity decreases
1/25/2016 12
Image taken from: http://groups.csail.mit.edu/cb/struct2net/webserver/about.html
Receiver Operating Characteristics Curve
• When sensitivity increases, specificity decreases
• Therefore, when sensitivity increases, (1 – specificity) increases
• AUC – represents how good a test is
1/25/2016 13
Area under curve (AUC)
• Specificity is a measure of true negative; therefore (1 –specificity) is a measure of false positive
• While AUC of 1 represents a perfect test; AUC of 0.5 is a worthless test (a.k.a for every one true positive, there is an equal chance of getting one false positive)
• Interpretation:• 0.90 -1 = excellent • 0.80 - 0.90 = good• 0.70 - 0.80 = fair • 0.60 - 0.70 = poor• 0.50 - 0.60 = fail
1/25/2016 14
Example:
1/25/2016 15
Reichlin T, Hochholzer W, Bassetti S,
Steuer S, Stelzig C, Hartwiger S, et al.
Early Diagnosis of Myocardial Infarction
with Sensitive Cardiac Troponin Assays. N
Eng J Med 2009;361(9):858-67.
Methods
• Multi-center, n = 718, symptoms suggestive of MI
• Diagnostic accuracy of different troponin assays• Abbott–Architect Troponin I
• Roche High-Sensitive Troponin T
• Roche Troponin I, and Siemens Troponin I Ultra)
• vs standard assay (Roche Troponin T).
• Final diagnosis determined by 2 independent cardiologists: reviewing clinical history, physical findings, labs, ECG, echo, angio findings, etc
1/25/2016 16
Reichlin et al 2009
Results
1/25/2016 17
Reichlin et al 2009
Results
• AUC significantly higher for:
• Abbott–Architect Troponin I, 0.96 (95% CI 0.94 to 0.98)
• Roche High-Sensitive Troponin T, 0.96 (95% CI 0.94 to 0.98)
• Roche Troponin I, 0.95 (95% CI, 0.92 to 0.97)
• Siemens Troponin I Ultra 0.96 (95% CI, 0.94 to 0.98)
• standard assay, 0.90 (95% CI, 0.86 to 0.94)
1/25/2016 18
Reichlin et al 2009
Results
1/25/2016 19
Reichlin et al 2009
Likelihood Ratios
• Positive likelihood ratio refers to the likelihood of a patient with the disease to be tested as positive compared to a patient without the disease
• Negative likelihood ratio refers to the likelihood of patient with the disease to be tested negative as compared to a patient without the disease
• Every test has both LR (+) and LR (-)
1/25/2016 20
Likelihood Ratios
• LR are more helpful than sensitivity and specificity because sensitivity and specificity are derived from population where we already know whether they have or do not have the disease
• Whereas LRs tell us prospectively how a positive or negative test results affect the likelihood of patient to have a disease when we do not know whether they have it or not
• Likelihood ratios have factored in the sensitivity, specificity of the test (the TP, TN, FP, FN)
1/25/2016 21
Likelihood Ratios
• Positive likelihood ratio refers to the likelihood of a patient with the disease to be tested as positive compared to a patient without the disease
• LR (+)
• = (True positive)/(False positive)
• = (sensitivity)/(1-specificity)
• The higher LR (+), the better the test to RULE IN the disease
1/25/2016 22
Likelihood ratios
• Negative likelihood ratio refers to the likelihood of patient with the disease to be tested negative as compared to a patient without the disease
• LR (-) = (False Negative)/(True Negative)
• = (1 – sensitivity)/(specificity)
• The smaller the LR (-), the better the test TO RULE OUT the disease
1/25/2016 23
Usefulness of LRs
• To choose a diagnostic test
• E.g. which test would be the best to RULE IN a disease?
• Which test would be the best to RULE out a disease?
• To calculate a post-test probability (use Fagan Normogram)
1/25/2016 24
Example:
1/25/2016 25Collins et al, J Cardiac Failure 2015:21(1)
Fagan Nomogram
1/25/2016 26
Example:
• Why PERC score should only be used when Well’s criteria is in the low risk category?
• LR (-) of PERC is 0.17 (95% CI: 0.11 – 0.25)
• Ref: Carpenter CR, et al (2009). Differentiating low-risk and no-risk PE patients: the PERC score. J Emerg Med, 36 (3), 317-22
1/25/2016 27
PERC Score
• PERC score is a rule-out criteria for pulmonary embolism where if none of the 8 PERC criteria are present in a patient, PE can be ruled out clinically
• B = Blood in sputum (hemoptysis)
• R = Room air O2 Sat>95%
• E = estrogen or homonal use
• A = Age >50 years
• T = Thrombotic events (DVT, PE) or its possibility
• H = HR >/= 100/min
• S = surgery past 4 weeksl
1/25/2016 28
Determine your point of equipoise?
• Point of equipoise is the balance point when the risk-benefit of investigating further for PE vs risk-benefit of NOT investigating further for PE.
• Kline et al (2004) – point of equipoise for PE is 1.8%.
1/25/2016 29
Expolarating a LR (-) of 0.17
and a Post-test probability of 1.8%
Therefore, the pre-test probability
must be below 10%
1/25/2016 30
Determine that your post-test probability is no more than 1.8% (point of equipoise)
LR (-) for PERC
Wells criteria
Only in the low risk category
of Wells Criteria where the
probability of PE is below
10%. Therefore, PERC score
should be used only when the
Wells score is in the low risk
category
1/25/2016 31
Recommended Video Tutorials
• 6 short video series on sensitivity and specificity:
• https://www.youtube.com/watch?v=U4_3fditnWg&list=PL41ckbAGB5S2PavLIXUETzAmi5reIod23
• On likelihood ratios:
• https://www.youtube.com/watch?v=TzPvCSFZUSQ
1/25/2016 32