Diagnostic Test Characteristics: Diagnostic Test Characteristics: What does this result mean?What does this result mean?

Basic Research Lecture SeriesBasic Research Lecture Series

Adam J. Singer, MDAdam J. Singer, MD

Professor and Vice Chairman for ResearchProfessor and Vice Chairman for Research

Department of Emergency MedicineDepartment of Emergency Medicine

Stony Brook UniversityStony Brook University

Why get diagnostic tests?Why get diagnostic tests?

Rarely confirm or exclude disease with Rarely confirm or exclude disease with certaintycertainty

Results strengthen clinical estimate that Results strengthen clinical estimate that disease likely or unlikely in particular disease likely or unlikely in particular patientpatient

Test results often labeled as Test results often labeled as ““positivepositive”” ““negativenegative””, , ““highhigh”” or or ““low probabilitylow probability””– Do not guarantee magnitude by which test Do not guarantee magnitude by which test

results strengthen clinical assessmentresults strengthen clinical assessment

Traditional measures of the Traditional measures of the diagnostic value of a testdiagnostic value of a test

SensitivitySensitivity SpecificitySpecificity

– These measure a testThese measure a test’’s diagnostic s diagnostic discrimination compared to that of a criterion discrimination compared to that of a criterion standard (that has 100% sensitivity and standard (that has 100% sensitivity and specificity)specificity)

– Inherent characteristics of test unaffected by Inherent characteristics of test unaffected by disease prevalencedisease prevalence

Positive Predictive ValuePositive Predictive Value Negative Predictive ValueNegative Predictive Value

SensitivitySensitivity

Measures the proportion of those with Measures the proportion of those with disease correctly identified by testdisease correctly identified by test

If my patient has disease, what is the chance If my patient has disease, what is the chance that my test will detect it?that my test will detect it?

True Positives/all people with diseaseTrue Positives/all people with disease = TP/(TP+FN)= TP/(TP+FN)

SpecificitySpecificity

Measures the proportion of those without Measures the proportion of those without disease correctly identified as disease freedisease correctly identified as disease free

If my patient is healthy, what is the chance If my patient is healthy, what is the chance the test will be negative?the test will be negative?

True Negatives/all healthy peopleTrue Negatives/all healthy people TN/(TN+FP)TN/(TN+FP)

Positive Predictive ValuePositive Predictive Value

The proportion of those with positive test The proportion of those with positive test who have diseasewho have disease

If the test is positive, how likely is it that If the test is positive, how likely is it that my patient has the disease?my patient has the disease?

PPV=TP/all positivesPPV=TP/all positives PPV=TP/(TP+FP)PPV=TP/(TP+FP)

Negative Predictive ValueNegative Predictive Value

The proportion of those with a negative test The proportion of those with a negative test who do not have the diseasewho do not have the disease

If the test is negative, how likely is is that If the test is negative, how likely is is that my patient is healthy?my patient is healthy?

NPV=TN/all negativesNPV=TN/all negatives NPV=TN/(TN+FN)NPV=TN/(TN+FN)

Two by Two TableTwo by Two TableGold Standard Gold Standard

DiseasedDiseasedGold Standard Gold Standard Disease FreeDisease Free

Test PositiveTest Positive A=number of A=number of diseased and +diseased and +

TPTP

B=number disease B=number disease free and +free and +

FPFP

Test NegativeTest Negative C=number C=number diseased and –diseased and –

FNFN

D=number disease D=number disease free and –free and –

TNTN

A+C=number A+C=number with diseasewith disease

B+D=number B+D=number disease freedisease free

Example: Sensitivity=80%, Example: Sensitivity=80%, Specificity=90%, Prevalence=10%Specificity=90%, Prevalence=10%

80 90

20 810

DiseaseTest + -

+

-

100 900

PPP=TP/(TP+FP)PPP=80/(170)=47%NPV=TN/(TN+FN)NPV=810/830=98%

170

830

Example: Sensitivity=80%, Example: Sensitivity=80%, Specificity=90%, Prevalence=1%Specificity=90%, Prevalence=1%

8 99

2 891

DiseaseTest + -

+

-

10 990

PPP=TP/(TP+FP)PPP=8/(107)=7%NPV=TN/(TN+FN)NPV=891/893=99.8%

107

893

Problems with traditional Problems with traditional measures of testsmeasures of tests

Sensitivity and specificitySensitivity and specificity– Refer to characteristics of testRefer to characteristics of test– Do not help determine likelihood of disease in Do not help determine likelihood of disease in

any given patientany given patient PPV and NPVPPV and NPV

– Highly dependent on prevalence of disease inHighly dependent on prevalence of disease in given populationgiven population

Little interest in test quality in patients with Little interest in test quality in patients with known diseaseknown disease

BayeBaye’’s Theorems Theorem

Expresses the results of test in terms of how much Expresses the results of test in terms of how much it increases or decreases the existing prior clinical it increases or decreases the existing prior clinical probability of diseaseprobability of disease

The likelihood that a positive or negative test is a The likelihood that a positive or negative test is a true positive or negative depends on sensitivity true positive or negative depends on sensitivity and specificity of test as well as pretest probability and specificity of test as well as pretest probability that patients has the diseasethat patients has the disease

Calculates probability of event given another Calculates probability of event given another eventevent

Calculation of BayeCalculation of Baye’’s Theorems Theorem

P (D+)P(T+D+)P(D+)P(T+D+) + [1-P(D+)][1-P(T-D-)

P(D+T+)=

where:where:

P(D+T+) = probability of disease given a positive testP(D+T+) = probability of disease given a positive test

P(D+) = probability of diseaseP(D+) = probability of disease

P(T+D+) = probability of a positive test given presence of diseaseP(T+D+) = probability of a positive test given presence of disease

P(T-D-) =probability of a negative test given absence of diseaseP(T-D-) =probability of a negative test given absence of disease

Likelihood Ratios or Playing the OddsLikelihood Ratios or Playing the Odds The likelihood of disease BEFORE testing The likelihood of disease BEFORE testing

(pre-test probability) is its prevalence in that (pre-test probability) is its prevalence in that particular population (local or published data)particular population (local or published data)

The likelihood of disease AFTER knowing the The likelihood of disease AFTER knowing the test result is the post-test probabilitytest result is the post-test probability

LR measures the MAGNITUDE of change LR measures the MAGNITUDE of change from initial assessment to post test assessment from initial assessment to post test assessment of disease probabilityof disease probability

How will results of test change likelihood of How will results of test change likelihood of diseasedisease

Likelihood RatiosLikelihood Ratios

Diagnostic tests only useful if results Diagnostic tests only useful if results substantially alter pre=test probabilitysubstantially alter pre=test probability

Treatment thresholdTreatment threshold– Level of disease probability requiring no Level of disease probability requiring no

further testing and prompts treatmentfurther testing and prompts treatment Test thresholdTest threshold

– Level of disease probability that effectively Level of disease probability that effectively rules out disease requiring no further testingrules out disease requiring no further testing

Likelihood RatiosLikelihood Ratios

Measure accuracy of testMeasure accuracy of test Ratio of a given test result in patients Ratio of a given test result in patients with with

diseasedisease to probability of the same test result to probability of the same test result in patients in patients without diseasewithout disease

Indicates how much a given test result will Indicates how much a given test result will increaseincrease or or decreasedecrease probability of disease probability of disease

Calculating Likelihood Ratios: Calculating Likelihood Ratios: Sensitivity 80%, Specificity 90%Sensitivity 80%, Specificity 90%

Sensitivity1-Specificity

1-SensitivitySpecificity

+LR=

-LR=

+LR= 0.8/0.1=8

-LR= 0.2/0.9=0.2

Application of LR to clinical Application of LR to clinical decision making decision making

A useful diagnostic test has a very high or A useful diagnostic test has a very high or very low likelihood ratio very low likelihood ratio

As LR approaches 1, utility of test As LR approaches 1, utility of test decreases (probability remains the same)decreases (probability remains the same)

Use of nomogram easiest way for clinician Use of nomogram easiest way for clinician to calculate post test probabilityto calculate post test probability

Calculating post test probability Calculating post test probability from pretest probability and the LR from pretest probability and the LR Pretest probability 5%, LR = 3.7Pretest probability 5%, LR = 3.7

1.1. Convert probability to oddsConvert probability to odds

2.2. Calculate posttest odds from pretest odds Calculate posttest odds from pretest odds and LRand LR

3.3. Convert odds back to probabilityConvert odds back to probability

Probability 0.5 11-Probability 0.95 19

Pretest odds = = =

Posttest odds=Pretest odds x LR =1

19 X 3.7 = 0.19

Post test probability = Odds 0.19

1+odds 1+0.19= = 0.16

Impact of LR on post test Impact of LR on post test probabilityprobability

High LRHigh LR’’ss Low LRLow LR’’ss Effect on post-Effect on post-test probabilitytest probability

>10>10 <0.1<0.1 LargeLarge

5-15-1 0.1-0.20.1-0.2 ModerateModerate

2-52-5 0.2-0.50.2-0.5 SmallSmall

11 11 No ChangeNo Change

Effect of LREffect of LR’’s 10 and 0.1 on qualitative s 10 and 0.1 on qualitative ranges of pretest probabilityranges of pretest probability

LRLR Pretest Pretest probability (%)probability (%)

Posttest Posttest probability (%)probability (%)

1010 10-30 (low)10-30 (low) 53-80 (moderate 53-80 (moderate to high)to high)

1010 30-60 30-60 (intermediate)(intermediate)

80-95 (high)80-95 (high)

0.10.1 30-60 30-60 (intermediate)(intermediate)

3-12 (low)3-12 (low)

0.10.1 60-90 (high)60-90 (high) 12-50 (low to 12-50 (low to intermediate)intermediate)

Example 1Example 1

36 y/o female36 y/o female Sudden breathlessnessSudden breathlessness Sharp pain in sideSharp pain in side No h/o DVT/PENo h/o DVT/PE No OCP or smokingNo OCP or smoking No hemoptysisNo hemoptysis HR 95HR 95

Well’s Criteria for PEAlternative DiagnosisDVTHemoptysisRecent surgeryCancerHr > 100

Score = 0 Low Probability (9.5%, 95% CI, 7.5% to 11.3%)

Pre test probability 9.5%LR of + D-Dimer 2 (1.9-2.2)LR of – D-Dimer 0.02 (0.003-0.16)

Well’s Ann Intern Med 1998;2001;135

Post test probability = 17%

Post test probability = 0.2%

Example 2Example 2

Obese 45 y/o femaleObese 45 y/o female Breathlessness, bil leg swellingBreathlessness, bil leg swelling H/o CHF, COPDH/o CHF, COPD PE: tachypnea, bil ralesPE: tachypnea, bil rales CXR: Cardiomegally, unerpenetratedCXR: Cardiomegally, unerpenetrated DDDD

– CHFCHF– COPDCOPD– PEPE

Pre test probability of CHF 50%LR + BNP = 4.1

LR – BNP = 0.09

McCullough Acad Emerg Med 2003;10:275

Post test probability = 78%

Post test probability = 8%

Example 3Example 3

8 y/o boy with sore throat and fever8 y/o boy with sore throat and fever Erythema and anterior adenopathyErythema and anterior adenopathy Clinical likelihood of + GABHS on culture Clinical likelihood of + GABHS on culture

50% (pretest probability)50% (pretest probability) Rapid Strep ELISA +LR=20 (large effect), -Rapid Strep ELISA +LR=20 (large effect), -

LR=0.2 (moderate effect)LR=0.2 (moderate effect) Likelihood of +GABHS given + Rapid Likelihood of +GABHS given + Rapid

Strep = 97% Strep = 97% – Clinical decision: treat with antibioticsClinical decision: treat with antibiotics

Example 3 – continuationExample 3 – continuation

Likelihood of +GABHS given negative Likelihood of +GABHS given negative Rapid Strep = 20%Rapid Strep = 20%– Clinical decision: send formal throat cultureClinical decision: send formal throat culture

Example 4 – PIOPED StudyExample 4 – PIOPED Study High probability V/Q scanHigh probability V/Q scan

– Sensitivity 41%, PPV 87%, Sensitivity 41%, PPV 87%, – LR = 17 (post test odds 17 times higher than pretest LR = 17 (post test odds 17 times higher than pretest

probability)probability)– If high pretest probability: treat If high pretest probability: treat

Normal V/Q scanNormal V/Q scan– Sensitivity 2%, PPV 4%, specificity 96%, NPV 19%Sensitivity 2%, PPV 4%, specificity 96%, NPV 19%

– LR = 0.1 (post test odds 10 times lower than pretest LR = 0.1 (post test odds 10 times lower than pretest probability)probability)

– If low pretest probability: do not treat or conduct If low pretest probability: do not treat or conduct further testing further testing

Example 4 - continuationExample 4 - continuation

Intermediate probability Intermediate probability V/Q scanV/Q scan– Sensitivity 41%, PPV 30%Sensitivity 41%, PPV 30%– LR=1 (post test probability the same as pretest)LR=1 (post test probability the same as pretest)– Does not change pretest probabilityDoes not change pretest probability– Decision: useless result, conduct further testingDecision: useless result, conduct further testing

Low probability V/Q scanLow probability V/Q scan– Sensitivity 16%, PPV 14%Sensitivity 16%, PPV 14%– LR = 0.4 (likelihood of PE drops by 60%)LR = 0.4 (likelihood of PE drops by 60%)– Decision: if pretest probability high, further Decision: if pretest probability high, further

testingtesting

Example 5Example 5 BNP is a new diagnostic test for CHFBNP is a new diagnostic test for CHF Sensitivity 90%Sensitivity 90% Specificity 76%Specificity 76% Assume patient with SOB, smoker, no Assume patient with SOB, smoker, no

edemaedema– Pre-test probability 20%Pre-test probability 20%– Post-test probability if BNP elevated?Post-test probability if BNP elevated?

http://araw.mede.uic.edu/cgi-bin/testcalc.pl

Maisel et al. N Engl J Med 2002;347:161

Example 6Example 6

34 y/o male, acute flank pain34 y/o male, acute flank pain Diagnostic characteristics of hematuriaDiagnostic characteristics of hematuria

– Sensitivity 84%Sensitivity 84%– Specificity 74%Specificity 74%

Probability of kidney stoneProbability of kidney stone– No RBCNo RBC’’ss– HematuriaHematuria

http://araw.mede.uic.edu/cgi-bin/testcalc.plLuchs et al. Urology 2002;59:839

http://www.lrdatabase.com/index.php