Validity and Reliability of

Analytical Tests

Analytical Tests include both:

• Screening Tests

• Diagnostic Tests

Two Important Objectives

To distinguish between people in the population who have the diseases and

those who do not

To determine how good the test is in separating populations of people with and without the disease in question?

Epidemiological Surveillance

vs.Screening

Epidemiological Surveillance

• What is it?

• Why do it?

Epidemiological Surveillance

• Definition - ongoing & systematic collection, analysis & interpretation of data related to health, disease & conditions

• Two typesPassive Surveillance – uses available data or

reporting from health care provider or regional health officer

Active Surveillance – periodic field visits to health care facilities to identify new cases

• The present approach is the survey

Epidemiological Surveillance

• Why do it?Can help discover and control the

transmission of infectious diseasesPrevention and control programs can be

planned and implemented

Screening

• Definition - use of quick and simple testing procedures to identify and separate persons: who have a disease from those that do not

ORwho are apparently (appear to be) well, but

who may be at risk of a disease, from those who probably don’t have the disease.

Terms Related to Screening Tests

• Validity - relates to accuracy (correctness)

• Reliability - repeatability

• Yield - the # of tests that can be done in a time period

Terms Related to Screening Tests (cont’d)

• Sensitivity - ability of a test to identify those who have disease

• Specificity - ability of a test to exclude those who don’t have disease

Terms Related to Screening Tests (cont’d)

• Tests with dichotomous results – tests that give either positive or negative results

• Tests of continuous variables – tests that do not yield obvious “positive” or “negative” results, but require a cutoff level to be established as criteria for distinguishing between “positive” and “negative” groups

An important public health consideration, particularly in screening free-living populations, is:

How good is the test at identifying people with the disease and without the disease?

In other words:If we screen a population, what proportion of people who have the disease will be correctly identified?

POPULATION

Test Results With Disease Without Disease

PositiveTrue Positive

(TP)False Positive

(FP)

NegativeFalse Negative

(FN)True Negative

(TN)

Sensitivity = = X 100True positives

True positives +false negatives

True positives

All persons with the disease

= TP

TP + FN

Specificity = = X 100True negatives

True negatives+false positives

True negatives

All persons without the

disease

= TN

TN + FP

Percent false negatives = % of people with the disease who were not detected by the test

FN FN + TP X 100

Percent false positives = % of people without the disease who were incorrectly labeled by the test as having the disease

FPFP + TN X 100

In the clinical setting, a more important question is:

If the test results are positive (or negative) in a given patient, what is the probability that this patient has (or does not have) the disease?

In other words:What proportion of patients who test positive (or negative) actually have (or do not have) the disease in question?

Predictive Value

Pos. PV = X 100 = % True Positives TP + FP

Neg. PV = X 100 = % True Negatives TN + FN

Biologic Variation of Human Populations &

Diagnostic Issues

Distribution of Tuberculin Reactions

Bimodal Distribution

Easy to distinguish between exposed group and those not exposed.

Distribution of Systolic Blood Pressure

Unimodal Distribution

With continuous variables, a cutoff level must be established

to separate the hypertensive group. Could choose based on statistics, but better to base on

biologic considerations.

Effects of Choosing

Different Cutoff Levels for Diabetes

Diagnosis in Population with 50% Prevalence

Real World

Pseudo-Real World

The major issue with deciding to set a cutoff high or low

is the problem of false positives

and false negatives.

Possible Groups with Dichotomous Test

True Disease Status is Known,

as with dichotomous tests.

Grouping All Positives and All Negatives

True Disease Status is Unknown,

as with continuous variables.

Artificial Cutoff

Use of Multiple Screening Tests

Sequential (Two-stage) Testing

Simultaneous Testing

Hypothetical Two-Stage Screening

Only Pos. Test 1 are given Test 2

Hypothetical Two-Stage Screening (cont.)

TEST 2 (Glucose Tolerance Test)Sensitivity = 90%Specificity = 90%

DIABETES

+ -TEST

RESULTS +315 190 505

- 35 1710 1745

350 1900 2250

Net Sensitivity = 315/500 = 63%

Net Specificity = 7600 + 1710 = 98% 9500

Predictive Value

Prevalence & Predictive Value

Positive

As prevalence increases, positive predictive value increases.

Prevalence & Predictive Value

Note: Test has 95% sensitivity and 95% specificity

Specificity & Predictive

Value

As specificity increases,

positive predictive value increases. As sensitivity increases, positive predictive value also increases, but to a

much lesser extent.

Specificity & Predictive Value

As specificity increases, positive predictive value increases.

Results reliable but NOT valid

Results reliable and

valid

Reliability (Repeatability) of Tests