Usefulness of Computerized Methodfor Lung Nodule Detection in DigitalChest Radiographs Using Temporal

Subtraction Images

Takatoshi Aoki, MD, Nobuhiro Oda, PhD, Yoshiko Yamashita, MD,

Keiji Yamamoto, PhD, Yukunori Korogi, MD

Ac

FrEnKiRa(NMNto

ªdo

10

Rationale andObjectives: The aim of this studywas to evaluate the usefulness of a novel computerizedmethod for lung nodule detectionon digital chest radiographs using temporal subtraction images.

Materials and Methods: To significantly reduce the number of false-positive results while maintaining high sensitivity, temporal subtrac-

tion images, which can enhance interval changes on sequential chest radiographs, were used. Fifty-one caseswith lung nodules <3 cmand51 cases without lung nodules were selected for an observer performance test. Twelve radiologists participated in this observer perfor-

mance test. The radiologists’ performance was evaluated using receiver-operating characteristic analysis, on a continuous rating scale.

To estimate the numbers of cases affected beneficially and those affected detrimentally using this computerized method, the computer

output was assumed to have an effect on an observer’s diagnosis when there was a difference in rating score of $30% between the firstand second ratings.

Results: The average area under the curve for all radiologists increased significantly from 0.849 to 0.950 with the computerized method

(P < .001). The mean number of cases affected beneficially was significantly higher than that of cases affected detrimentally (8.92 vs 1.25,P < .001).

Conclusions: The novel computerized method using temporal subtraction images would be useful in detecting lung nodules on digital

chest radiographs.

Key Words: Lung nodule; lung cancer; computer-aided-diagnosis; chest radiograph.

ªAUR, 2011

Although chest radiography is the most prevalent

screening procedure for detecting lung lesions, because

it is economical and easy to use, radiologists are some-

times unaware of lung nodules that are perceptible on chest

radiographs in retrospect (1,2), and it is even more difficult to

detect small lung nodules (3). Computer-aided diagnosis

(CAD) is now considered an approach that might improve

the efficacy of radiologic image interpretation. For the detec-

tion of subtle lung nodules, however, CAD of a single chest

radiograph suffers from interference by anatomic noise, leading

to low specificity for an acceptable level of sensitivity (4). The

ad Radiol 2011; 18:1000–1005

om the Department of Radiology, University of Occupational andvironmental Health, School of Medicine, Iseigaoka 1-1, Yahatanishi-ku,takyushu 807-8555, Japan (T.A., Y.Y., Y.K.); the Department ofdiological Technology, Kyoto College of Medical Science, Kyoto, Japan.O.); and the Information Network System Department, Kansai Division,itsubishi Space Software Company, Ltd, Hyogo, Japan (K.Y.). Receivedovember 11, 2010; accepted March 22, 2011. Address correspondence: T.A. e-mail: a-taka@med.uoeh-u.ac.jp

AUR, 2011i:10.1016/j.acra.2011.04.008

00

temporal subtraction technique is a CAD method in which

a previous chest radiograph is subtracted from a current radio-

graph so that interval changes are enhanced. Several studies

have found that this technique improves the diagnostic accu-

racy of newly formed lung nodule (5–7). However,

misregistration artifacts of this system, due to mismatching of

normal anatomic structures in current and previous images,

cause false-positive findings and sometimes lead tomisinterpre-

tation. In addition, both previous and current images are

required for this CAD system, which may limit its clinical use.

We are developing a novel computerized method to assist

radiologists in the detection of lung nodules by using temporal

subtraction images. Our purpose in this study was to evaluate

the usefulness of this computerized method on radiologists’

performance for lung nodule detection.

MATERIALS AND METHODS

Scanning Protocol

All chest radiographs were exposed at 100 kV with a 10:1 grid

and were obtained using a computed radiographic system

Academic Radiology, Vol 18, No 8, August 2011 CAD FOR LUNG NODULE IN DIGITAL RADIOGRAPHS

(FCR; Fuji Photo Film, Tokyo, Japan). The imaging plate

(model ST-V; Fuji Photo Film) was 35 � 35 cm (matrix

size, 1760 � 1760; gray level, 10 bits; pixel size, 0.2 mm).

Hospital institutional review board approval was obtained,

and informed consent was not required by the institutional

review board.

Figure 1. Overall scheme of computerized method for lung noduledetection.

Computerized Scheme

A diagram of our computerized scheme for lung nodule

detection is shown in Figure 1 (8). To significantly reduce

the number of false-positive findings while maintaining high

sensitivity, we used temporal subtraction images produced

using a commercially available temporal subtraction system

(Truedia/XR; Mitsubishi Space Software, Amagasaki, Japan)

(9,10). In this system, a registration algorithm previously

described (9) was used. First, extraction of the chest region

was applied to ribcage edge detection on chest radiographs

(11). The CAD system was applied to correct the background

density trend on chest subtraction image (12). Then, nodule

candidates were extracted using multilevel thresholding with

various pixel values, which were determined from the area

of the histogram of pixel values on the temporal subtraction

image (13). Twenty-one image features for each nodule candi-

dates were obtained from both subtraction images and current

chest radiographs. The image features on nodules included

three morphologic features (area, circularity, and irregularity)

obtained from the subtraction image and nine gray-level

features (mean, maximum, minimum, variance, contrast,

skewness, kurtosis, energy, and entropy) obtained from histo-

graphic analysis of pixel values within the nodule on both

subtraction and current images. A rule-based approach and

aMahalanobis distance discriminant analysis with 21morpho-

logic and gray-level features were applied to remove false-

positive candidates corresponding to non-nodules (14,15).

Finally, the computer output was superimposed on the

detected nodules on current chest radiographs using markers

(plus signs; Fig 2).

Performance of the CAD System

This CAD system was first preliminarily applied to a database

consisting of 88 radiographswith 90 lung nodules. The nodule

presence and absence information is provided by the database.

Two board-certified radiologists independently classified the

radiographs with lung nodules according to the subtlety of

each lung nodule using a five-point scale: 1 = extremely

subtle, 2 = very subtle, 3 = subtle, 4 = relatively obvious,

and 5 = obvious. The evaluationswere based on both the orig-

inal image and the CAD output. The subtlety of the nodules

was calibrated beforehand using a large number of training

cases included in a digital image database for chest radiographs,

which is publicly available (16). Among the 90 lung nodules,

there were 21 (23%) with subtlety scores of 1, 31 (35%) with

subtlety scores of 2, 23 (26%) with subtlety scores of 3, 13

(14%) with subtlety scores of 4, and two (2%) with subtlety

scores of 5. Using this database with a high degree of subtlety,

we achieved detection sensitivity of 80%, with an average of

2.6 false-positive detections per image.

Observer Performance Study

We selected 51 cases with lung nodules <3 cm (17 lung carci-

nomas, 17 metastatic lung tumors, and 17 benign nodules) and

51 cases without lung nodules on the basis of computed tomo-

graphic confirmation for an observer performance test of lung

nodule detection. Calcified lung nodules were excluded in

this study. The subtlety ratings of the nodules were classified

by two board-certified radiologists in the same way as for

the classification of the database. There were 15 (29%) with

subtlety scores of 1, 26 (51%) with subtlety scores of 2, five

(10%) with subtlety scores of 3, five (10%) with subtlety scores

of 4, and none with subtlety scores of 5.

Assessment of each radiologist’s diagnostic accuracy was

determined using receiver-operating characteristic analysis

with a continuous rating scale. Twelve radiologists, including

four attending radiologists with$10 years of experience, four

radiology fellows with 4 to 6 years of experience, and four

radiology residents with <2 years of experience, participated

in this observer performance test. The sequential test method

was used in the observer performance study (17). Confidence

level regarding the presence or absence of lung nodules was

marked above both ends of the 7-cm line using a black ball-

point pen on the line for the first rating (without CAD).

Only current radiographs were displayed on the 1536 �2048 line, 20.8-inch (diagonal), liquid crystal display monitor

(Dome C3; Planar Systems, Inc, Beaverton, OR). Neither

previous radiographs nor subtraction images were displayed.

Then, the current image with the computer output was dis-

played on the liquid crystal display monitor. The radiologists

could freely remove the markers indicating nodule candidates

or change the sizes of the markers during the observer perfor-

mance test. Observers were allowed to change their first

ratings using a red ballpoint pen on a same line, if the second

ratings were different from the first ones (with CAD). Before

the test, observers were shown 20 training cases (10 cases with

1001

Figure 2. Lung carcinoma in a 68-year-old

man. (a) Current chest image using for input

image. (b) Previous chest image. (c) Temporal

subtraction image using for input image.(d) Computer-aided diagnosis system output

image. Two suspected areas indicated by small

plus signs. One area contains an actual nodule(arrow), and the other contains a false-positive

finding (arrowhead).

AOKI ET AL Academic Radiology, Vol 18, No 8, August 2011

normal findings and 10 nodule cases), so that they could learn

the rating method and how to take into account the computer

output in their decisions. The performance of the CAD

system and clinical parameters (patients’ ages and sex) were

not indicated. Radiographs were presented randomly, and

the reading time was not limited.

To estimate the numbers of cases affected beneficially and

those affected detrimentally by this computerized method,

we assumed that the candidate nodules of the CAD system

had an effect on an observer’s diagnosis when there was

a difference in rating score of $30% between the first and

second ratings.

Statistical Analysis

Areas under the receiver-operating characteristic curve (Az)

were computed using the computer program LABROC5,

provided by Metz et al (18). The statistical significance of

the difference in Az values without and with CAD output

images was estimated using the Dorfman-Berbaum-Metz

method (19). Comparisons of the improvement in Az values

among three observer categories were performed using one-

way analysis of variance. If a significant difference was found,

Bonferroni’s correction was used for post hoc analysis. Using

Bonferroni’s correction, we set the significance level at

1002

P < .0033 to adjust for multiple comparisons. The difference

between the average number of cases affected beneficially and

those affected detrimentally by CAD output images was

analyzed using a two-tailed paired Student’s t test. All statistical

analyses were performed using StatView version 5.0 (SAS

Institute Inc, Cary, NC).

RESULTS

The Az values without and with the CAD for each observer

are summarized in Table 1. All of the radiologists achieved

improved diagnostic performance when the computer output

was available. The average Az value for all radiologists

increased significantly from 0.849 to 0.950 with the comput-

erized method (P < .001; Fig 3). All three observer categories

showed significant improvements in diagnostic accuracy with

the CAD system (P < .001). The difference in the level of

improvement in diagnostic accuracy with the CAD system

was significant among the three observer categories (P <

.001). After Bonferroni’s correction was applied, the improve-

ments in the average Az values were significantly greater for

the radiology residents than for the attending radiologists

(P < .001).

The numbers of cases with clinically relevant changes

in confidence levels for each observer are shown in

TABLE 1. Comparison of AUC Values without and with CAD

Observer

AUC

PWithout CAD With CAD

Residents

A 0.814 0.961

B 0.826 0.959

C 0.768 0.927

D 0.759 0.874

Average 0.792 0.930 <.001

Fellows

E 0.853 0.951

F 0.895 0.952

G 0.799 0.937

H 0.812 0.934

Average 0.840 0.944 <.001

Attending radiologists

I 0.931 0.982

J 0.903 0.970

K 0.933 0.972

L 0.900 0.979

Average 0.917 0.976 <.001

Average total 0.849 0.950 <.001

AUC, area under the curve; CAD, computer-aided diagnosis.

Figure 3. Comparison of average receiver-operating characteristic

curves of all radiologists for lung nodule detection without and withthe computer-aided diagnosis (CAD) system. The average area

under the curve (Az) for all radiologists increased significantly from

0.849 to 0.950 with the CAD system (P < .001, Dorfman-Berbaum-

Metz method [19]).

Figure 4. Bar graph showing the number of cases affected by the

computer output in confidence levels with regard to lung nodulecases. The average number of cases affected beneficially (4.0) was

significantly larger than the number affected detrimentally (0.58)

(P < .001, two-tailed paired t test).

Figure 5. Bar graph showing the number of cases affected by thecomputer output in confidence levels with regard to healthy cases.

The average number of cases affected beneficially (4.9) was signifi-

cantly larger than the number affected detrimentally (0.7) (P = .002,

two-tailed paired t test).

Academic Radiology, Vol 18, No 8, August 2011 CAD FOR LUNG NODULE IN DIGITAL RADIOGRAPHS

Figures 4 and 5. Among the 51 lung nodule cases, the average

number of cases affected beneficially (4.0) was significantly

larger than the number of cases affected detrimentally (0.6)

(P < .001; Fig 4). Among the 51 healthy cases, the average

number of cases affected beneficially (4.9) was significantly

larger than the number of cases affected detrimentally (0.7)

(P = .002; Fig 5).

DISCUSSION

Even though computed tomographic techniques continue to

develop, chest radiography remains the first and the most

common examination for the detection of lung nodules.

However, it has been shown that detecting small lung lesions

on chest radiographs is a difficult task for radiologists (1,2).

Currently, CAD technology has a complementary role in

clinical practice as a second opinion. Given the increasingly

digital nature of chest radiography, CAD will most certainly

be an integral part of clinical practice. Freedman et al (20)

reported positive findings from a clinical trial for an indepen-

dent validation test with 80 lung cancer cases with nodules 7

to 30 mm in diameter and 160 cases without nodules using of

a commercial CAD system, achieving 66% nodule detection

sensitivity with an average of 5.3 false-positive findings per

image. However, the current standard of CAD’s performing

a supporting role for lung nodule detection is rather limited.

CAD of a single chest radiograph is impeded by normal

anatomic structures, resulting in low specificity for a satisfac-

tory level of sensitivity. Consequently, CAD is expected to

1003

AOKI ET AL Academic Radiology, Vol 18, No 8, August 2011

continue to evolve to meet the increasing new challenges of

chest radiography.

The emergence of digital chest radiography in the 1980s led

to new techniques to improve the detection of subtle pulmo-

nary lesions. The temporal subtraction technique, which

selectively highlights areas of interval change by subtracting

the patient’s previous radiograph from the current one, is

one notable approach. Generally, areas that stand out in the

uniform gray background indicate interval changes. The

fact that temporal subtraction helps radiologists discern subtle

lung abnormalities, including nodules, is supported by several

studies (5–7). At the same time, the image quality of the

difference image strongly depends on the two-dimensional

registration of the two radiographs. Misregistration artifacts

are caused by mismatching normal anatomic structures in

current and previous images and are due mostly to the ribs,

diaphragm, clavicle, and blood vessels. The temporal subtrac-

tion system cannot indicate nodule candidate directly, so that

radiologists must decide whether the standout area on the

image is a true-positive finding or artifact. These misregistra-

tion artifacts cause false-positive findings and sometimes lead

to misinterpretation. Thus, an automated lung nodule detec-

tion method using temporal subtraction is expected to help

radiologists.

Because of the use of this CAD system, significant improve-

ments in diagnostic accuracy for pulmonary nodule detection

were experienced by all 12 radiologists in the observer perfor-

mance study. Although all observer categories benefited from

this CAD system, those with less experience tended to benefit

more. This evidencemay indicate that the radiologists’ perfor-

mance in the detection of lung nodule depended on their

experience. Radiologists need some experience in nodule

detection based on the temporal subtraction system, but the

use of CAD may reduce the impact of traditional experience.

For nodule and healthy cases, we analyzed the influence of

CAD output images separately. The CAD output affected

observer confidence in terms of both beneficial and detri-

mental effects in both cases. The average number of cases

affected beneficially by computer output was significantly

larger than the number of cases affected detrimentally, even

in healthy cases. The use of this CAD system can decrease

false-positive findings and may lead to avoidance of unneces-

sary further examinations, such as chest computed

tomography.

Temporal subtraction may be most practical in follow-up

examinations of patients with a high risk for lung tumors,

such as patients with interstitial pneumonia, pulmonary

emphysema, and extrapulmonary malignant neoplasm. Chest

radiography is most commonly used for follow-up and is very

important for lung tumor detection in these patients.

Although considerable attention has recently been given to

the detection of early-stage lung cancer through low-dose

computed tomographic screening of individuals at high risk

(21,22), chest radiography is still the preferred imaging

modality used for lung nodule detection. Also, this CAD

system may be useful in repeated lung cancer screening

1004

using digital chest radiographs and in assisting physicians

who do not have easy access to computed tomographic

scanners in their daily practice.

Our study had several limitations. First, we used our own

image database for chest radiographs with lung nodules.

Although we achieved detection sensitivity of 80%, with an

average of 2.6 false-positive detections per image using the

database containing many subtle nodules, our results cannot

be compared directly to those obtained in other previous

studies, because of the differences in the database. Further

studies using common CAD testing databases may be needed

to validate the performance of our CAD system. Second, time

spent reviewing interpretations without and with this CAD

system was not recorded for comparison, even though this

study showed that the new CAD system, using temporal

subtraction images, significantly improved diagnostic accu-

racy for lung nodule detection. Radiologists’ performance

in daily practice is affected not only by diagnostic accuracy

but also by interpretation time. Comparison of reviewing

time in the interpretation of radiographs without and with

this CAD systemmay also be important to evaluate the useful-

ness of this system on radiologists’ performance for lung

nodule detection. Third, this study was retrospective. Further

prospective study of a greater number of cases is likely neces-

sary to confirm the clinical usefulness of this computerized

method.

In conclusion, although the new computerized method

using temporal subtraction images has its limitations with

regard to clinical use, it can assist in subtle lung nodule detec-

tion on digital chest radiographs.

REFERENCES

1. Austin JH, Romney BM, Goldmith LS. Missed bronchogenic carcinoma:

radiographic findings in 27 patients with a potentially resectable lesion

evident in retrospect. Radiology 1992; 182:115–122.

2. Muhm JR, Miller WE, Fontana RS, et al. Lung cancer detected during

a screening program using 4-month chest radiographs. Radiology 1983;

148:609–615.

3. Heelan RT, Flehinger BJ, Melamed MR, et al. Non-small-cell lung cancer:

results of the New York Screening Program. Radiology 1984; 151:

289–293.

4. Samei E, Flynn MJ, Eyler WR. Detection of subtle lung nodules: relative

influence of quantum and anatomical noise on chest radiographs. Radi-

ology 1999; 213:727–734.

5. Uozumi T, Nakamura K, Watanabe H, et al. ROC analysis of detection of

metastatic pulmonary nodules on digital chest radiographs with temporal

subtraction. Acad Radiol 2001; 8:871–878.

6. Johkoh T, Kozuka T, Tomiyama N, et al. Temporal subtraction for detec-

tion of solitary pulmonary nodules on chest radiographs: evaluation of

a commercially available computer-aided diagnosis system. Radiology

2002; 223:806–811.

7. Kakeda S, Nakamura K, Kamada K, et al. Improved detection of lung

nodules by using a temporal subtraction technique. Radiology 2002;

224:145–151.

8. Oda N, Kido S, Shono H, et al. Development of computerized

system for detection of pulmonary nodules on digital chest radio-

graphs using subtraction images. Jpn J IEICE Trans 2004; 87D:

208–218.

9. Kano A, Doi K, MacMahon H, et al. Digital image subtraction of temporally

sequential chest images for detection of interval change. Med Phys 1994;

21:453–461.

Academic Radiology, Vol 18, No 8, August 2011 CAD FOR LUNG NODULE IN DIGITAL RADIOGRAPHS

10. Ishida T, Katsuragawa S, Nakamura K, et al. Iterative image warping tech-

nique for temporal subtraction of sequential chest radiographs to detect

interval change. Med Phys 1999; 26:1320–1329.

11. Xu XW, Doi K. Image feature analysis for computer-aided diagnosis: accu-

rate determination of ribcage boundary in chest radiographs. Med Phys

1995; 22:617–626.

12. Katsuragawa S, Doi K, MacMahon H. Image feature analysis and

computer-aided diagnosis in digital radiography: detection and character-

ization of interstitial lung disease in digital chest radiographs. Med Phys

1988; 15:311–319.

13. Xu XW, Doi K, Kobayashi T, et al. Development of an improved CAD

scheme for automated detection of lung nodules in digital chest images.

Med Phys 1997; 24:1395–1403.

14. Duda RO, Hart RE, Stork DG. Pattern Classification. 2nd ed. New York:

John Wiley, 2001.

15. Jain AK, Duin PW, Mao J. Statistical pattern recognition; a review. IEEE

Trans Pattern Anal Mach Intell 2000; 22:17–24.

16. Shiraishi J, Katsuragawa S, Ikezoe J, et al. Development of a digital image

database for chest radiographs with and without a lung nodule: receiver

operating characteristic analysis of radiologists’ detection of pulmonary

nodules. AJR Am J Roentgenol 2000; 174:71–74.

17. Kobayashi T, Xu XW, MacMahon H, et al. Effect of a computer-aided diag-

nosis scheme on radiologists’ performance in detection of lung nodules on

radiographs. Radiology 1996; 199:843–848.

18. Metz CE, HermanBA, Shen JH.Maximum-likelihood estimation of receiver

operating characteristic (ROC) curves from continuously-distributed data.

Stat Med 1998; 17:1033–1053.

19. Dorfman DD, Berbaum KS, Metz CE. ROC rating analysis: generalization

to the population of readers and cases with the jackknife method. Invest

Radiol 1992; 27:723–731.

20. Freedman MT, Osicka T, Lo SCB, et al. Methods for identifying changes in

radiologists’ behavioral operating point of sensitivity-specificity tradeoffs

within an ROC study of the use of computeraided detection of lung cancer.

Prop SPIE 2001; 4324:184–194.

21. Henschke CI, Naidich DP, Yankelevitz DF, et al. Early lung cancer action

project: initial findings on repeat screenings. Cancer 2001; 92:153–159.

22. Patz EF, Black WC, Goodman PC. CT screening for lung cancer: not ready

for routine practice. Radiology 2001; 221:587–591.

1005