Download - Enhancement of chest radiographs with gradient operators

IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 7, NO. 2 , JUNE 1988 109

Enhancement of Chest Radiographs with Gradient Operators

JOHN S . DAPONTE AND MARTIN D. FOX, MEMBER, IEEE

Abstract-The use of gradient operators for image enhancement has been widely reported in the literature, but they have not been used routinely in the medical arena, particularly in the most common radiographic plain film procedure, chest radiographs. Gradient operators such as Sobel and Roberts operators, not only enhance image edges but also tend to enhance noise. Overall, the Sobel operator was found to be superior to the Roberts operator in edge enhancement. A theoretical explanation for the superior performance of the Sobel operator was developed based on the concept of analyzing the x and y Sobel masks as linear filters. By applying pill box, Gaussian, or median filtering prior to applying a gradient operator, noise was reduced, but the pill box and Gaussian filters were much more computationally efficient than the median filter with approximately equal effectiveness in noise reduction.

INTRODUCTION N image processing an edge is defined as a difference I in image characteristics within a local region [ 13. Edge

detection, a technique that has been used for the purpose of enhancing radiographic images, is carried out by computing the rate of change with respect to gray levels for two adjacent regions within an image [2]-[4]. Gradient operators are a well-known form of edge detection which are based on digital approximations of partial differential equations [ 5 ] . Alternative techniques for estimating derivatives of a digital image include differentiation of least-square fitting polynomials and Wiener filter approximations 161, but these techniques tend to require more computation than the gradient operator approach.

The ability to emphasize the fine details of a radiographic plain film image while suppressing noise was demonstrated qualitatively for cephalometric radiographs by employing a median filter prior to applying a gradient operator 171. Although the ability of a median filter to suppress noise has been previously established 181, [9] our results indicate that approximations to the pill-box blur and the Gaussian low-pass filter yield a similar amount of noise reduction with much better computational efficiency and subjectively no loss of useful image detail.

Edge detection can either provide an enhanced image for a human observer, or allow significant data reduction prior to further computer processing. A potential application of this is the early detection of lung nodules [ 101,

Manuscript received July 26, 1987; revised January 20, 1988. This work was supported in part by grants from the Whitaker Foundation and the State of Connecticut Department of Higher Education.

J . S . DaPonte is with the Department of Computer Science, Southern Connecticut State University, New Haven, C T 06515.

M. D. Fox is with the Department of Electrical and Systems Engineer- ing and Radiology, University of Connecticut, Storrs, CT 06268.

IEEE Log Number 8820192.

[ 111. Previously developed computer controlled lung nod- ule searches have often used edge detection as a preliminary processing step [ 121, [ 131.

The primary objective of our study was to investigate the application of gradient operators to the most common radiographic procedure, plain film chest X-rays [ 141 using a frequency domain formulation to explain the effects of Sobel and Roberts gradient operators on the image. Im- ages were digitized using a video frame grabber on a microcomputer based system [ 151, [ 161. In addition, we have quantitated the advantages of combining various prefiltering techniques with the Sobel gradient operator, which proved to be the most efficient, and compare the signal- to-noise ratio as well as the amount of time required for each combination.

THEORY

A common method for computing the rate of change with respect to gray level in an image is to use gradient operators which in their simplest form consist of partial difference operators. While these partial difference operators lead to relatively simple calculations, they suffer from a dependence on directionality. Thus, more nonlinear techniques derived from two-dimensional filtering may be necessary to effectively display localized changes in gray level about a pixel.

Two-dimensional linear digital filtering of images is ac- complished by storage of values, multiplication by con- stants, and addition. For example, consider the M X N input image X ( mp, nq) which has the two-dimensional z- transform [ 171.

M N

X(Z,, Z,,) = c c x ( m p , nq)Z,"Z," (1) m = O n = O

where 2, = and Z,, = eJ2Tz'4, U and v are spatial frequencies and p and q are pixel spacings in the x and y directions, respectively. If we apply a filter to the image X(Z,, Z , ) , the input and output images can be related through the expression

where H(Z,, Z,,) is the transfer function of the filter. The two-dimensional digital convolution operation y = x ** h can be defined [ 171

w m

Y(m, n ) = c c h(a, b ) X ( m - a , n - b ) ( 3 ) a = - w h = --m

0278-0062/88/0600-0109$01 .OO @ 1988 IEEE

110 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 7 . NO 1, I U N E 1988

which is the corresponding operation to the filtering operation of (2) in the image domain where in this context m, n and a , b are pixel indexes. Note that by substitution for the complex variables Z , = e Z , , = e '**li4 in ( 2 ) we can determine the transfer function H ( u , U ) in terms of spatial frequency.

Gradient operators can be distinguished from simple convolutional filters as defined by ( 3 ) in that they require the use of two masks, one to obtain the x-direction gradient, the second to obtain the y-direction gradient, the results of which are then combined to obtain the orthogonal components of a vector quantity whose magnitude represents the strength of the gradient or edge at a point in the image and whose angle represents the gradient angle. In general, while gradient operators are thus nonlinear operators, which are not necessarily quantifiable, through linear filter analysis, the x and y masks can be separately understood as linear filters.

A. Sobel Operator The Sobel operator can be defined by the two kernels

[21, [31

- 1 0 1 -1 -2 -1

S, = - 2 @ 2 and S, = 0 @ 0

-1 0 1 1 2 1

which are sequentially convolved with the original image file. The result is stored in the center or circled pixel and can be represented as a vector quantity with magnitude

lY(mP, n q ) ( = (x(mp, nq) ** + I X h P ? nq) ** s,jl (4)

and angle

Arg [ Y b P , n s ) ]

= arctan [x(mp, nq) ** ~ , / x ( m p , nq) ** s,] ( 5 )

where ** represents two-dimensional convolution and i a n d j are unit vectors in the x and y directions, respectively. The two-dimensional z transform of (4) is

Y ( z u , Zt , ) = X(Zu, Zr,) sx(Zu, Zrl)i

+ X(ZU> Z , ) S,(ZU> Z , ) j ( 6 )

and the transfer function of the Sobel operator can be ex- pressed

HS(ZU, ZV) = WU9 Z , , > / X ( Z u , Z , , )

= S,(Z, , ZV)i + q z , , Z , ) j . (7)

The z transform of the Sobel operator S, will be

Z , , ) = '(1 + z,' + Z ? ' )

- zb(1 + z,' + z;') + z,' - z,+' ( 8 )

T

- 5 - 5

(b)

Fig. 1 . Three-dimensional mesh plots of the magnitude of the spatial fre- qlrency response of a single component of the (a) Sobel and (b) Roberts operators. Results are plotted out to the sampling theorem limit in each case. Note that the Sobel response is better controlled orthogonal to the direction of differencing.

which can be simplified to obtain

&(U, U ) = 2j sin (2xup)[2 + 2 cos (27rvq)I. (9) Similarly,

sY(u, U ) = 2 j sin (27ruq)[2 + 2 cos TUP)]. (10)

Note that fo rp= q, &(U, U ) = &(U, U ) aftera 90 degree rotation. A three-dimensional plot of 1 S., ( U , U ) 1 is shown in Fig. 1.

B. Roberts Operator The Roberts operator convolves the kernals

0 + 1 - 1 0 r-45 =

O Q r4s = -1 @

with the image to obtain orthogonal gradients along the 5 4 5 degree axes. These operators are convolved with the input image x(mp, n q ) to obtain the complex modified image

Note that any orthogonal pair of axes, in this case k45 can provide an estimate of the strength and angle of an

DA PONTE AND FOX: CHEST RADIOGRAPHS WITH GRADIENT OPERATORS 111

Gradient

F i l t e r Magnl tude

3 '4 Phase

G = Origlnal Image M = F i l t e r Mask

M = Second Gradient Mask M 1 = Fi rs t Gradient Mask

2 3

Fig. 2 . Block diagram of experimental schema

edge. The more important issue is how well the operator can detect edges, particularly in the presence of noise.

blur operator which is a convolution type filter with the following mask:

1 1 1 1 1 The two-dimensional 2 transform of the Roberts operator r45 1s

2,) = 2;' - 2," (12) and along the 45 degree line if p = (U' + U * ) is the radical coordinate i t t h e frequency domain and p = q, we

1 2 2 2 1

1 2 0 2 1

1 2 2 2 1

1 1 1 1 1 have U = v = p / J 2 and we obtain

~ 4 5 ( p , 450) = -2 j sin (2apq/&) -( 13)

(14) The second mask is a 3 x 3 approximation to a Gauss-

~ - 4 5 ( p , 450) = -2j sin (27rpq/Jz e-*j*q/fi. ian low-pass convolution type filter defined as

A three-dimensional plot of I r45(Zu, Z,,) 1 is shown in Fig. 1 for comparison to the Sobel operation.

Inspection of the frequency response plots shown in Fig. 1 reveals that the Sobel operator tends to filter out spatial

1 2 1

2 @ 2

1 2 1 frequencies orthogonal to the direction of differencing (edge detection) better than does the Roberts gradient operator. This behavior in the frequency domain thus pro-

The third filter was a 3 x 3 median filter with the following mask:

vides a possible explanation for the superior performance of the Sobel operator in detecting edges in a noisy envi- ronment.

While gradient operators are commonly used for edge

1 1 1

1 0 1

1 1 1 detection, they have the drawback of enhancing image noise. One way of reducing this problem is to filter high The fourth was a median with the spatial frequencies containing noise prior to applying the mask: gradient operators. This process can best be described with the aid of a functional block diagram as depicted in Fig. 2. If we let G denote the gray levels of the original image and let MI denote the filter mask, then the first step is to filter the image. Next the filtered image is processed by two gradient masks, M2 and M 3 , producing orthogonal components which are passed to a magnitude operator which takes the absolute value of each pixel in the orthogonal matrices and sums them. Finally, the output magnitude is compressed to fit in the available gray levels.

Four types of noise reduction filters were investigated in this study for use prior to the application of gradient operators. The first was an approximation to the pill-box

1 1 1 1 1

1 1 1 1 1

1 1 0 1 1

1 1 1 1 1

1 1 1 1 1

For each of these filters the pixel of interest located in the center of the mask has been circled.

The pill-box and the Gaussian operators smooth the image by convolving the corresponding mask, while the median filters are a nonlinear operator that sort each pixel in its neighborhood according to intensity and replaces the

112

center pixel by the median of the sort. For example in a 5 X 5 region of interest there are 25 points that must be sorted when using a median filter. Thus, as the mask is translated about the region of interest, a large number of pixels must be sorted over and over again resulting in a very time consuming process.

METHODS The system used to acquire and process the images pre-

sented in this paper consists of a commercial frame grabber' installed on a personal compute? [ 151 containing 640 Kbytes of memory. A functional block diagram of a typical frame grabber system is shown in Fig. 3 .

The frame grabber board is memory mapped into the address space of the host computer and has the ability to store two images at a time in its on-board memory. Ap- plication software resides in the host computer's memory and interacts with the data via the bus interface.

Radiographic images were captured from the teaching files at the University of Connecticut Health Center using a video camera which adheres to the RS-170 video standard [IS]. Once the board had digitized an image, it was passed through an input look-up table which could per- form real time gray scale manipulations before being stored in the frame memory. Then the various software algorithms in the host memory were applied to obtain the processed image which was passed through an output look-up table and converted to analog before being dis- played on an external video monitor. Hard copies of output images were produced using a thermal video ~ r i n t e r . ~

RESULTS A variety of chest radiographs were processed using the

techniques described in this paper. We found that the So- bel operator subjectively produced much better results than did the Roberts operator for the same image. The output edges of the Roberts operator were so faintly dis- played that histogram equalization was necessary to make them visible which resulted in the unacceptable output depicted in Fig. 4. Thus, the resulting sequence of photo- graphs are limited to demonstrating the effects of the So- bel gradient operator on the same image combined with various types of filtering.

Fig. 5 contains three unprocessed chest radiographs, a normal (a), a subject with a hamartoma (b), and subject with a squamous cell carcinoma (c). The results of applying the Sobel operator to these images without prefiltering is illustrated in Fig. 6. Note the occurrence of noise in these images demonstrating the amplification of noise caused by applying the Sobel gradient operator. The images in Fig. 7 were obtained by first applying the pill-box blur and then the Sobel operator to the chest radiographs. In these cases, the image noise subjectively appears to have been reduced while the edges, containing signal, can be distinguished.

'Imaging Technology, Inc 'IBM PC/AT Series 'Mitsubishi Model P60U

IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 1, NO. 2. JUNE 198X

[ I 1lel D m

Fig. 3 . Block diagram of a typical frame grabberiimage procesaor system.

Fig. 4. Typical chest radiograph after Roberts operator and histogram equalization.

Following a similar pattern the results of prefiltering with a 3 X 3 Gaussian, a 3 X 3 median, and a 5 X 5 median to each of the original chest radiographs are pre- sented in Figs. 8, 9, and 10. Subjectively, all of the pre- filtered images appear to have reduced noise when com- pared to cases in which the Sobel is applied without prior filtering.

In order to compare these prefiltering techniques, a quantitative approach was used in which both the signal- to-noise ratio and computational time of each filter was considered. The signal-to-noise ratio was calculated by first identifying a region of interest without anatomical data as a measure of image noise and then selecting a second region of interest including the cardiac shadow and the lung fields as a measure of typical image signal. Then the gray level histogram for each region was evaluated as well as the mean, standard deviation, and range of the

DA PONTE AND FOX: CHEST RADIOGRAPHS WITH GRADIENT OPERATORS 113

(c) Fig. 6 . Chest radiographs after application of Sobel gradient operator but

without prefiltering. (a) Normal. (b) Hamartoma. (c) Squamous cell carcinoma.

(C)

Fig. 5 . Original unprocessed chest radiographs. (a) Normal. (b) Hamar- toma. (c) Squamous cell carcinoma.

gray level intensities. The signal-to-noise ratio in each case was computed as the range of gray level intensities in the signal region divided by the standard deviation of the gray level intensities in the noise region.

Calculating the signal-to-noise ratio of an image in this

way is an extension of theory established in one-dimensional linear systems. For a two-dimensional image, let

(15) g(x9 Y > = f ( x , Y ) + ( x , Y ) wheref(x, y ) is the signal and n ( x , y ) is the noise. Now

IEEE TRANSACTIONS ON MEDICAL IMAGING. VOL. 7. NO. 2. JUNE 1988

(C)

Fig. 7. Chest radiographs after prefiltering with pill-box blur and application of the Sobel gradient operator. (a) Normal. (b) Hamartoma. ( c ) Squamous cell carcinoma.

assuming a linear system with an impulse response h ( x , y ) the resulting output I ( x , y ) is given by

( C )

Fig. 8. Chest radiographs after prefiltering with a 3 x 3 Gaussian filter and application of the Sobel gradient operator. (a) Normal. (b) Hamar- toma. ( c ) Squamous cell carcinoma.

for which Z f ( x , y ) = f(x, y ) * h ( x , y ) and Z,l(x, y ) = n ( x , y ) * h ( x , y ) are the output components due to signal f ( x , y ) and noise n ( x , y ) . Then the signal-to-noise ratio is defined as 1191

~

DA PONTE AND FOX CHEST RADIOGRAPHS WITH GRADIENT OPERATORS

(a)

(C)

Fig. 9. Chest radiographs after prefiltering with a 3 x 3 median filter and application of the Sobel gradient operator. (a) Normal. (b) Hamartoma. (c) Squamous cell carcinoma.

For our images the deterministic value I If ( x , y ) 1 is taken as range, ( I , , , - Z m l n ) , of the histogram in the signal re-

I15

(c)

Fig. 10. Chest radiographs after prefiltering with a 5 x 5 median filter and application of the Sobel gradient operator. (a) Normal. (b) Hamartoma. (c) Squamous cell carcinoma.

gion and the stochastic expression J E ( I Z,l(x, y ) I )' is taken as the standard deviation of the histogram in the noise region.

The results of this analysis are depicted in Fig. 11 in which the signal-to-noise ratio is plotted as a side-by-side bar chart for each chest radiograph. The bars are grouped

1 I6 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 7, NO. 2, JUNE 1988

Ctii 3 1 RADIOGRAPH S/N R4TlO

I - 1

NO PrrLi lL iFR C1115S WLLBOX ~ ~ 0 3 3 1 3 MLDSB15

N U H M I L t+AMAl?TOM* a SOUAM

Fig. I I . Signal-to-noise ratio for normal, hamartoma, and squamous cell carcinoma ( q u a m ) chest radiographs after Sobel gradient operator with no prefiltering, after a 3 x 3 Gaussian low-pass prefilter, after a pill-box blur prefilter, after a 3 x 3 median prefilter, and after a 5 X 5 median prefilter.

so that the signal-to-noise ratio obtained by applying the Sobel gradient operator without prefiltering is illustrated first, followed by the results of prefiltering with a Gauss- ian, a pill-box blur, a 3 X 3 median, and a 5 X 5 median. Each of the prefiltering combinations demonstrate a signal-to-noise ratio that is better than the ratios obtained from images in which the Sobel was applied without prefiltering. For example, the signal-to-noise ratio for the normal chest radiograph after processing with the Sobel gradient operator, but without prefiltering, is 16. The signal-to-noise ratio in the same image which has been pre- filtered with a 3 X 3 Gaussian low-pass filter is 27 and it is 34 if the pill-box blur is used as the prefilter. When the 3 x 3 median and the 5 X 5 median filters are used to prefilter the image, the signal-to-noise ratio is 20 and 23, respectively.

In general, for the three cases studied, the signal-to- noise ratio of the unprocessed original images was degraded by all of the edge detection methods investigated. For the hamartoma and the squamous cell carcinoma the signal-to-noise ratio were approximately the same both before processing, with the Sobel gradient operator, at 89 and 92, and after processing where the degraded signal- to-noise ratio were still quite close at 31 and 29. For the normal image the signal-to-noise ratio was 42 before processing and was degraded to 16 after processing.

The other important quantitative factor is the amount of time each process requires. On our system, approximately 1 min is required to process an entire image consisting of 480 rows and 5 12 columns with a Sobel gradient operator. The 3 by 3 Gaussian low-pass filter was the fastest prefilter studied requiring only 50 s, followed closely by the pill-box blur which required 54 s. The median filters required the most time. To prefilter an entire image with a 3 x 3 median filter required approximately 7 min while a 5 x 5 median filter required slightly over 34 min.

DISCUSSION AND CONCLUSION We have outlined a theoretical framework based on the

2 transform for analyzing gradient operators. In addition, we have demonstrated on a microcomputer based system that image noise in gradient operator edge detection can be substantially reduced by prefiltering the image. Over- all, the Sobel operator appears to be more suitable for processing chest radiographs than the Roberts operator and we have proposed a theoretical explanation for this result.

While each of the prefiltering techniques were all suc- cessful in improving the signal-to-noise ratio, the pill-box blur and the Gaussian were found to require much less computer time than either of the median filters. This may prove significant in development of practical algorithms for the present generation of microcomputers. As a result of the extra time required and the lack of significant im- provement in signal-to-noise ratio neither the 3 x 3 nor the 5 X 5 median filters are recommended with the present hardware for processing chest radiographs. The 3 x 3 Gaussian low-pass filter produced edges that were slightly better defined than those obtained with the pill- box blur. It should be noted that since the Gaussian used a 3 x 3 kernel, it had a wider frequency passband allow- ing through more high spatial frequencies than did the pill- box filter, which may account for the improved edge detection. The most efficient procedure was to filter the image with the Gaussian filter approximation before applying the Sobel operation.

In conclusion, our results indicate that gradient operators are a conceptually tractable and computationally ef- fective means to enhance edges of objects. The resulting improved visualization can be particularly helpful for objects that can be difficult to visualize without processing, such as lung nodules.

REFERENCES [ I ] R. C. Gonzalez and P. Wintz, Digital Image Processing. Reading,

MA: Addison-Wesley, 2nd ed., 1987. 121 E. R. Dougherty and C. R. Giardina, Image Processing-Continuous

to Discrete, Volume 1. Englewood Cliffs, NJ: Prentice-Hall, 1987. 131 -, Matrix Structured Image Processing. Englewood Cliffs, NJ:

Prentice-Hall, 1987. 141 S . M. Kay and G . J . Lemay, “Edge detection using the linear model,”

IEEE Trans. Acoust., Speech, Signal Processing, vol. A S S - 3 4 , Oct. 1986.

[ 5 ] K. S. Fu and J. K. Mui, “A survey on image segmentation,” Pattern Recognition, vol. 13, no. 1, pp. 3-16, 1981.

[6] M . Hashimoto and J. Sklansky, “Multiple order derivatives for detecting local image characteristics,” Compur. Vision Graphics Image Processing, vol. 39, pp. 28-55, July 1987.

[7] P. H. Jackson, G. C. Dickson, and D. J . Brine, “Digital image processing of cephalometric radiographs: A preliminary report,” Brit. J . Orthodontics, vol. 12, pp. 122-132, 1985.

[81 W. K . Pratt, Digital lmage Processing. 191 I . Pitas and A. N. Venetsanopoulos, “Nonlinear mean filters in image

processing,” IEEE Trans. Acoust., Speech, Signal Processing, vol. ASSP-34, June 1986.

[IO] H . L. Kundel, N. F. Nodine, D. Thickman, and L. Toto, “Searching for lung nodules; A comparison of human performance with random and systematic scanning models,” Investigative Radiol. , vol. 22, May 1987.

[11] H. L. Kundel, G . Revesz, and L. Toto, “Contrast gradient and the detection of lung modules,” Investigafive Radiol. , vol. 14, 1979.

New York: Wiley, 1978.

DA PONTE AND FOX: CHEST RADIOGRAPHS WITH GRADIENT OPERATORS

[ 121 J . Hasegama, J . Toriwaki, et a l . , “Further experiments of the software system AISCR-V3 for computer aided screening of chest pho- toflurograms, ” in Proc. Inc. Conf. Pattern Recognition, 1982, pp. 273-275.

[ 131 W. A. Lampetor and J . C. Wandtke, “Computerized search of chest radiographs for nodules,” Investigative Radiology, pp. 384-390, May 1986.

1141 G . A . Johnson, N. Dankly, and C. E. Ravin, “Processing alterna- tives for digital chest imaging,” Radiologic Clinics of North Amer-

117

ica, vol. 23, June 1985. [I51 M . D. Fox, “Analysis of medical images with microcomputers,”

Amer. J . Physinl. Imaging, vol. 2, pp. 72-78, 1987. [ 161 -, “Evaluation of a microcomputer based processor for ultrasound

images,” J . Clinical Eng., vol. 13, pp. 37-43, Jan;Feb. 1988. [ 171 R . R. Rabiner and B. Gold, Theory and Application of Digital Signal

Processing. [ 181 EIA Standard-170, Electrical Performance Standards-Monochrome

Television Studio Facilities, Electronic Industries Association. [ 191 A. Papoulis. Signal Analysis.

Englewood Cliffs, NJ: Prentice-Hall, 1975, ch. 7.

New York: McGraw Hill, 1977.

Download - Enhancement of chest radiographs with gradient operators

Top Related