automatic rib detection in chest radiographs

COMPUTER VISION, GRAPHICS, AND IMAGE PROCESSING 23, 129- 16 1 (I 983)

Automatic Rib Detection in Chest Radiographs F%TERDESOUZA*

IBM U. K. Scientific Centre, St. Clement Street, Winchester SO23 9DR, Engiund

Received February 3, 1982; revised April 22, 1982

A detailed algorithm is presented for automatic detection of posterior (dorsal) ribs in posterior/anterior chest radiographs. It is entirely automatic and includes algorithms for locating the lung fields and the lung boundary. Written in PL/I for an IBM 303 1, it requires about 8 set to process a 400 x 400 digitized full chest radiograph which is significantly faster than previously publ ished results. The reason for the efficiency can be attributed to the fact that no two-dimensional image processing is performed; it works instead by locating the ribs on a small set of one-dimensional sections taken through the lung fields. Curves are then fitted through these locations to represent the rib edges. The detection of anterior (ventral) ribs is also discussed.

I, INTRODUCTION

The ribs provide a convenient frame of reference in chest radiographs, and the ability to detect them automatically is an important step in striving towards computer aided diagnosis, or in designing an automatic chest screening system. For these reasons automatic rib detection in posterior/anterior chest radiographs has received considerable attention in recent years.

Wechsler et al. [l-3] achieved some impressive results in their attempt to outline the visible parts of the posterior (dorsal) ribs and anterior (ventral) ribs. They reported an error rate of lo-15% on a small test set of five radiographs in an approach which used powerful image processing techniques including filtering, edge detection, and Hough transforms. Although their results are encouraging, the ma jor drawback of their method lies in the amount of computation involved. Programmed in Fortran on a DEC PDP 11/45 their algorithm required about 18 m m of processor time when applied to a 256 X 256 digitized radiograph [3]. This is too slow to represent a practical proposition where an analysis in real time is the ultimate aim.

Persoon et al. [4, 51 also reported some success al though they were interested in posterior ribs only. Their method relied heavily on edge detection and a novel edge detector was proposed for this purpose. Again the principal drawback of this method lies in the computational workload. Wechsler and Fu [3] found that it took about 31 m in to process a 256 x 256 radiograph on a PDP 1 l/45 when pro- g rammed in Fortran and Assembler.

Toriwaki et al. [6-81 devised a set of posterior and anterior rib detection algorithms which used a linear filter to detect the ribs and represented the rib contours as quadratic curves. Their program required about 1.5 m in to process a 250 X 250 radiograph on a FACOM 230-75 [6] which is a considerable improvement on the times above but is still appreciably slower than real time. Although the principal concepts of Toriwaki’s system are clear, details are not always available, making it difficult to contrast this method with others. Furthermore, as Wechsler [l]

*Present address: IBM, T. J. Watson Research Center, PO Box 218, Yorktown Heights, New York, 10598.

129 0734-189X/83 $3.00

Copyright Q I983 by Academic Press. Inc. All rights of reproduction in any form reserved

130 PETER DE SOUZA

and Fu et al. [4] noted, there appear to be difficulties with some of Toriwaki’s underlying principles. Indeed, Toriwaki et al. [6] agree that “a complete procedure for locating rib borders has not been developed.”

Two other groups that have proposed noteworthy rib detection algorithms are Kulick et al. [9, lo] and Li et al. [ 1 l-131. Their methods have much in common with each other and are both based on vertical sections (one pixel wide) taken through the lung fields. The ribs are identified by locating the rib edge points on these sections and then either tracking the entire rib edge from these points, or by linking together edge points on different sections and fitting quadratic curves through them.

The great merit of these latter methods is that they are undoubtedly very fast, the majority of the processing being performed on one-dimensional sections. Very little in the way of conventional image processing is required except a little smoothing. The disadvantage of working on a few sections rather than the entire image is that much information is discarded, and any noise or irregularity in the sections is likely to have a more detrimental effect than would otherwise be expected. It is not surprising, therefore, that both groups reported some reliability problems.

In this article a posterior rib detection algorithm is described which has been programmed in PL/I for an IBM 3031 and which requires approximately 8 set to process a 400 x 400 radiograph. This is significantly faster than the times quoted above, particularly as it refers to radiographs which are more than twice the size of those reported previously.

The reason for the apparent efficiency of the algorithm is that no expensive image processing operations are performed. This occurred more by evolution than by design, although it seems to me at least that there is actually no particularly good reason for using image processing techniques in this kind of project. A digitized radiograph is nothing more than an array of data; it only becomes an “image” in the perception of man, and only after the values in that array have been converted into light. In automatic rib detection the radiograph is not presented for viewing in this manner and therefore it remains just an array of numbers. The difference between an image and an array of numbers can best be appreciated, perhaps, by realizing that finding the ribs visually on a radiograph is simple. Finding the ribs manually given only a printout of a digitized radiograph is much more difficult and requires a very different technique involving logical reasoning rather than visual processing. Note also that image enhancement is of virtually no benefit in the latter case. The computer has only the second representation and it seems appropriate that it should be programmed to perform the task in the way that a person might attempt it under similar circumstances, i.e., based more on deduction than on image processing.

That is not to say that there is nothing to learn from the way in which we perform the task visually, but it is relevant only insofar as it acts as a stimulus in the design of a sensible strategy. It is clear, for example, that the eye does not detect ribs by tracking their edges; the eye is not confused by gaps in the rib outlines. Rather it appears to use parallel processing to detect a rib along a broad front instantaneously. We might be influenced by this observation into rejecting tracking algorithms which get into difficulties wherever the track is obscured, and which are also vulnerable to being sidetracked by other irrelevant tracks. Instead we might attempt to simulate parallel processing and hence detect the entire length of the rib at once. This thought was, in fact, the principal motivating force behind the method about to be described.

AUTOMATIC RIB DETECTION 131

2. OVERVIEW OF THE ALGORITHM

The object of the algorithm being described is to detect the location of the posterior ribs entirely automatically. Therefore it must be capable of processing the whole of the exposed area of the radiograph and must not require that an area be preselected, although it should also function correctly in the latter case. Under normal operation, the whole of the exposed area is input to the program in the form of a rectangular array, and it is then up to the program to establish where the lung fields are and to process them. Since this area varies in size from one person to another, it follows that the input array is not usually of constant size. In our system the radiographs were digitized into arrays which averaged 422 pixels in height and 390 in width. One of these radiographs is shown in Fig. 1. In accordance with standard viewing practice, the left lung is on the right and vice versa.

The first task of the program is to establish the position of the lung fields. Having done so, it processes them separately. For programming simplicity, the left lung field is reflected about its central column so that it appears to be a right lung field. This simplification means that the program need only be defined in terms of right lungs, and in effect can achieve the desired processing by being invoked twice: once to process the right lung, and once to process the reflected left lung.

250

200

150

100

50

too 150 200 250 300 350

FIG. 1. Chest radiograph.

132 PETER DE SOUZA

The right lung tends to be easier to process than the left because in the latter the presence of the heart tends to obscure parts of the lower ribs. For this reason the right lung is analyzed first and some of the results are carried over to assist in processing the left. This both reduces problems due to the heart and saves time by avoiding unnecessary duplication of certain procedures. In doing this, there is an implicit assumption that the radiograph is roughly symmetric about the mediastinum. If this is not valid, as might be the case with scoliotics, for example, the program might not detect the ribs properly. However, several checks are included to ensure that the program is proceeding properly, and if an irrecoverable error is detected it displays an error message and terminates.

Having identified the lung fields the program treats each one separately, first locating the outer edge of the lung, i.e. the edge of the rib cage, and then locating the ribs. The ribs are detected by taking several vertical sections through the lung and simultaneously estimating the rib locations on all of them. These locations are then refined on each section individually, and finally the rib contours are modeled by curves fitted through the rib edge points.

Composite curves are employed for the modeling as the contours have an infinite gradient at the outside edge of the lung where they become vertical, and they can also have a zero gradient and turning point elsewhere in the lung. These properties mean that simple quadratics as used by Toriwaki et al. [6] and Brace et al. [lo] are inadequate, a point of criticism that has been noted before [4]. It should be appreciated, however, that the fitted curves are mathematical conveniences rather than actual rib edges. But they are sufficient for specifying the rib locations, and for providing a frame of reference for “navigating” round the chest. Additionally, they enable estimates to be made of the rib edge contours even when these are partially obscured by tumours, vascular tissue, or other obstructions.

Our original aim in designing a rib detector was to enable the lungs to be identified and to facilitate automatic screening of the lung fields. In this context only the lower edge of the top rib is of interest; the upper edge lies outside the lung and is largely irrelevant. Therefore no attempt is made to identify the upper edge of the top rib. The bottom of the lung is taken to be the locus of points comprising the upper edge of the lowest visible rib or the top of the diaphragm, whichever is the higher. This is also taken as the lowest “rib” edge of interest, although clearly it may be the diaphragm edge in part. Again no attempt is made to identify the lower edge of the bottom rib. It is often practically invisible in whole or in part and is also largely irrelevant.

In most radiographs, between these two extreme edges there are seven visible ribs, although in a few cases there are eight. In the latter cases the top two ribs tend to be so close together that there is no intercostal space between them, or only a very small amount. It is not only very difficult to separate these ribs, it is also not very helpful to do so in terms of delineating the intercostal spaces for screening. Consequently, the program looks for 7 ribs only and outlines both edges of all 7, making a total of 16 rib edges altogether. Attempts were made to detect an eighth rib where one was visible but they were unsuccessful and were abandoned. It was felt that the meager benefits did not justify the effort. It is noteworthy that Persoon [5] also found that in some radiographs “the ribs above the clavicle were not always found.” However, it would be intuitively pleasing to locate every single rib for the sake of completeness, and there is room for further development in this regard.


Let us turn now to the algorithm itself and consider the major components individually. The radiograph shown in Fig. 1 will be used for illustration purposes. This is the radiograph of a child which was originally digitized at a resolution of 200 p into a 1344 X 1080 array, and was then reduced to a 448 X 360 array by averaging over 3 X 3 windows. The gray values were quantized to 8 bits and therefore lie in the range 0 to 255. White was coded as 0 and black as 255.

3. LOCATING THE LUNG FIELDS

The first step in the algorithm is to determine the location of the lung fields, and this is accomplished in a manner similar to that suggested by Toriwaki et al. [6]. The vertical signature of the input radiograph is computed by adding together the elements in each column. In our system the top quarter and bottom quarter of each column are ignored and only the elements in the central half of each column are summed. The signature so obtained for the radiograph in Fig. 1 is shown in Fig. 2a. The lung regions show up as the two humps, and the edges of the rib cage show up as the two inverted peaks at each end. The column lim its of the lung fields are identified from the signature as follows.

Let L denote the largest value of the signature, and let S denote the smallest value in the m iddle half. Then a threshold 0 is defined as

0 = s + (L - S)/lO 0)

which lies 10% of the way between S and L. The right lung is then taken as being that region of the left hump which exceeds 0. From the definition of 0 it is clear that the signature always falls below 0 in the center, but it can happen that it does not do so at the outer edge. In this case the outer edge of the lung is undefined and is taken instead as being that point in the first quarter of the signature where the signature takes its lowest value; this would be the first inverted peak in the current example.

The outside edge of the left lung field is computed in the same way, and the inner edge is then selected so that the left lung field has the same width as the right. This forces the fields to be of identical size and also avoids problems due to the heart which can distort the signature near the inner edge of the left lung.

The column lim its obtained for the lung fields of the radiograph in Fig. 1 can be seen in Fig. 2b. Row lim its are not computed since they are not required.

From this point onwards, all the operations are defined in terms of the right lung as explained in Section 2.

4. COMPUTING VERTICAL SECTIONS THROUGH THE LUNG

Having located a lung field, a number of vertical sections are defined through it. The ultimate intention is to identify the ribs on each section, thereby being able to build up a picture of the ribs on a wide front across the lung without actually having to process every column along that front. The number of sections so defined is 2n + 1 where in our system n = 5.

The sections are chosen so that none of them lies in the first eighth or last eighth of the lung field. The first eighth tends to be so near the edge of the lung that the posterior ribs are almost vertical which together with the presence of the anterior

134 PETER DE SOUZA

a

25 - I

50 100 150 200 250 300 350

b

50 100 150 200 250 300 350

FIG. 2. Signature (a) and position of lung fields (b) of the radiograph in Fig. 1.


ribs makes them difficult to identify on vertical sections in this region. The last eighth tends to be cluttered with blood vessels which obscure the ribs.

The m iddle vertical section is defined so as to lie in the center of the lung field, and the other sections are equally spaced so as to fill out the central three quarters of the lung field as far as possible without going beyond its edges.

Having decided upon the location of the vertical sections, it is desirable to smooth them before attempting to identify where they intersect with the posterior ribs. Kulick et al. [9] and Fong et al. [ 131 used hysteresis smoothing for this purpose, although the latter group expressed some reservations about doing so. However, this does little to counter the other previously mentioned problems associated with taking sections: the loss of information due to discarding information between the sections, and the exaggerated effect of irregularities or serious noise on any given section. We will now consider an alternative means of smoothing the sections while simultaneously reducing these difficulties.

It is well known that the variance of a mean is less than the variance of the data from which the mean is calculated. Hence, if we could somehow average several columns together to form a mean section, it is clear that the effect of noise would be reduced, less information would be discarded, and the mean section would be smoother than any of the individual sections. It is also clear, however, that since the ribs are not horizontal this simple procedure would blur the rib edges, making them less well defined. Fortunately, this problem can be circumvented.

Because the posterior ribs are approximately parallel, it follows that any two vertical sections taken through the lung will display roughly the same pattern of ribs, although the patterns will not usually be exactly aligned. Consider, for example, the two sections shown in Figs. 3a and b which were taken 30 pixels apart through the right lung of Fig. 1. (The bottom of the radiograph is on the left and the top is on the right.) They are not aligned identically but they do both show the same pattern: seven rib troughs and an eighth trough due to the clavicle and/or the top of the rib cage, with very similar spacing between them all. Furthermore, the closer the sections are together the more similar their patterns will be. Thus it is clear that if we take vertical sections only a few pixels apart, and shift them vertically relative to each other so as to align their patterns, we can then average them together without introducing significant blur. This is the principle of the smoothing procedure used on the vertical sections.

In our case each vertical section selected earlier was replaced by the average of itself and the two (shifted) columns located five pixels away from it on either side. Five pixels was found to be about half the gap between the selected vertical sections, or a little less than half. Thus, each section encompasses information over a window 10 pixels in width, and these windows collectively cover most of the lung field. The averaged columns are far enough apart so that if one of them passes through some local irregularity the other two probably do not, and its impact will be diminished. Yet because they all pass through the same posterior ribs, they reinforce each other in this regard. Hence the effects of noise and irregularities are reduced, information loss is reduced, and the sections are smoother. Clearly these characteristics can be amplified further by averaging more sections together, with possibly different spacings; the price paid is an increased workload.

Shifting the columns to align their patterns is accomplished as follows. Taking each of the two outer columns separately, they are shifted one row at a time relative

136 PETER DE SOUZA

to the center column until the “difference” between it and the center column reaches a local minimum. The “difference” is defined as

difference = c Ix, - y,+,Vl (2)

where x denotes the central column, y denotes the outer column, s denotes the size of the shift, and the sum is computed over the middle half of the central column.

Fig. 3c shows the result of smoothing the section in Fig. 3a in this manner. It is appreciably smoother than the original and there are fewer irregularities apparent. For example, the region between the third and fourth ribs in the original (counting from the left), which is untypical due to an intruding anterior rib, is now much more normal. Furthermore, the rib edges are not significantly degraded.

Before the ribs are located on the vertical sections, it is necessary first of all to find the outer edge of the lung. This is because only the parts of the vertical sections which lie inside the lung are relevant for rib detection, and the lung boundary enables these parts to be established.

5. LOCATING THE OUTER EDGE OF THE LUNG

The lung edge is located first of all in the middle row of the radiograph and is then tracked upwards and downwards one row at a time from this point. During tracking, the search for the lung edge is limited to a window of 2p + 1 pixels centered on the location of the lung edge in the previous row. In our case p = 5. The search for the first lung edge point is limited to a window of lop + 1 pixels centered on the first vertical section as defined in Section 4.

Within each search window the horizontal profile of the radiograph is smoothed using a rectangular moving average of five pixels before performing the search. The location of the first lung edge point is taken as the position of the smallest local minimum in the window. Thereafter, the lung edge location is taken as the local minimum nearest the center of the window. If no minimum exists, the location is taken as the column containing the smallest value in the window.

50 100 I50 200 250 300 350 400 225- , I , , , , ,.. 175 -

50 100 150 200 250 300 350 400

50 100 150 200 250 300 350 400

FIG. 3. Two vertical sections (a) and (b) taken through the right lung field of Fig. 1, and the result (c) of smoothing the top section.


While the lung is being tracked upwards, the lung edge location is modified at each step to be the maximum of the computed location and the location on the previous row. This reflects the fact that the lung edge bends inwards towards the top, and also prevents it from being sidetracked when it meets the clavicle. A similar constraint cannot be applied to the downward tracking as the lung edge can bend both outwards and inwards in this direction.

After tracking, the lung edge is smoothed using a rectangular moving average which passes along its length.

The computed lung edge for the right lung of Fig. 1 is shown in Fig. 4. The important region of the boundary is correct: the side of the lung. The upper part of the boundary forms a useful upper bound for the top of the lung, but does not

250

25 75 125 175

FIG. 4. Computed boundary for the right lung of Fig. I.

138 PETER DE SOUZA

actually adhere to it. The bottom part of the computed boundary (below the lung) tends to wander somewhat randomly on the whole, but it is of no consequence since we are ultimately only interested in the actual lung itself.

Alternative methods for determining the lung boundary have been proposed by Fu et al. [4] and Toriwaki et al. [6]. Their methods are very similar to each other and involve the identification of a set of feature points around the lung. The boundary is then approximated using piecewise linear segments and can be further refined if required. The results of this method are clearly dependent upon accurate identification of the feature points, and it is felt that this is by no means a trivial task, especially when it is appreciated how hard it is to identify large structures such as the ribs.

Let us now consider the task of identifying the ribs from the vertical sections.

6. LOCATING THE RIBS

As mentioned in the Introduction, it was decided to try to simulate some kind of parallel processing to detect the ribs across a broad front instantaneously. The way in which this is achieved will now be described.

It was argued in Section 4 that vertical sections through the lung all display approximately the same rib pattern as a consequence of the posterior ribs being roughly parallel. This being so, we can average a set of vertical sections together-shifted as necessary-and still retain a coherent picture of the ribs. If this technique is applied to all the vertical sections created in Section 4 then a single one-dimensional profile will be obtained which encompasses information derived from a broad front across the lung. The ribs can then be located on this profile in a single pass and the results related back to the sections which were combined to create the profile. This is equivalent, in effect, to locating the ribs on all the sections simultaneously. The details of the procedure are as follows.

Starting with the middle vertical section, a mean section is formed by merging the other sections with it two at a time working from the center outwards, using much the same technique as described in Section 4. The one slight difference is that only those elements lying inside the lung, as defined by the computed boundary, are averaged together. At any given stage, the elements of the mean section will not all have been averaged over the same number of individual elements. Therefore a tally of the number of contributing elements must be maintained so that the appropriate weights can be given to the mean section every time it is updated.

Figure 5A shows the mean section obtained in this manner for the right lung of Fig. 1. It is an extremely smooth profile with seven clear rib troughs. The only feature of the profile which is slightly confusing is due to the seventh rib trough and the trough caused by the top of the rib cage which are so close together that they are almost coincident.

The ribs are detected on the profile using a sliding t test which is discussed in detail by de Souza [14]. A sliding window of length 2n + 1 pixels is defined, where in our case n = 10. Within this window a t test is performed which compares the first n pixels with the last n, and assigns the result to the location of the middle pixel. As the window slides along the profile, the t test undergoes turning points in the vicinity of riblike edges. There is an implicit assumption in doing this that both the ribs and the intercostal spaces are at least n pixels in width, and n should be chosen


Cb) 0 -2.5

-5 -75 -10

50 100 150 200 250 300 350 400

FIG. 5. Profile (a) obtained by averaging all the vertical sections, and the result (b) of applying a sliding t test to the profile.

accordingly. The window slides along the whole of the profile except that portion above the lung edge (as computed for the m iddle vertical section).

The l test can be defined as follows. Let x,, x2,. . . , x, denote the values of the first n pixels in the window, and let y,, yZ,. . . , y, denote the last n. Then t is given by

z= (y-x)/s (3)

where x and p are the means of the {x,} and {r,}, respectively, and

s2 = (xx; - nx2 + CyJ - nJ2)/n(n - 1). I (4

In some cases s2 can take very small or even zero values, thus distorting t. To avoid this, we imposed an arbitrary lower lim it of 10 on s2.

The result of applying the sliding t test to the profile in Fig. 5a is shown in Fig. 5b. It has well-defined positive and negative peaks in the vicinity of most rib edges. Positive peaks represent upper edges and negative peaks represent lower edges. Although there are, in fact, peaks separating the seventh rib from the top of the rib cage, they are not so pronounced as the others. This reflects the lack of clarity in this part of the profile.

The edges on the profile are identified by a simple peak-picking routine which is applied to the t sequence. Essentially the routine identifies significant local maxima and m inima which are then interpreted as riblike edges. “Significant” turning points are defined so that maxima and m inima alternate and each differs from its two adjacent turning points (of opposite type) by an amount which exceeds a threshold 6. In our case 6 = 4. Proceeding from left to right, the routine locates a significant maximum first, and then identifies m inima and maxima alternately, ending with a m inimum. Since the first maximum cannot be compared with a preceding m inimum to establish its “significance,” it is compared with the lowest value of the sequence up to that point. Similar remarks apply to the last significant turning point also.

The located riblike edges are then checked for obvious errors. The basis of the error checking lies in the empirical observation that the gaps between rib edges of

140 PETER DE SOUZA

the same type decrease by a factor of about 0.9 from one rib to the next going from the bottom rib to the top. A tolerance of kO.4 is allowed in this ratio, and an error is assumed if any consecutive edges of the same type do not satisfy this condition. This allows a maximum ratio of 1.3 which is consistent with Persoon’s observations 151.

In practice, the ribs in the middle of the lung are reliably detected. When errors do occur, they most frequently arise in the top few ribs where the clavicle may cause confusion, and below the bottom rib where the diaphragm may do the same.

Suppose that 2n edges have been detected. Then they are scanned from the nth edge to the bottom edge looking for errors as described above. If an unlikely gap is found it is assumed to lie below the bottom rib, and all the edges below this gap are deleted.

The other edges are then checked in an upwards direction. When an error is found, the edge location is deleted and replaced as follows. The 1 sequence is examined to see if there is a turning point of the appropriate type which satisfies the gap test. It is possible that a turning point was previously ignored as being “insignificant.” If one is found and it satisfies a less severe test of significance then it is assumed to be the correct location of the edge. Otherwise the edge location is estimated so that the gap between it and the previous edge of the same type is 0.9 times the previous such gap. For a local maximum to pass the lesser test of significance, the t sequence must drop by more than 6/4 on each side of the maximum within a neighborhood of p pixels, where in our case p = 5. A local minimum must satisfy an equivalent condition. This definition of “significance” is intended to prevent irrelevant wiggles being identified as rib edges.

Since a total of 16 rib edges are ultimately located, as explained in Section 2, a minimum of 16 edge points is determined at this stage. If fewer than 16 have been found, additional ones are forced into existence using the ratio of the rib edge gaps as described in the previous paragraph.

Having identified riblike edges on the mean profile, their corresponding positions on the individual sections can easily be estimated. The middle section is not shifted relative to the profile and therefore the rib edge locations on the middle section should be approximately equal to those on the profile. The other sections are all shifted relative to the middle section prior to being averaged together, but since these shifts are known it is a trivial matter to shift the rib edge locations by the same amount and hence obtain the approximate rib edge locations on the unshifted sections. Fig. 6 shows these approximate locations for the right lung of Fig. 1. There are 11 sections altogether and on each one the estimated positions of the 16 rib edges are displayed as dots.

The dots do not fall exactly on the rib edges, since the ribs are not exactly parallel. The next step is to improve these estimated positions. This is done by searching in the neighborhood of the dots for a riblike edge. The mechanics of the search are as follows.

The same sliding t test as described above is applied to the vertical sections obtained earlier. The resulting t sequences will be referred to as “t waves.” Each t wave has positive peaks in the vicinity of upper edges and negative peaks near lower edges as in Fig. 5b, although the individual t waves would not normally be as smooth as the one in the figure. Each edge is then refined separately, starting with the middle dot.


400

350

25 75 125 175

FIG. 6. Dots showing crude estimates of rib locations derived from the edge points detected on the profile in Fig. 5a.

The m iddle t wave is searched in the vicinity of the m iddle dot for a turning point of the appropriate type. The search is lim ited to a window of 2n + 1 pixels centered on the dot, where in our case n = 10. This constraint is applied to prevent the dot being moved onto an adjacent rib by m istake. If a turning point is not found, the search is abandoned and the dot deleted. Otherwise the dot is moved to the location of the turning point. Assuming that the search is successful, the other dots are processed from the center outwards. At each step the corresponding t wave is searched for the nearest appropriate turning point to the estimated position of the rib edge. No window is applied to these searches. The estimated position of the rib

142 PETER DE SOUZA

edge is taken as p + E where p is the position of the dot, and E is computed from the previously processed adjacent section as

E = final dot position - initial dot position. (5)

The dot is then moved to the location of the turning point unless it lies outside the lung as defined by the computed lung boundary in which case the dot is deleted.

If the initial search on the middle section is unsuccessful, a constrained search is applied to both the adjacent sections using a slightly larger window. If either of these searches is unsuccessful the procedure is repeated on the next adjacent section, and so on until the search is successful or all sections have been processed. At each step the window is increased slightly (in our case by two pixels) to reflect the fact that the dots are most accurate in the middle and tend to deviate increasingly the further they are from the middle.

Figure 7 shows the results of applying this procedure to Fig. 6. On the whole, the dots are much more tightly positioned although some notable errors remain. The bottom edge is aligned along the diaphragm instead of the bottom rib-a common error. The clavicle has distorted the top two edges, and has indirectly caused the last four dots of the top edge to be placed on the rib below where they are coincident with the dots for that rib-an error that will be detected later. Other dots are misplaced due to the interference of blood vessels, anterior ribs, noise, etc. Nonethe- less, the dots provide a good first approximation to the position of the ribs and it now remains to identify and correct as many errors as possible.

7. ERROR DETECTION AND CORRECTION

The rib edge dots are first checked to see if they are strictly monotonic on each section. That is to say, if the edges are numbered from 1 upwards starting with the bottom edge, then on any given section the dot belonging to edge i must be higher than the dot for edge i - 1. We shall refer to these dots as dot i and dot i - 1, respectively. The last four dots on the 16th edge in Fig. 7 fail this test as they lie below the corresponding dots of the 15th edge.

When two consecutive dots are found which fail the check, an attempt is made to decide which of the two is in error, and it is then corrected if possible. In practice, the higher the edge the less reliable its dots tend to be, and the error detection procedure partly reflects this. Suppose that dot i is not higher than dot i - 1. Then if dot i is also not higher than dot i - 2 it is assumed that dot i is the one in error. Otherwise both dots are compared with their original positions, as shown in Fig. 6, and the one which has been moved furthest is assumed to be wrong.

Having decided which dot is incorrect, an attempt is made to correct it by searching along the appropriate t wave for a turning point of the right type. The search is limited to the interval between the dot on the edge above and the dot on the edge below, where these exist, and is also limited to the interior of the lung as defined by the computed lung boundary. If the search is successful then the incorrect dot is replaced; otherwise it is deleted.

Next, the edges are checked to see if they have a certain minimum number of dots. Edges with only a few dots tend to be spurious. Ribs are comparatively large objects traversing the greater part of the width of the lung and should have dots on most of the vertical sections. The top and bottom edges are not paired with other edges and are treated separately; the others occur in pairs (both edges of a rib) and are treated


25 75 125 175

FIG. 7. Estimates of rib edge points after moving each dot in Fig. 6 to a riblike edge point

as such. If the bottom edge has less than three dots the bottom two edges are deleted repeatedly until this is no longer true. Similarly the top two edges are deleted until the top edge has at least three dots. Note that it is necessary to delete the edges two at a time to ensure that the bottom edge remains the upper edge of a rib, and the top edge remains a lower edge. Any pair of edges with fewer than six dots between them is also deleted. Furthermore, any pair of edges is deleted if one edge has no dots.

At this stage an attempt is made to locate every m issing dot. Wherever one is noted the corresponding t wave is searched for a turning point as described above. The search is again lim ited by the dots above and below the m issing dot, and by the lung boundary.

The eventual intention is to fit curves through the dots describing the rib edges. It is desirable, therefore, to have as many dots as possible to ensure that the fitted

144 PETER DE SOUZA

curves are as accurate as possible. The edges which occur in pairs usually have a sufficient number of dots between them for this purpose. But the top and bottom edges are not paired with other edges and so it is more critical in these cases that as many dots as possible be located. The bottom edge intersects all the vertical sections and a dot is normally located on most, if not all of them. However, the top edge may only intersect with a few sections especially in those cases where there is a marked narrowing of the rib cage towards the top. For this reason the top edge is extended by merging it with the computed lung boundary; this is reasonable since they are one and the same thing at the top of the lung.

The dot on the top edge furthest from the vertical center line of the radiograph is located. If this dot is not in the outermost vertical section, a dot is placed in the outer section next to it at the point where it crosses the lung boundary. This is repeated until the outermost section is reached, or until the vertical displacement between successive dots exceeds a threshold. This restriction prevents the top edge being extended all the way down the lung edge. The threshold is set dynamically to be twice the gap between the vertical sections, or about 20 in our case. This is a somewhat arbitrary threshold and a fixed threshold would probably work equally well for radiographs of fixed size.

Figure 8 shows the dot positions at this stage of the procedure. The top edge has been substantially corrected and has also been extended out in the direction of the lung boundary. Many of the errors visible in Fig. 7 remain, however, and further checks are required.

First of all, the program checks that there are precisely 16 edges. If there are any other number, the program terminates. Otherwise they are numbered from 1 to 16 going from the bottom upwards.

Edges 2 to 15 occur in pairs and each of these pairs is edited as follows. On each section the paired dots are isolated and the midpoint between each pair is computed. Taking each pair in turn, the lower dot is moved to the location between it and the midpoint where the corresponding t wave takes its lowest value. Similarly the upper dot is moved to coincide with the highest value. The effect of this is to leave correct dots alone, and to correct some of the dots which are caught up on minor edges due to blood vessels etc.

The dots on edge 16 are edited in a similar way. For each section they are moved to the location where the t wave takes its lowest value between dots 16 and 15.

The bottom edge requires major editing since it is commonly confused with the diaphragm, as in Fig. 8. In practice it proved to be more convenient to completely reestimate this edge rather than correct it. This is done in the following way. An initial approximation to the edge is made using edge 2 as a template. The average gap between the dots on edges 3 and 4 is computed and divided by 0.9 to give an estimate of the gap between edges 1 and 2. The dots on edge 1 are then replaced by dots this distance below the dots on edge 2.

This approximation to edge 1 is then improved. All the dots are moved simultaneously one pixel at a time until a goodness criterion reaches a local maximum. The goodness criterion is just the sum of the t wave values evaluated at the location of the dots.

Having refined the edge as a whole, the dots are then refined individually. For each dot, a search is made on the corresponding t wave for the nearest significant local maximum. The search is limited to a window of 2n + 1 pixels centered on the


250

25 75 125 175

FIG. 8. Estimates of rib edge points after correction for monotonicity.

dot, where in our case n = 20. If no maximum is found the point is deleted. The criterion for being “significant” is the same as the “lesser test of significance” described in Section 6.

The three dots on the bottom edge nearest the center of the radiograph often lie on top of the heart where the rib edge may be almost undetectable. Even when the edge is visible, it may be so faint that it fails the test of significance. Therefore the search is abandoned in this region if the first encountered local maximum proves insignificant.

The results of this editing can be seen in Fig. 9. The bottom edge is now largely correct, and the other edges are the same as in Fig. 8. Some dots are still removed

146 PETER DE SOUZA

400

350

300

250

200

150

100

50

FIG. 9. E: itimates

25 75 125 175

of rib edge points showing correction of the bottom c :dge.

from rib edges and we will now consider how they can be recognized and possibly corrected.

8. DETECTION AND CORRECTION OF OUTLIERS

The principle of this procedure is simple. Quadratic curves are fitted through the rib edge dots. These curves closely adhere to the rib edges except near the outside of the lung. Consequently those dots which do not actually he on a rib edge tend to he further away from the fitted curves than do correct dots, and therefore can easily be recognized. An attempt can then be made to correct the outlying dots by searching in the vicinity of the fitted curve for a rib edge.


The curves fitted to edges 1 and 16 are of the form

y = ax2 + bx + c (6)

where y denotes the row in which a dot lies, x denotes the section it lies on, and the sections are numbered from one upwards starting with the section nearest the outside of the lung. Edges 2 to 15 occur in pairs and a curve of the following form is fitted to each pair:

y = ax2 + bx + c + 6 (7)

where 6 is zero for the lower edge of the pair and nonzero for the upper edge. In essence, 6 is the width of the rib, and Eq. (7) describes two parallel quadratics spaced 6 pixels apart.

The curves are fitted by weighted least squares, i.e., so as to minimize the weighted sum of squares

s = Cw,(y, - j;)2. 63)

Here We is a weighting factor and pi is the predicted value of y, using the fitted curve. The weights {wi} are chosen as follows. Where the model (7) is being fitted, the

weights are largely determined by the vertical distance between pairs of dots lying on the same vertical section. Assume for the moment that the width of a rib is known. Then it is clear that dots which are separated by this width are more likely to be correct than dots which are separated by a greater or lesser distance. Where dots are not separated by the correct distance it may be that only one of the dots is in error, but since it is difficult to state which it is, it is convenient to treat both dots as unreliable and give them less weight in (8). Let R denote an estimate of the width of a rib, and let r denote the gap between two dots. Then the weight w given to the two dots is given by

w = min(R, r)/max(R, r). (9)

Since the actual rib widths are not known they are estimated as follows. The curves are fitted to each rib in turn going from the bottom to the top, except the very bottom edge which is done last. Thus except for edges 2 and 3, the width of a rib R can be estimated from the parameter 6 obtained on the rib below. In the case of the first pair of edges (2 and 3), R is taken as the average distance between the dots computed over the three ribs described by edges 2 to 7.

Where one of a pair of dots is missing, this method cannot be used to weight the remaining dot. Instead it is given the average weight of all the weighted dots.

Having weighted the dots on both edges of a rib, the dots on each edge have their weights multiplied by the number of dots on the edge. Often this makes no difference to the relative weights as the number of dots is frequently the same on each edge. However, when one edge is unclear or partly obscured this is often reflected in an increased number of missing dots and poorer accuracy in the remaining dots. In these cases, it makes sense to give the more complete edge greater weight. A typical case is that of the clavicle having an adverse effect on the upper edge of a rib whilst leaving the lower edge unaffected.

148 PETER DE SOUZA

The dots on edges 1 and 16 cannot be weighted in this way. No formula was found for weighting the dots on the bottom edge which improved over equal weights. Therefore they are all just given equal weights of 1. The top edge is often affected by the clavicle and it is advisable to try to detect unlikely dot positions and weight them accordingly. Because the edges are smooth curves, a large vertical displacement between adjacent dots usually implies that at least one of the dots is wrong, and this can be used to give the dots meaningful weights.

For each dot on the top edge, the maximum vertical displacement d between it and the dots on the two neighboring sections is computed. Essentially the dot is then given a weight of l/d. However this leads to differences in weight which are too gross, and fails completely if d = 0. To avoid this, the range of d is restricted by setting a lower limit of 3 on it and an upper limit of 10. If a dot has no neighbors it is given the minimum weight of l/10.

After a quadratic curve has been fitted to an edge or pair of edges, a check is performed to ensure that it looks reasonable, i.e., approximately rib-shaped. The degree of convexity of the curves is determined by a, the coefficient of x2, in (6) and (7) and for posterior ribs this should always be negative. It is possible to place more precise restrictions on the coefficient than this but the value of a is dependent upon the resolution of the digitized radiograph and also the number of vertical sections taken through the lung. Therefore simple global limits cannot be quoted, but must be calculated to suit the environment. In essence, the coefficient should always lie in a restricted range (< -0.2 in our case) and should not differ much from one rib to the next (not more than 0.2 in our case). The latter constraint is particularly useful for detecting errors in the upper ribs where the dots are much less reliable than those on the lower ribs.

Where necessary, the quadratic curves are recomputed so that Q is forced to take an acceptable value. In our case when an error in the coefficient is detected its value is computed from an adjacent rib. Let a, denote this value and let z be defined as

z = y - a$. (10)

The remaining parameters of the quadratic curves are then computed using the linear models

z=bx+c (11)

z=bx+c+6 (12)

instead of (6) and (7), respectively. The models are again fitted by the method of least squares, but on this occasion the weights are ignored, since they do not appear to have helped.

After an acceptable curve has been fitted to the edge or edges in question, the residuals are calculated, i.e., the vertical gaps between the dots and the curve. The largest residual is located and an attempt is made to correct the corresponding dot by searching along the t waves it lies on for a turning point of the right type in the vicinity of the curve. If one is found which is nearer the curve than the dot, then the


dot is moved to that location regardless of the “significance” of the turning point. If a turning point is not found and the residual exceeds a threshold, it is assumed to be wrong and is deleted. In our case the threshold is set at four pixels. Every time a dot is moved or deleted, the weights are recomputed and the curve refitted. If the dot corresponding to the largest residual is neither moved nor deleted then the next largest residual is located and treated in the same way. This is continued until all the residuals have been examined. The procedure stops when no dot is either moved or deleted.

If, as the result of deletions, fewer dots remain than the number of quadratic parameters being fitted, it becomes impossible to fit the curve. In these cases it is

250

25 75 125 175

FIG. IO. Estimates of rib edge points after detection and correction of outliers.

150 PETER DE SOUZA

most likely that the edge being processed is not a rib edge and the program terminates.

The detection of outliers represents the final stage of error detection and correction. Figure 10 shows the results of applying this procedure to the data in Fig. 9. More errors have been either corrected or deleted although a few dots remain which are slightly out of position.

At this stage the dots could be used to guide a routine which tracked the rib edges. For our purposes, however, it is adequate to model the rib edges by composite curves. The computed quadratic curves form part of the composition and the quadratic parameters are retained for this purpose. The remaining parts of the composite curves will now be discussed.

9. DRAWING THE RIB EDGES

The quadratic curves computed in Section 8 provide an adequate description of the rib edges except in the vicinity of the outside edge of the lung. In this region the gradient of a rib edge becomes infinite and cannot be represented by a quadratic of the form (6) or (7). It is necessary, therefore, to fit an alternative curve to this part of the rib.

The curve selected for this purpose is given by

y = 2&(x - b)) + c (13)

where y denotes the row in which a dot lies, x denotes the section number as before, u is greater than zero, and the square root sign denotes the positive square root. This is the equation of a parabola whose focus is at the point (a + b, c), and which passes through the point (b, c) with an infinite gradient.

If b is known, Eq. (13) can easily be transformed into a simple linear equation with two parameters, and can therefore be estimated by least squares in the usual way. Alternatively, the three parameters in (13) can be estimated directly in an iterative manner. For reasons of computational expediency the former approach was adopted. However, it is necessary to estimate b since it is not actually known.

In effect, for a given rib edge, the parameter b is the column number in which the edge becomes vertical. This occurs at the outside edge of the rib cage, or equivalently on the lung boundary. A convenient estimate of b may be obtained, therefore, by finding the column in which the fitted quadratic meets the lung boundary.

Given b, we may define z and d as

z = J(X - b) (14 d = 2Ja (15)

so that (13) may be rewritten as

y = dz + c. (16)

The parameters d and c in (16) may be estimated very easily using least squares, and a may be obtained from (15) to complete the fitting of the parabola.

Parabolas of the form (13) are fitted to the outer portions of each rib edge except the top edge. By this stage, the top edge will already have been artificially extended


to coincide with the lung boundary and no further modeling is required. Edges 2 to 15 occur in pairs and are treated as such. The vertical width of these ribs is known: it is given by the parameter 6 of Section 8. Therefore the upper dots can be lowered by 6 pixels and then pooled with the lower dots to form a single set of points for curve fitting. The resulting parabola describes the lower rib edge, and by adding 6 to the parameter c in (13) a parallel parabola is obtained 6 pixels higher which describes the upper edge.

Since the parabola (13) has a positive gradient for all values of x, the parabolas should not be computed from dots which lie on a rib edge where the gradient is either negative or zero. Therefore dots are included for estimation purposes from the outermost section inwards until the gradient of the rib edge falls below a threshold,

250

200

150

loo

50

25 75 125 175

FIG. 1 I. Right lung of Fig. 1 showing computed rib and lung boundaries.

152 PETER DE SOUZA

subject to there being dots from at least two sections present. The gradient of a rib edge is taken as the gradient of the fitted quadratic after converting it to real coordinates, i.e., the x coordinate is converted from section numbers to column numbers. In our case the gradient threshold is set at 0.4.

The rib edges can then be modeled by the parabolas near the outside edge of the lung, by the quadratics in the center and inner part of the lung, and by an average of the two in between. In our case the limits of the curves are placed on adjacent vertical sections and they are merged together over the interval between them using a weighted average of the two, where at any given pixel the curves are weighted in inverse proportion to the distance from the pixel to the ends of the curves.

The computed lung boundary is then tidied up so that it merges smoothly with the top rib edge, and does not extend below the bottom rib edge. The rib edges are extended outwards until they meet the lung boundary, and inwards as far as the (n + m)th column where n is the location of the innermost vertical section, and m is the spacing between the sections.

Figure 11 shows the computed rib edges and lung boundary for the right lung of Fig. 1.

IO. FINDING THE RIBS IN THE LEFT LUNG

As mentioned in Section 2, the left lung field is reflected about its central column and treated like a right lung field. It is processed after the right lung and capitalizes on some of the information extracted from that lung.

The lung boundary and vertical sections are computed in exactly the same way as before, but it is not necessary to compute the crude approximation to the rib edges by averaging the sections together and locating the rib edges. Instead the dot positions of Fig. 6, as computed for the right lung, are just superimposed on the left lung. Since only a crude approximation is required at this stage this is an adequate procedure in the majority of cases. The dots are then refined, and the curves fitted in the same way as before.

I I. RESULTS

The program described in the preceding sections has been applied to a small set of eight radiographs. Included among these were radiographs of adult men, adult women, adolescents, and children. Six of the radiographs were used to develop the algorithm, although for most of the development phase only the radiograph in Fig. 1 was used. The others helped to uncover weaknesses in the algorithm and ultimately led to improvements being made.

All the radiographs were digitized to an accuracy of eight bits on a Joyce-Loebl Scandig 3 microdensitometer which can digitize films up to about 10 X 14 inches. Two of the radiographs were of small children and were small enough to be digitized directly. The other six were too large and were photographically reduced by a factor of four onto negatives which were then digitized at a resolution of 200 CL. This is equivalent to having digitized the originals at a resolution of 800 p. A significant loss of contrast occurred in copying the radiographs in this way, making rib detection even more difficult than would otherwise have been the case.

Two radiographs were not used for algorithm development, but were used instead to test it. This is an important step since an algorithm which performs correctly on the data from which it has been developed but on no other data is practically useless.


It is imperative that acceptable results can also be demonstrated on entirely independent data. The two extra radiographs provided the opportunity to do this.

Satisfactory results were obtained in all eight cases. Figures 12 to 14 show the computed boundaries for three of them, the first of these being the radiograph of Fig. 1. Figures 12 and 13 are the two children’s radiographs which were digitized directly without photographic reduction. Note that they have chests of very different shape yet both are treated equally successfully by the program. Note also that there is a pocket of bowel gas below the left lung in Fig. 13a which combines with the diaphragm and lower ribs to create a confusing picture in this area. Fig. 14 shows one of the two test radiographs which played no role whatever in the development of the algorithm. The ribs are not as clear as they are on the original radiograph as a consequence of the photographic reduction which took place prior to digitization. The results do not appear to be any worse than those shown in the previous two figures. Furthermore, this is the radiograph of a female with breast shadows superimposed on the ribs. This additional complication has not affected the rib detector.

The results can be claimed to be satisfactory in the sense that the ribs and lung edges have been correctly identified. Close examination, however, shows that the

50 100 150 200 250 300 350

FIG. 12. Computed rib and lung boundaries for radiograph in Fig. 1.

154 PETER DE SOUZA

FIG. 13. Radiograph of a small child (a) and the computed rib and lung boundaries (b).

AUTOMATIC RIB DETECTION

F con

14 .R ed rib

adiograph of a female (a) which was not used during algorithm develc and lung boundaries (b).

:nt, a tnd the

156 PETER DE SOUZA

plotted boundaries do not follow the rib edges precisely; this should be expected with boundaries which are essentially just mathematical conveniences. If “precise” edges are required they could be obtained by invoking a tracking routine which used the displayed boundaries as guide rails as described in the following Section.

Where the ribs overlap with the heart in the left lung, they are frequently indiscemable. The plotted edges are essentially just extrapolations in this region and do not always follow the actual ribs. This is of little consequence with regard to automatic screening, however, as the inner boundary of the left lung is usually taken as the heart boundary. Therefore it is not normally necessary to locate the ribs where they overlap with the heart as they are of little relevance in selecting intercostal spaces for screening.

Most of the rib edges appear as smooth curves despite the fact that they are made up, essentially, of two distinct parts. The one exception is the bottom edge in the left lung of Fig. 13b which has a kink in the middle. This could no doubt be smoothed out, but in fact it follows the rib/diaphragm edge fairly closely, as it is intended to. It can be seen in Fig. 13a that this composite edge is itself not very smooth.

Because of the need to make an intermediate negative in the process of digitizing the radiographs, and the resultant loss of quality in the digitized images, it has not been possible to perform thorough testing of the algorithm beyond the limited analysis described above. Planned acquisition of more sophisticated equipment is intended to relieve this problem and more rigorous experimentation will be possible then. It seems clear, however, that high energy X rays which result in low contrast radiographs are liable to provide a far greater challenge to the rib detector than low energy X rays. In high energy X rays some rib edges become very faint and it is inevitable that the number of missing and deleted rib edge dots will rise in these circumstances. This is liable to lead to incorrect curves being fitted, or the failure to detect entire ribs. The latter case would, of course, cause the program to terminate but the other defect could go undetected.

The areas in which problems are most likely to occur are the bottom edge which can be confused by the diaphragm and by pockets of bowel gas; the top few ribs which are very close together and which can be confused by the clavicle; and the top edge which may be very short and which can also be confused by the clavicle. These were the areas which proved most troublesome during the development of the algorithm. To overcome these difficulties it may be necessary to increase the number of vertical sections taken through the lung, although this will necessarily result in the program running more slowly.

Note that a good approximation to the complete lung boundary is obtained as a by-product of rib detection. The lung is essentially bounded by the top and bottom rib edges, the outside edge of the lung as computed, and the limits of the ribs as plotted. The boundary can be completed by locating the heart and modifying the inner edges of the lungs accordingly.

12. DISCUSSION OF POSTERIOR RIB DETECTION

A key feature of the rib detection program is that it contains no fixed threshold or constant which is specified more precisely than one significant figure. One of the problems in pattern recognition generally is that it is nearly always possible to tune a threshold or find a set of thresholds which enable an ad hoc decision rule to perform 100% correctly on a small training set of data. But there is no guarantee that these


rules and thresholds will work equally well on other data, and frequently they do not.

If it is necessary to specify a threshold to many significant figures to make a rule work, it should be recognized that the rule is possibly being used to decide between two options which have overlapping features or parameters. In this case no threshold exists which will enable the options to be separated with 100% reliability, although if the overlap is small it is usually possible to contrive test data in which no overlap is evident. Such contrivance or fine tuning of thresholds leads to excellent “test” results but the algorithms may be so unreliable in the general case that they are virtually useless.

A more robust approach is to recognize that a decision rule will make a certain proportion of errors, and to devise error detection and correction routines to complement the rule. Under these circumstances the consequences of a decision rule error are not nearly so disastrous as in the previous case, and a considerably more general algorithm results. This is the philosophy that guided the development of the present rib detector which practices error detection and recovery rather than fine tuning of thresholds.

Restricting thresholds to only one significant figure is possibly excessively severe, but it does provide a discipline which ensures that we are not fooling ourselves into thinking that the thresholds are unimportant, and it also reduces the probability of being misled by small amounts of test data into underestimating the true difficulty of the task.

Another important feature of the algorithm is its lack of dependence on conventional edge detection. There are two approaches to rib detection. In the first, rib edges are detected and it can then be deduced that a rib lies between each pair of upper and lower edges. And in the second, the ribs themselves are detected and the edges are therefore obtained implicitly as being the upper and lower limits of the ribs. The emphasis in the former method is firmly on edge detection whereas in the latter method it may not play a role at all.

The difficulty with the edge detection approach is that there are a lot of edges in a chest radiograph, and a large amount of work is required to compute the gradients, determine which edge points belong to ribs and which do not, thin the rib edge points, and link the appropriate points together. This is partly the reason why the programs of Wechsler and Persoon take so long to execute [3].

In contrast, there are relatively few riblike objects in a chest radiograph, and therefore there is less chance of becoming confused when locating ribs rather than rib edges. Additionally, since the approximate size, shape, and orientation of the ribs are known in advance, it was decided to capitalize on this information and to detect the ribs directly. Thus edge detection in general is unimportant here and is not carried out.

Toriwaki et al. [6] also opted for the rib detection approach rather than that of edge detection but they used a fixed linear filter which tacitly assumes that the ribs have a constant, known width. It is not necessary to make such a strong assumption; the ribs can be detected assuming only that they have a certain minimum width, as we have seen.

In some applications it may be considered that the fitted curves are not sufficiently accurate descriptors of the actual rib contours which tend to be irregular rather than mathematical niceties. In these cases the “precise” rib edges may be

158 PETER DE SOUZA

found by searching for the points of maximum gradient in the vicinity of the fitted curves under the constraint that the edges should be continuous. Thus the curves can be used as guide rails to prevent the search from being sidetracked along another edge, and they also provide a “look ahead” facility to overcome tracking problems caused by the fact that in some instances parts of a rib edge disappear.

As was mentioned in the Introduction, the posterior rib detection algorithm is likely to fail in the case of scoliotics. In fact, it is true to say that any illness or disease which either distorts or obscures the ribs significantly is likely to cause the algorithm to fail. Now the algorithm, as described, contains several checks on its own progress, and it is designed to refer radiographs to a radiologist if an error is detected. Thus a likely outcome of processing an abnormal radiograph is that it would simply be referred to a radiologist. If the ribs are being detected as part of a screening process which is designed to detect abnormal radiographs so as to draw them to the attention of a radiologist, then this outcome may be quite acceptable.

If, on the other hand, rib detection is being carried out for some other purpose, it may be essential to locate the ribs even on abnormal radiographs. In this case the described algorithm may prove to be inadequate and an alternative, more robust approach might be required.

It has been suggested [ 1, 41 that increased reliability might be obtained by incorporating a model of the rib cage into the rib detection algorithms. Indeed some attempts have been made to design model-directed rib detectors [ 151 but no decisive results have been demonstrated, nor is it clear what the computational cost of such approaches is likely to be. However, it is an avenue that should be considered if a more general rib detector is required.

13. DETECTING THE ANTERIOR RIBS

An algorithm for detecting anterior ribs on the basis of one-dimensional sections was also derived. Although it gave satisfactory results on the radiographs from which it was developed, it failed to meet the acceptance criterion discussed above: the demonstration of satisfactory results when applied to data independent of the training set. For this reason the approach will only be sketched briefly. Other algorithms have been described by Wechsler et al. [l-3] and Toriwaki et al. [8] but it is not clear from their reports whether they achieved satisfactory results on independent data or only on the data in their training sets.

The posterior ribs show up more clearly on radiographs than do the anterior ribs, and they tend to dominate in most one-dimensional profiles taken through the lung. It is desirable to reduce this dominance when locating the anterior ribs. One way that this might be done is by subtracting out the posterior ribs once they have been located. However, they are not of uniform intensity and it is difficult to do the subtraction reliably. An alternative approach, and the one that was adopted, is to take curved sections “parallel” to the posterior ribs, i.e., running along the center of a rib, or running equally spaced between two posterior ribs. Such “parallel” sections are guaranteed not to cross the posterior ribs and therefore any rib troughs encountered on them must be due to anterior ribs.

In our system two equally spaced sections were taken between each pair of adjacent posterior ribs, and further sections were taken along the middle of each rib. These sections cannot be pooled together as they were in the case of posterior rib detection since they do not display a common rib pattern, nor do they even intersect


with the same anterior ribs. Thus the rib edges must be located on each section individually, and this is a much less reliable process than when working with a global average where noise is effectively suppressed.

The rib edges can be detected on each section using the same sliding t test as before or a sliding r test [14], a variant of the t test which compensates for a gradation in background intensity. In principle, after the edges have been detected on each section, they can be linked together, and can have quadratic curves fitted through them to describe the rib edges. Two edge points on adjacent sections are linked together only if there is evidence of a riblike edge running between them.

Satisfactory results were obtained on several radiographs in this way. Figure 15 shows an experimental result obtained for the radiograph in Fig. 1. Other results, however, were not always as good. The reasons for the lack of reliability can be attributed to two main causes. Firstly, the detection of rib edge points on the individual sections is difficult due to the anterior ribs often being faint, and also because noise and irregularities on the sections can lead to edges being m issed or spurious ones being found. And secondly, establishing whether an edge runs between two candidate edge points is difficult because the points m ight not lie

50 100 150 200 250 300 350

FIG. 15. Experimental result showing computed anterior and posterior rib boundaries for the radiograph in Fig. 1.

160 PETER DE SOUZA

exactly on the rib edges, and also because there is a gradation in background intensity going from the outside of the lung towards the center of the radiograph which makes it harder to detect changes in intensity caused by the ribs.

14. SUMMARY

An algorithm for automatic detection of posterior (dorsal) ribs has been described in detail. Programmed in PL/I for an IBM 3031, it requires about 8 set to process a 400 x 400 digitized radiograph, which is substantially faster than previously pub- lished algorithms. The procedure is entirely automatic requiring no preselection of an area of the radiograph for processing. Its computational efficiency can be attributed to the fact that no two-dimensional image processing operations are performed, as the algorithm is defined in terms of a few one-dimensional sections taken through the image.

No threshold or constant is specified to an accuracy of more than one significant figure, and no fine tuning of the algorithm has been performed. This was deliberate policy aimed at keeping the algorithm general rather than tuning it to a particular training set. Instead, the procedure follows a strategy of error detection and correction which ultimately leads to a more robust program.

The algorithm has been tested on radiographs which were not included in the training set from which it was developed and satisfactory results were obtained.

An approach to the detection of anterior (ventral) ribs has also been sketched, but it has not proven sufficiently reliable to represent a practical proposition in its current form. It did, however, yield acceptable results on a small training set of radiographs.

ACKNOWLEDGMENTS

I thank M. Cocklin, A. Gourlay, P. Jackson, G. Kaye, and A. Pullen for their assistance and for many helpful discussions, and E. de Souza for geometrical advice. I am also grateful to the Biophysics Department of Imperial College, London, for kindly permitting me access to their microdensitometer, and to the Department of Radiology at the Brompton Hospital, London, who provided the radiographs used in this study. Finally, I am indebted to IBM U.K. Ltd. for awarding me the fellowship under which this work was carried out.

REFERENCES 1. H. Wechsler, Automatic Detection of Rib Contours in Chest Radiographs, Birkhauser-Verlag, Basel,

1911. 2. H. Wechsler and J. Sklansky, Finding the rib cage in chest radiographs, Pattern Recognition 9. 1977,

21-30. 3. H. Wechsler and K. S. Fu, Image processing algorithms applied to rib boundary detection in chest

radiographs, Computer Graphics Image Processing 7, 1978, 375-390. 4. K. S. Fu, Y. P. Chien, and E. Persoon, Computer systems for the analysis of chest X-rays. IEEE

Electronics and Aerospace Systems Convention Record, 72A-72P, 1975. 5. E. Persoon, A new edge detection algorithm and its applications in picture processing, Computer

Graphics Image Processing 5, 1976, 425-446. 6. J. Toriwaki, J. Hasegawa, and T. Fukumura, Computer analysis of chest photofluorograms and its

applications to automated screening, Automedica 3, 1980, 63-8 1, 7. J. Toriwaki, Y. Suenaga, T. Negoro, and T. Fukumura, Pattern recognition of chest X-ray

images, Proceedings, First International Joint Conference on Pattern Recognition, pp. 125- 137, 1973.


8. J. Toriwaki, Y. Suenaga, T. Negoro, and T. Fukumura, Pattern recognition of chest X-ray images,

Computer Graphics Image Processing 2, 1913, 252-27 1. 9. J. H. Kulick, T. W. Challis et al., An image processing laboratory for automated screening of chest

X-rays, Proceedings, Third International Joint Conference on Pattern Recognition, pp. 233-237, 1976.

IO. C. M. Brace, J. H. Kulick, and T. W. Challis, Automatic rib detection in chest radiographs, Proceedings of the 1977 International Joint Conference on Artificial Intelligence, pp. 697-698, 1978.

Il. C. C. Li, A. M. Savol et al., Computer analysis of small opacities in coal miners’ chest X-rays, Proceedings, IEEE International Conference on Cybernetics and Society, pp. 422-428, 1977.

12. C. P. Fang, C. C. Li et al., Automated extraction of subimages in lung regions of chest X-rays for detection of pneumoconiosis opacities, Proceedings, Fourth International Joint Conference on Pattern Recognition, pp. 919-924, 1978.

13. C. P. Fang, C. C. Li et al., Extraction of dorsal rib contours in chest radiographs by parallel-type processing, Proceedings of the International Computer Symposium, Taipei, Vol. I, pp. 705-7 12, 1978.

14. P. de Souza, Edge detection using sliding statistical tests, Computer Vision, Graphics, and Image Processing, 23, 1983, l-14.

15. D. H. Ballard, Model-directed detection of ribs in chest radiographs, Proceedings, Fourth Intema- tional Joint Conference on Pattern Recognition, pp. 907-910, 1978.

automatic rib detection in chest radiographs

Documents