Multiple Genetic Snakes for Bone Segmentation

Lucia Ballerini and Leonardo Bocchi

1 Dept. of Technology, Orebro UniversityFakultetsgatan 1, 70182 Orebro, Sweden

lucia@aass.oru.se2 Dept. of Electronics and Telecommunications, University of Florence

Via S.Marta 3, 50139 Firenze, Italyleo@asp.det.unifi.it

Abstract. Clinical assessment of skeletal age is a frequent, but yet dif-ficult and time-consuming task. Automatic methods which estimate theskeletal age from a hand radiogram are currently being studied. Thiswork presents a method to segment each bone complex in the radio-gram, using a modified active contour approach. Each bone is modelledby an independent contour, while neighbouring contours are coupled byan elastic force. The optimization of the contour is done using a geneticalgorithm. Experimental results, carried out on a portion of the wholeradiogram, show that coupling of deformable contours with genetic op-timization allows to obtain an accurate segmentation.

1 Introduction

Bone age assessment is a procedure frequently performed in pediatric radiology.A discrepancy between bone age and chronological age indicates the presenceof some abnormality in skeletal growth. The assessment of bone age is almostuniversally performed by examination of the left-hand radiogram. This procedurerequires only a minimal exposure, with an high degree of simplicity. Moreover,the hand presents a large number of ossification centers which can be analyzedin order to obtain an accurate evaluation of the skeletal age.

In contrast with such advantages, inspection of the resulting radiogram is aquite complex task. A correct evaluation of the degree of maturation of the bones,requires an high degree of expertise. Several methods have been proposed toperform this evaluation. The most commonly used method is the atlas matchingmethod by Greulich and Pyle [1]. The hand radiogram is visually compared witha series of images reproduced in the atlas, grouped by age and sex. The patternwhich appears to be the most similar to the clinical image is selected, and thecorresponding age is indicated to assess the skeletal age. The major drawbackin this method is its subjectivity, which produces an high degree of variabilityin the outcoming results, both inter-observer, and intra-observer.

A more complex approach uses the Tanner and Whitehouse (TW2) method [2].This method involves a detailed analysis of a group of about 20 bones of handand wrist. Each bone complex is assigned to one of eight classes reflecting the

various development stages, depending on the degree of calcification and theshape of the complex. In this way, a maturation score is assigned to each bone.A weighted sum of all scores is the used to evaluate the skeletal age. This methodyields the most reliable results, but, due to its complexity, it does not present ahigh application rate (less than 20%).

Several research groups are working to develop automatic methods which canspeed up the evaluation process. A complete method can roughly be subdividedin a segmentation step, where the bones are identified and labelled, a feature andshape analysis step which assesses the bone age of each region, and a classificationstage which summarizes the partial data to produce the final age assessment.

In this work, we focus on the first stage, the segmentation procedure. Thesegmentation stage presents an high degree of complexity due to several factors;among those, we face with the presence of several overlapping regions of interest(ROI), the presence of ROI having completely different degree of calcificationand overlapping of soft tissue.

We propose the use of Genetic Snakes [3], that are active contour models, alsoknown as snakes [4], with an energy minimization procedure based on GeneticAlgorithms (GA) [5]. Snakes optimization through Genetic Algorithms provedto be particularly useful in order to overcome problems of the classical snakesrelated to initialization, parameter selection and local minima. New internal andexternal energy functionals have been proposed in our previous works and theyhave been successfully applied to a variety of images from different domains.

The purpose of this paper is to extend the Genetic Snakes model to handlecomplex contours, composed of distinct regions, allowing introduction of ex-ternal knowledge expressed by additional energy terms. Each bone contour isassociated to an independent snake, while the anatomical knowledge about rel-ative placement of hand bones is modelled by means of a binding energy whichcouples together the contours.

The organization of the paper is as follow: in Section 2 we briefly review activecontours, the basic notions, their limitations and some improvements proposedin literature. In Section 3 we describe the Genetic Snakes model. In Section 4we extend our previous formulation by the introduction of the multiple snakesstructure and of the binding energy. Experimental results are reported in Sec-tion 5.

2 Active Contours

Snakes are planar deformable contours that are useful in several image analysistasks. They are often used to approximate the locations and shapes of objectboundaries on the basis of the reasonable assumption that boundaries are piece-wise continuous or smooth.

Representing the position of a snake parametrically by v(s) = (x(s), y(s))with s ∈ [0, 1], its energy can be written as:

Esnake =∫ 1

0

Eint [v(s)] ds +∫ 1

0

Eext [v(s)] ds (1)

where Eint represents the internal energy of the snake due to bending and it isassociated with a priori constraints, Eext is an external potential energy whichdepends on the image and accounts for a posteriori information. The final shapeof the contour corresponds to the minimum of this energy.

In the original technique of Kass et al. [4] the internal energy is defined as:

Eint [v(s)] =12

[α(s)

∣∣∣∣∂v(s)∂s

∣∣∣∣2 + β(s)∣∣∣∣∂2v(s)

∂s2

∣∣∣∣2]

. (2)

This energy is composed of a first order term controlled by α(s) and a secondorder term controlled by β(s). The two parameters α(s) and β(s) dictate thesimulated physical characteristics of the contour: α(s) controls the tension of thecontour while β(s) controls its rigidity.

The external energy couples the snake to the image. It is defined as a scalarpotential function whose local minima coincide with intensity extrema, edges,and other image features of interest. The external energy, which is commonlyused to attract the snake towards edges, is defined as:

Eext [v(s)] = −γ|∇Gσ ∗ I(x, y)|2 (3)

where I(x, y) is the image intensity, Gσ is a Gaussian of standard deviation σ,∇ is the gradient operator and γ a weight associated with image energies. Dueto the wide and successful application of deformable models, there exist surveypapers focusing on different aspects of the model and its variants proposed inthe literature [6–9].

The application of snakes and other similar deformable contour models tosegment structures is, however, not without limitations. For example, snakeswere designed as interactive models. In non-interactive applications, they mustbe initialized close to the structure of interest to guarantee good performance.The internal energy constraints of snakes can limit their geometric flexibility andprevent a snake from representing long tube-like shapes or shapes with signifi-cant protrusions or bifurcations. Furthermore, the topology of the structure ofinterest must be known in advance since classical deformable contour models areparametric and are incapable of topological transformations without additionalmachinery. Due to its own internal energy, the snake tends to shrink in caseof lack of image forces, i.e. constant image backgrounds or disconnected objectboundaries, and not to move towards the object.

Various methods have been proposed to improve and further automate thedeformable contour segmentation process. See the above mentioned surveys fora review of some of them.

As concerns the energy minimization, the original model employs the vari-ational calculus to iteratively minimize the energy. There may be a number ofproblems associated with this approach such as algorithm initialization, exis-tence of local minima, and selection of model parameters. Simulated anneal-ing [10, 11], dynamic programming [12, 13] and greedy algorithm [14, 15] havebeen also proposed for minimization. However they are restricted either by the

exhaustive searches of the admissible solutions either by the required accurateinitialization.

Some authors propose the application of GA to active contours. Among those,MacEachern and Manku [16] introduce the concept of active contour state andencode the variants of the state in the chromosome of the genetic algorithm.Tanatipanond and Covavisaruch [17] apply GA to contour optimization witha multiscale approach. The fitness function is “trained” from a previously seg-mented contour. Ooi and Liatsis [18] propose the use of co-evolutionary geneticalgorithms. They decompose the contour into subcontours and optimize eachsubcontour by separate GA working in parallel and co-operating. However, inthese approaches the optimization is done in the neighborhood of the snakecontrol points.

In other existing GA-based active contours the optimization is done indi-rectly, i.e. optimizing the parameters of the contour such as encoding the poly-gon [19], Point Distribution Models [20, 21], Fourier Descriptors [22], ProbabilityDensity Functions [23] or edge detector and elastic model parameters [24].

3 Genetic Snakes

In this section we review the genetic snake model, i.e. our model of activecontours, where the energy minimization procedure is based on genetic algo-rithms [3].

The parameters that undergo genetic optimization are the positions of thesnake in the image plane vi = (xi, yi), for i = 0, ...N where N is the total num-ber of snake points. To simplify the implementation we used polar coordinatesvi = (ri, θi) with the origin in the center of the contour. Actually, this pointmust lie inside the object, but its position may be arbitrary. The magnitudesri are codified in the chromosomes, while θi = 2πi/N . The polar representationintroduces ordering of the contour points and prevents the snake elements fromcrossing each other during evolution. The genetic operators can be implementedstraightforward on this representation, no additional check is required to ensurethat mutation and crossover produce valid individuals.

The fitness function is the total snake energy as previously defined in (1),where Eint and Eext are defined in (2) and (3).

The initial population is randomly chosen in a region of interest defined by theuser, and each solution lies in this region. This replaces the original initializationwith a region-based version, enabling a robust solution to be found by searchingthe region for a global solution. The region of interest can be the image itself, sothe solution can be searched in the whole image, making the initialization fullyautomatic.

An accurate description of implementation details along with a discussion onthe choice of the model coefficients can be found in [3].

The genetic search strategy works against constant image background andovercomes difficulties related to spurious edge-points that can drive the snake toa local minima. To reach an optimal minimum while avoiding local minima, some

approaches suggest to consider the whole set of admissible curves and choose thebest one.

Other snake optimization methods are for local optimization, where only sub-optimal solutions can be guaranteed. The genetic algorithms are particularlyuseful in simultaneously handling possible solutions and looking for a globalminimum, while avoiding an exhaustive search.

4 Multiple Genetic Snakes

A straightforward extension from a single contour to multiple contours posesa few questions which could prevent convergence. First of all, the spatial rela-tions between facing bones produce sets of parallel edges. Each contour needto be univocally associated to the correct edge to achieve a correct segmenta-tion, while (3) does not allow to discriminate between the edges, because it issensitive to the gradient modulus, but not to its orientation. We solve this prob-lem with the introduction of a first-order derivative energy, which introduces adirectionality in the contours.

iv

vright

leftv

∆

∆

vi−1

vi+1

Fig. 1. Points used in the evaluation of derivative energy

For each point vi belonging to the snake (see Fig. 1), we define vleft =(xleft , yleft) and vright = (xright , yright), which are placed on a line orthogonal tovi−1vi, and spaced of a small distance ∆ from vi. To allow a faster computation,the orientation of the line has been constrained to be multiple of π/4, and thedistance ∆ is assumed to be one pixel. In this way, the derivative energy is com-puted as the difference between two pixels belonging to the 8-neighbourhood ofvi, selected accordingly to the direction of the snake in that point. The derivativeenergy used can then be expressed as:

Eder[vi] = δ [(F ∗ I)(xleft , yleft)− (F ∗ I)(xright , yright)] (4)

where δ is a weight used to balance the derivative energy with the other termson external energy and F is a smoothing filter. This definition introduces anenergy term that presents a minimum point when the snake is positioned onthe image edge, having the brighter region on the left side of the snake, and

the darker region on the right side. In our implementation, snakes are runningcounterclockwise around their center, so the left side corresponds to the internalregion, and the right side to the external region. With a positive value of δ, wereach the minimum (negative) energy when the snake encloses a bright regionon a darker background, while a null energy corresponds to an uniform image.Using a genetic minimization algorithm, the presence of a negative term does notalter the behaviour of the process, as the sigma scaling method used to evaluatethe performance value tunes itself to the average value of the fitness function.

v

u

uij

v

Fig. 2. Binding between adjacent snakes

As introduced before, we also add an additional term to the internal energy,which we call binding energy. This term models the anatomical relationshipsbetween adjacent bones, by introducing and elastic force that connects togetherappropriate points of adjacent snakes, as shown in Fig. 2. The energy associatedto the elastic force is assumed to be represented by the relation:

Ebind[u(s),v(t)] = µ|u(s)− v(t)|2 (5)

where µ represents the elastic constant of the spring. The application pointsof the elastic forces are selected accordingly to the physical relationships whichexist between the anatomical regions.

5 Application

The image data set is composed of radiographic images of the left hand andwrists, acquired by means of a conventional radiographic system, and digitizedwith a spatial resolution of 300 dpi, and a pixel depth of 12 bits. Patient ageranges from 0 to 12 years. Afterward, the images have been downsampled by afactor of three to speed up the segmentation process (see Fig. 3).

In this work we evaluate the application of genetic snakes to the segmentationof a subpart of the hand, the first finger. This allows to develop and test asimpler model, although the task includes most of the segmentation problems.In particular, the three phalanx present a different contrast and mean gray level,

Fig. 3. Radiographic image of the hand

due to their different thickness, have different size, and may have an incompleteboundary due to the presence of cartilaginous tissue.

The situation has been modelled using three snakes, which represent thethree bones, chained together by means of the binding force. Information aboutthe geometry of the bones are not necessary, but only their relative position.

A region of interest is defined for each snake by setting its center and theminimum and maximum magnitude for each v(s). Each snake is composed of36 points, and the binding energy acts on five couples of consecutive points ineach junction. In order to simulate the anatomical relationships, binding energyis assumed to act between the upper part (θ ' π/2) of the first snake and thelower (θ ' −π/2) part of the second snake. Analogously we connect the upperpart of the second snake to the lower one of the third.

The internal energy of the model is given by a weighted sum of Eint andEbind, defined respectively in (2) and (5). In our implementation it is possibleto use different α and β for each snake.

The image energy is computed as an appropriate combination of Eext andEder ((3) and (4)). Equation (3) has been applied by incorporating three gradientof Gaussians with different σ. The filter F used in (4) is a Diffence of Gaussians,computed for three different couples of σ to obtain three smoothed versions ofthe image (see Fig. 4)

The fitness function is the total energy of the model i.e. the sum of theinternal and image energy:

E = Eint + Ebind + Eext + Eder (6)

Fig. 4. Gradient of Gaussian images (σ = 0.3, σ = 3, σ = 10) and Difference of Gaus-sian images (σ1 = 0.5, σ2 = 1.5; σ1 = 1, σ2 = 3; σ1 = 5, σ2 = 8)

The GA implementation adopted in this work is GAucsd-1.4 [25]. We usedmost of the default options proposed by the GAucsd package, i.e. Gray-code, fit-ness sigma scaling, two point crossover, roulette wheel selection. The parametersof the GA were: length of the genome = 324, population size = 65000, maximumnumber of generations = 700, crossover rate = 0.59, mutation rate = 0.00001.Each run takes about 15 min. Figure 5 shows the fitness evolution of the bestindividual.

We performed several experiments varying the snake energy weighting coef-ficients. As more internal and external energy terms are considered in our modelthan in the classical snake, it is hard to determine the appropriate ratio be-tween different forces in the total energy function of the model, due to lack ofunderstanding on the effect of each force on the energy function.

The results showed in Fig. 6 have been obtained with following weights:α1 = 0.9, β1 = 6, α2 = 0.5, β2 = 5, α3 = 0.7, β3 = 10, µ = 1.25, δ = −1.2,γ1 = 2.5, γ2 = 1. γ1 = 0.5. For this setting of weights we did 25 GA runs.

Results have been evaluated by comparing the obtained segmentation withan hand-drawn outline. We can define as TP the portion of bone area correctlyenclosed in the automatic segmentation, FP the portion of image not belongingto the bone, but included in the segmented region, and FN as the bone areawhich is not included in the segmented image. With this assumption, we canevaluate A+ = FP

TP+FN and A− = FNTP+FN . In our tests, we found approximately

a value of A+ = 8%, and A− = 3.5%.

0

50000

100000

150000

200000

250000

300000

350000

0 100 200 300 400 500 600 700

fitne

ss

generation

Fig. 5. Evolution of the fitness

Fig. 6. Segmentation results of the three phalanx

6 Conclusions

In this paper a method for automatic segmentation of hand radiograms is de-scribed along with some results obtained with the proposed approach on a sub-part of the hand.

The complexity of the skeletal structure in the hand and the variability be-tween different subjects makes very difficult to realize an automatic segmentationof the bones. The proposed method allow to combine the a-priori knowledge onthe hand structure to the adaptative behaviour of active contours and genetic al-gorithm. Binding and derivative energy allow to introduce adequate constraintson the geometry of the snake to obtain a satisfactory segmentation.

It is known that the snake model requires either a local minimizer withgood initialization or otherwise a global minimizer. Genetic snakes confront andovercome at the same time the two primary problems of initialization and opti-mization, and provide a global optimization with an automatic initialization.

The encouraging results reported prompt us that the method could be ex-tended and applied to other bone structures as well as to other images. Otherextensions could consider the study of the parameters and the functionals govern-ing the snake behaviour. A method to reduce user interaction by automaticallyassigning snake energy weights is still an open problem. Therefore the evolutionof weights could be considered for future studies.

References

1. Greulich, W.W., Pyle, S.I.: Radiographic atlas of skeletal development of the handand wrist. 2nd edn. Stanford University Press, Palo Alto, CA (1959)

2. Tanner, J.M., Whitehouse, R.H., Marshall, W.A., Healy, M.J.R.: Assessment ofskeletal maturity and prediction of adult height (TW2 method). 2nd edn. AcademicPress, London (1983)

3. Ballerini, L.: Genetic snakes for medical images segmentation. In: Evolutionary Im-age Analysis, Signal Processing and Telecommunications. Volume 1596 of LecturesNotes in Computer Science., Springer (1999) 59–73

4. Kass, M., Witkin, A., Terzopoulos, D.: Snakes: Active contour models. Interna-tional Journal of Computer Vision 1 (1988) 321–331

5. Goldberg, D.E.: Genetic Algorithms in Search, Optimization, and Machine Learn-ing. Addison-Wesley, Reading, MA (1989)

6. McInerney, T., Terzopoulos, D.: Deformable models in medical image analysis: Asurvey. Medical Image Analysis 1 (1996) 91–108

7. Jain, A.K., Zhong, Y., Dubuisson-Jolly, M.P.: Deformable template models: Areview. Signal Processing 71 (1998) 109–129

8. Xu, C., Pham, D.L., Prince, J.L.: Image segmentation using deformable models.In Sonka, M., Fitzpatrick, J.M., eds.: Handbook of Medical Imaging. Volume 2.SPIE Press (2000) 129–174

9. Cheung, K.W., Yeung, D.Y., Chin, R.T.: On deformable models for visual patterrecognition. Patter Recognition 35 (2002) 1507–1526

10. Storvik, G.: A bayesian approach to dynamic contours through stochastic samplingand simulated annealing. IEEE Transactions on Pattern Analysis and MachineIntelligence 16 (1994) 976 –986

11. Grzeszczuk, R.P., Levin, D.N.: Brownian strings: Segmenting images with stochas-tically deformable contours. IEEE Transactions on Pattern Analysis and MachineIntelligence 19 (1997) 110–1114

12. Amini, A., Weymouth, T., Jain, R.: Using dynamic programming for solving vari-ational problems in vision. IEEE Transactions on Pattern Analysis and MachineIntelligence 12 (1990) 855–867

13. Geiger, D., Gupta, A., Costa, L., Vlontzos, J.: Dynamic programming for detect-ing, tracking and matching deformable contours. IEEE Transactions on PatternAnalysis and Machine Intelligence 17 (1995) 294–302

14. Williams, D.J., Shah, M.: A fast algorithms for active contours and curvatureestimation. CVGIP: Image Understanding 55 (1992) 14–26

15. Ji, L., Yan, H.: Attractable snakes based on the greedy algorithm for contourextraction. Pattern Recognition 33 (2002) 791–806

16. MacEachern, L.A., Manku, T.: Genetic algorithms for active contour optimiza-tion. In: Proc. IEEE International Symposium on Circuits and Systems. Volume 4.(1998) 229–232

17. Tanatipanond, T., Covavisaruch, N.: An improvement of multiscale approach todeformable contour for brain MR images by genetic algorithms. In: Proc. IEEEInternational Symposium on Intelligent Signal Processing and Communication Sys-tems, Phucket, Thailand (1999) 677–680

18. Ooi, C., Liatsis, P.: Co-evolutionary-based active contour models in tracking ofmoving obstacles. In: Proc. International Conference on Advanced Driver Assis-tance Systems. (2001) 58–62

19. Toet, A., Hajema, W.P.: Genetic contour matching. Pattern Recognition Letters16 (1995) 849–856

20. Cootes, T., Taylor, C.J., Cooper, D.H., Graham, J.: Active shape models - theirtraining and application. Computer Vision and Image Understanding 61 (1995)38–59

21. Ruff, C.F., Hughes, S.W., Hawkes, D.J.: Volume estimation from sparse planarimages using deformable models. Image and Vision Computing 17 (1999) 559–565

22. Undrill, P.E., Delibasis, K., Cameron, G.G.: An application of genetic algorithmsto geometric model-guided interpretation of brain anatomy. Pattern Recognition30 (1997) 217–227

23. Mignotte, M., Collet, C., Perez, P., Bouthemy, P.: Hybrid genetic optimizationand statistical model-based approach for the classification of shadow shapes insonar images. IEEE Transactions on Pattern Analysis and Machine Intelligence22 (2000) 129–141

24. Cagnoni, S., Dobrzeniecki, A.B., Poli, R., Yanch, J.C.: Genetic algorithm-basedinteractive segmentation of 3D medical images. Image and Vision Computing 17(1999) 881–895

25. Schraudolph, N.N., Grefenstette, J.J.: A user’s guide to GAucsd 1.4. TechnicalReport CS92-249, Computer Science and Engineering Department, University ofCalifornia, San Diego, La Jolla, CA (1992)