Automatic segmentation of MR images of the developingnewborn brain q

Marcel Prastawa a,*, John H. Gilmore b, Weili Lin c, Guido Gerig a,b

a Department of Computer Science, University of North Carolina, CB #3175 Sitterson Hall, Chapel Hill, NC 27599, USAb Department of Psychiatry, University of North Carolina, Chapel Hill, NC 27599, USAc Department of Radiology, University of North Carolina, Chapel Hill, NC 27599, USA

Available online 12 July 2005

Abstract

This paper describes an automatic tissue segmentation method for newborn brains from magnetic resonance images (MRI). Theanalysis and study of newborn brain MRI is of great interest due to its potential for studying early growth patterns and morpho-logical changes in neurodevelopmental disorders. Automatic segmentation of newborn MRI is a challenging task mainly due to thelow intensity contrast and the growth process of the white matter tissue. Newborn white matter tissue undergoes a rapid myelinationprocess, where the nerves are covered in myelin sheathes. It is necessary to identify the white matter tissue as myelinated or non-myelinated regions. The degree of myelination is a fractional voxel property that represents regional changes of white matter asa function of age. Our method makes use of a registered probabilistic brain atlas. The method first uses robust graph clusteringand parameter estimation to find the initial intensity distributions. The distribution estimates are then used together with the spatialpriors to perform bias correction. Finally, the method refines the segmentation using training sample pruning and non-parametrickernel density estimation. Our results demonstrate that the method is able to segment the brain tissue and identify myelinated andnon-myelinated white matter regions.� 2005 Elsevier B.V. All rights reserved.

Keywords: Automatic brain MRI classification; Automatic brain MRI segmentation; Early brain development; Kernel density estimation; Neonatal

MRI; Robust estimation

1. Introduction

The segmentation of newborn brain structures frommagnetic resonance images (MRI) is crucial for thestudy of normal development and comparison to neuro-developmental disorders at early stages. The develop-ment of new segmentation methods for this age groupis driven by the increasing use of MRI to study new-

borns, for example our ongoing study of early braindevelopment in normal and high risk children (Zhaiet al., 2003; Gilmore et al., 2004) and the lack of appro-priate segmentation methodology. Manual segmenta-tion of newborn brains is tedious, time consuming,and lacks reproducibility. Therefore, it is necessary touse automatic segmentation methods for clinical studieswith multiple subjects. This task is considerably morechallenging compared to automatic segmentation ofadult brain MRI due to the early development process;Rutherford (2002) provides an excellent description ofnewborn MRI and the dynamic changes seen over theearly development period.

In newborn infant brains, the white matter structureundergoes myelination, where the fibers are being

1361-8415/$ - see front matter � 2005 Elsevier B.V. All rights reserved.

doi:10.1016/j.media.2005.05.007

q This research is supported by the UNC Schizophrenia Research

Center, an NIMH Conte Center MH064065 (PI J. A. Lieberman and

J.H. Gilmore) and the UNC Neurodevelopmental Disorders Research

Center HD 03110 (PI J. Piven). Marcel Prastawa is supported by NIH-

NIBIB R01 EB000219 (PI E. Bullitt).* Corresponding author. Tel.: +1 919 962 1836.E-mail address: prastawa@cs.unc.edu (M. Prastawa).

www.elsevier.com/locate/media

Medical Image Analysis 9 (2005) 457–466

covered in myelin sheathes. At birth, the white matter ofthe brain stem and the posterior limbs of the internalcapsule are myelinated and have white to gray mattercontrast similar to that of adults (white matter is bright-er than gray matter in the T1w image). Other regions ofwhite matter such as the centrum semi-ovale are notmyelinated and the white to gray matter contrast is in-verted (white matter is darker than gray matter in theT1w image). As the child ages from birth to one year,myelination progresses through the anterior limbs ofthe internal capsule, the occipital radiations, and thento the frontal white matter. As this happens, the MRrelaxation times of these regions change with the newmyelinated fibers consequently changing the MRI sig-nal. By 1.5 years, the MR image contrast is almostadult-like. Fig. 1 shows an example of a newborn MRimage with the myelinated and non-myelinated whitematter regions.

Several methods have been developed for automati-cally segmenting healthy adult brain MRI, mostly vari-ations of multi-variate statistical classificationtechniques. Wells et al. (1996) proposed an Expecta-tion-Maximization scheme that interleaves segmentationand intensity bias correction. This method was extendedby Van Leemput et al. (1999b) through the use of aprobabilistic brain atlas. Warfield et al. (2000) describeda k-nearest neighbor classification algorithm that iscombined with template matching. Cocosco et al.(2003) combines atlas sample selection and robust sam-ple pruning using minimum spanning trees (MST) forintensity-based classification. A different class of seg-mentation techniques uses deformable templates thattransfer the labeling of a template to each new subject.For example, the work by Collins et al. (1999) whichcombines neural network classification with nonlinearimage matching.

Automatic segmentation methods for healthy adultbrain MRI typically fail in segmenting all the differentstructures apparent in newborn brain MRI, particularlythe myelinated white matter regions. Methods that useprobabilistic brain atlases (Van Leemput et al., 1999b;Collins et al., 1999) or templates (Warfield et al., 2000)cannot be directly applied to newborn brain MRI sincethe spatial prior information for rapidly changing mye-lination property would be very difficult to define. War-field et al. (2000) uses a specific template for newbornbrains with predefined classifications for myelinatedand non-myelinated white matter. Methods that are dri-ven by image intensities (Wells et al., 1996; Cocoscoet al., 2003) would have difficulties in the initializationphase. The MR image intensities for newborn brainsare significantly affected by both low contrast and RFinhomogeneity, which can be difficult to overcome with-out spatial prior information.

Matsuzawa et al. (2001) presented a segmentationmethod for infant brain MRI, as part of a study of earlybrain development. Their method does not identify mye-linated white matter and non-myelinated white matterseparately. The results show that their method has diffi-culties dealing with tissue separation. Huppi et al. (1998)and Inder et al. (2005) showed segmentation results ofnewborn infants, using the method of Warfield et al.(2000). They study both prematurely born infants andnormal infants. The prematurely born infants tend tohave simpler cortical folding compared to normal new-borns. The segmentation method identifies non-myelin-ated and myelinated white matter. Boardman et al.(2003) used image deformation for detecting regions ofmajor development.

Automatic segmentation of newborn brain MRI issignificantly more challenging than the segmentationof adult brain MRI. This is mainly due to the biologyand the rapid growth process. The specific challengesare:

(1) The white matter and gray matter contrast to noiseratio (CNR) for newborn MRI can be as low ashalf of the one for adult brain MRI. Factors thatreduce CNR are the small size of the infant brainsand the short scanning period. The small head sizerequires them to be scanned at higher resolution,which leads to higher noise levels. The infants needto be scanned in very short time since they are notsedated or constrained. The low CNR causes diffi-culty in segmenting the partial volume regions.

(2) Typically, newborn brain MRI exhibits motionartifacts even with very short scan sequences. Theinfants may not stay motionless during the scanperiod. This problem can be difficult to solve sincethe infants are not mentally aware, and healthyinfants cannot be sedated or restrained due toethical reasons.

Fig. 1. MR images of a newborn brain (subject 0096, coronal view).

Left: T1w image, right: T2w image. The arrows show the white matter

structure. The arrow with the solid line indicates myelinated white

matter, the arrow with the dashed line indicates non-myelinated white

matter. Early myelination in white matter is shown as bright regions in

the T1w image and dark regions in the T2w image.

458 M. Prastawa et al. / Medical Image Analysis 9 (2005) 457–466

(3) The process of myelination separates white mattertissue into two types: myelinated and not myelin-ated. We treat myelination as a fractional propertybecause the MR image intensities reflect the degreeof myelination and partial voluming. The dividingboundaries between regions that are fully myelin-ated and non-myelinated are generally ambiguous(Rutherford, 2002). The myelinated white matterregions are mostly distributed near the spine (cen-tral posterior) and parts of the internal capsule.We also observed the presence of myelinated whitematter around the regions associated with the sen-sory and motor cortex.

(4) Each tissue type in newborn brain MRI exhibitssignificant levels of intensity inhomogeneity andvariability, which may be due to a combinationof RF inhomogeneity and biological properties ofthe developing tissue (Kandel et al., 2000).

(5) The different tissues have large overlaps in theirintensity characteristics, as shown in Fig. 2. Thedecision boundaries for intensity-based classifica-tion are typically ambiguous and complex.

We developed an atlas based segmentation algorithmfor newborn brain MRI that addresses the challengeslisted above. The method incorporates the robust clus-tering method proposed by Cocosco et al. (2003) andthe robust parameter estimation method presented byRousseeuw and Van Driessen (1999) to deal with noisydata. It uses the intensity inhomogeneity estimationscheme from spatial classification proposed by VanLeemput et al. (1999a). The complex decision bound-aries are modeled using non-parametric kernel densityestimates, using the efficient method of Girolami andHe (2003). The probabilistic atlas is used as a spatialprior in the classification process as proposed by VanLeemput et al. (1999a).

2. Method

Due to the large overlap in the tissue intensity distri-butions, it becomes necessary to use spatial priors forthe segmentation. The spatial priors that we use is partof a probabilistic brain atlas of newborn MRI, shownin Fig. 3. The atlas provides voxel prior probabilitiesfor white matter, gray matter, and cerebrospinal fluid(csf). Myelinated white matter and non-myelinatedwhite matter are combined as one white matter classin the atlas. This is necessary because it is difficult tomodel the different dynamic growth patterns across sub-jects given the significant changes during early braindevelopment. With the combined white matter prior,the discrimination between the two different white mat-ter classes is primarily driven by the image intensities.The atlas was created by averaging three semi-automaticsegmentations registered using affine transformation.Each segmentation was done by a human rater that se-lects samples for each tissue types for k-nearest neighborsegmentation. The outputs of the k-nearest neighborclassification are then edited by manual outlining.The number of subjects is insufficient to create prior

Fig. 2. Intensity characteristics of one coronal slice of a newborn brain

MRI dataset (subject 0096). Top, from left to right: T1w image, T2w

image, and the manually assigned labels. Purple is myelinated white

matter, green is non-myelinated white matter, yellow is gray matter,

and blue is cerebrospinal fluid. Bottom: the scatterplot of the tissue

intensities, the horizontal axis represents T1w intensities and the

vertical axis represents T2w intensities. There is significant overlap

between the intensities of different tissues, and there are ambiguities in

the decision boundaries.

Fig. 3. The probabilistic brain atlas of a newborn brain. From left to

right: (a) the T1w average image, (b) the T2w average image, and the

spatial prior probability values for (c) white matter (either myelinated

or non-myelinated), (d) gray matter, and (e) csf. Top: axial view.

Bottom: coronal view.

M. Prastawa et al. / Medical Image Analysis 9 (2005) 457–466 459

probabilities that reflect the variability in the popula-tion. At this point, we are limited by the size of our data-sets and the amount of time for manual processing. Tocompensate for the lack of available data, an additionalblurring is applied to the average segmentations. Theblurred spatial probabilities simulate an atlas with ahigher level of population variability.

Our segmentation framework is composed of threemajor steps, as shown in Fig. 4. First, it obtains roughestimates of the class intensity clusters. It then iterativelyperforms inhomogeneity correction and parametric clas-sification. Finally, it refines the segmentation using non-parametric kernel density estimates.

Before segmentation, we register the atlas to the sub-ject using affine transformation and the mutual informa-tion image match metric (Maes et al., 1997). In thisstudy, we use T1w and T2w MR images, where theT2w images are registered to the T1w images also by af-fine transformation and mutual information metric. Thelower resolution T2w images are up-sampled with splineinterpolation. The registered subject MR images are fil-tered using anisotropic diffusion (Gerig et al., 1992) toreduce noise and motion artifacts.

2.1. Robust intensity distribution estimation

The segmentation of newborn brain MRI involvesclassifying each voxel into different categories C, whereC is commonly defined to be {myelinated white matter,non-myelinated white matter, gray matter, and cerebro-spinal fluid}. The first step in the segmentation processis to determine the rough estimates of the class inten-sity distributions. We obtain samples for class Ci atlocation ~x with high atlas prior probability values, forexample Prð~xjCiÞ > 0.9 as presented by Cocosco et al.(2003).

The white matter samples are constrained to have lowimage gradient magnitude values to avoid choosingsamples near the transition regions between myelinatedand non-myelinated white matter and at white/graymatter boundaries. The value we use for the gradientmagnitude of our 3-D images is the 2-norm of the vectorof individual gradient magnitudes, where

Gð~xÞ ¼

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi

jrI1ð~xÞj2 þ � � � þ jrInð~xÞj

2q

. ð1Þ

We only retain samples for the white matter class withGð~xÞ lower than the average of Gð~xÞ over the whitematter prior, c ¼ ½

P

~x Prð~xjwhite-matterÞGð~xÞ�=½P

~xPrð~xjwhite-matterÞ�. The 2-norm gradient magnitude metricis more sensitive to noise compared to the vector fieldgradient magnitude metric described in Lee and Cok(1991). This is a desired property since we want to avoidsampling noisy regions.

We then process the obtained intensity samples to re-move outliers and false positives. We use the minimumcovariance determinants (MCD) estimator (Rousseeuwand Van Driessen, 1999) to generate the robust meanand covariance estimates of the unimodal distributions(gray matter and csf). The MCD estimator computesthe robust mean and covariance that have the smallestdeterminant of covariance and covers at least half ofthe data. For the bi-modal white matter distribution,we use a robust graph based clustering method, similarto the one described in (Cocosco et al., 2003). The clus-tering method creates the minimum spanning tree(MST) graph (Cormen et al., 2001) from the samplepoints and breaks long edges to form the clusters (Dudaet al., 2001). The minimum spanning tree is the graphwhere all the points are connected such that the totaledge lengths are minimized. The MST graph does nothave any closed loops (cycles). The removal of sampleswith high image gradient helps in the MST clusteringprocess, as shown in Fig. 5.

The algorithm searches for myelinated white matterand non-myelinated white matter intensity clusters byiteratively breaking long edges of the MST. At each iter-ation, we break an undirected edge e(v,w) that connectsvertices v and w if it is longer than A(v) · T or A(w) · T.A(v) is the average length of edges incident on vertex v,AðvÞ ¼ 1

ns

P

sjeðv; sÞj, while T is a distance multiplier. Theedge breaking results in subtrees where each subtreeforms an intensity cluster. For each detected cluster,an intensity location estimate is computed. The clusterintensity location estimate provides an approximationof where most points in the cluster is distributed in theintensity space. The iterative algorithm terminates when

Fig. 4. The segmentation framework.

460 M. Prastawa et al. / Medical Image Analysis 9 (2005) 457–466

two clusters are found with intensity location estimatesthat are in the proper order. For example, the order ofintensities for the classes in T2w from darkest to bright-est is myelinated white matter, gray matter, non-myelin-ated white matter, and csf.

The intensity location estimates for the two whitematter classes are also computed using the MCD estima-tor. We use the robust MCD mean values, as opposed tothe standard location estimates such as the mean ormedian, to make sure that we obtain reasonable sampleclusters. The standard location estimates such as meanor median may not always be optimal for the noisy new-born MRI data. The mean value can be skewed by a sin-gle outlier sample, while the median value only uses onesample point and ignores contributions of other sam-ples. The initial intensity distributions for non-myelin-ated white matter and myelinated white matter arecomputed as the MCD mean and covariance estimatesof the largest detected clusters. The MCD estimatortherefore serves to estimate the initial intensity distribu-tions. The initial gray matter and csf distributions arethe MCD estimates of the atlas sampled data. The initialwhite matter distributions are the MCD estimates of theatlas samples that are clustered and pruned using MST.The steps involved in the intensity distribution estima-tion are listed in Algorithm 1.

Algorithm 1. Initial intensity distribution estimation

1: Obtain samples by thresholding atlas priorprobabilities

2: Remove white matter samples with gradient magni-tude higher than c

3: Compute robust mean intensity values for graymatter and csf (lgm and lcsf) using the MCDestimator

4: Construct Minimum Spanning Tree from whitematter samples

5: T 26: repeat

7: Break edges longer than T · A, where A is the aver-age length of connected neighbor edges

8: Find largest myelinated white matter cluster, wherelmyelinated < lgm in T2w

9: Find largest non-myelinated white matter cluster,where lgm < lnon-myelinated < lcsf in T2w

10: T T � 0.0111: until both white matter clusters are found or T 6 112: if T < 113: Algorithm fails14: end if

15: Compute white matter Gaussian distributionparameters from detected clusters

2.2. Inhomogeneity correction

Newborn brain MRI exhibit higher intensity vari-ability for each tissue and low intensity contrast com-pared to adult brain MRI. These two factors severelyhamper the estimation of intensity inhomogeneity. His-togram based intensity inhomogeneity estimation meth-ods, such as the ones proposed by Sled et al. (1998)and Styner et al. (2000), are likely to have difficultiesin obtaining the optimal solution. The histogram of anewborn brain MR image is generally smooth withweak maximas.

In the case of inhomogeneity correction of newbornbrain MRI, the spatial information is useful to deal withthe low intensity contrast. We have chosen to use themethod developed by Van Leemput et al. (1999a). Thescheme uses the spatial posterior probabilities to esti-mate the intensity inhomogeneity, which helps to over-come problems with low contrast and high variability.The inhomogeneity estimation method is an iterativegeneralized expectation maximization algorithm. Itinterleaves classification with inhomogeneity estimationat each iteration. The Gaussian distributions obtainedfrom the previous segmentation step are used as initialparameters for the iterative inhomogeneity estimationalgorithm.

The intensity likelihoods are modeled using paramet-ric Gaussian functions, and the inhomogeneity is mod-eled using polynomials:

pð~Ið~xÞjCiÞ ¼ /Rið~Ið~xÞ � li �

X

k

bkqkð~xÞÞ; ð2Þ

where / is the Gaussian function, with mean li andcovariance Ri, the intensity inhomogeneity is the linearcombination of the coefficients bk and the basis polyno-mials qk. The intensity inhomogeneity is estimated byleast squares fitting of the polynomial coefficients tothe log difference of the original image and the recon-structed image. The reconstructed image is the homo-geneous image computed with the mean values li andthe class posterior probabilities. The class posteriorprobabilities are computed using the image intensitylikelihood probabilities and the atlas prior prob-abilities:

Fig. 5. Illustrations of the minimum spanning trees for white matter

obtained using different sampling strategies. Left: Samples with high

probability values. Right: Samples with high probability values and

low gradient magnitude. Choosing only samples with low gradient

magnitude helps to remove samples from the transition regions

between myelinated white matter and non-myelinated white matter

and gray/white boundary voxels. This is crucial for clustering based on

edge breaking. As seen on the right picture, breaking the longest edge

marked by the arrow would give two well separated clusters.

M. Prastawa et al. / Medical Image Analysis 9 (2005) 457–466 461

pðCij~Ið~xÞÞ ¼

P

ipð~Ið~xÞjCiÞPrð~x;CiÞP

iPrð~x;CiÞ; ð3Þ

where PrðCi;~xÞ is a combination of the atlas prior proba-bilities (Prð~xjCiÞ) and the global class prior probabili-ties ðPrðCiÞÞ: Prð~x;CiÞ ¼ Prð~xjCiÞPrðCiÞ. The myelinatedand non-myelinated white matter shares the same atlasprior, Prð~xjC1Þ ¼ Prð~xjC2Þ. The global class prior prob-abilities can be tuned based on the age of the newbornsto be segmented. For the results presented here, we setthe global class priors such that white matter is morelikely to be not myelinated: Pr(C1) = 0.2, Pr(C2) = 0.8,and Pr(Ci) = 1 for the other classes. The use of the atlasspatial prior probabilities Prð~xjCiÞ helps resolve ambigu-ities that are caused by the low image contrast, followingthe formulation used in Van Leemput et al. (1999b).

2.3. Segmentation refinement using kernel density

estimation

The class intensity likelihoods are modeled as Gauss-ian probability density functions in the segmentationand inhomogeneity correction to obtain an optimalparametric solution. The use of the parametric Gaussiandistribution eases the computation of the maximum like-lihood estimate. However, Gaussian distributions canhave significant overlap and therefore result in degener-ate decision boundaries. In order to capture the complexand ambiguous intensity characteristics of newbornbrain MRI, we switch from the parametric Gaussian dis-tribution to a non-parametric distribution estimate. Werefine the classification by sampling the inhomogeneitycorrected images, pruning the outliers and false positivesfrom the intensity samples, and then estimating theintensity distribution using kernel density functions(Duda et al., 2001; Hastie et al., 2001).

The non-parametric intensity probability densityfunction for each class is estimated as follows:

pð~Ið~xÞjCiÞ ¼X

N i

j¼1

wijKhð~Ið~xÞ � T ijÞ; ð4Þ

where Kh is the Gaussian kernel with standard deviationh, Ni is the number of training samples for class Ci, andTij is the jth training sample for the ith class. Each train-ing sample has an associated weight wij, where for eachclass Ci,

PN i

j¼1wij ¼ 1. The kernel density estimates areused to produce the final classification results, whichare the class posterior probabilities:

pðCij~Ið~xÞÞ ¼

P

ipð~Ið~xÞjCiÞPrðCi;~xÞP

iPrðCi;~xÞ. ð5Þ

The atlas spatial prior probabilities are also used atthis stage. The spatial priors are combined with thenon-parametric kernel densities to provide class poster-

ior probabilities that are capable of capturing morecomplex intensity characteristics.

The set of training samples T for the kernel densityestimates are obtained by sampling the MR imagesusing the previously obtained posterior probabilities.Each sample Tij is obtained by selecting features at loca-tion~x where

argmaxCk

pðCkj~Ið~xÞÞ ¼ Ci. ð6Þ

The samples are pruned and clustered using the ro-bust MST-based method proposed by Cocosco et al.(2003). This step removes the false positives and outliersin the intensity data resulting from using Gaussian dis-tribution estimates in the previous step.

The method proposed by Girolami and He (2003) isapplied to efficiently estimate the kernel density func-tion. This method speeds up the density estimation pro-cess by reducing the size of the training set. The weightswij are chosen to minimize the integrated squared errorbetween the true density function and the estimated ker-nel density function. Redundant training features are as-signed lower weight values compared to characteristictraining features. This minimization process for the sam-ple weight assignment is similar to the quadratic optimi-zation process for Support Vector Machines, for whichan efficient solution exists (Scholkopf et al., 2001). Thesamples with zero weights are removed from the trainingset, which effectively eliminates the redundant featuresin the training set. Compared to other fast density esti-mation techniques such as pre-binning (Scott andSheather, 1985) and multi-scale selection using hyper-discs (Mitra et al., 2002), this method has the advantageof having only one user specified parameter: the kernelwidth or the standard deviation of the Gaussian kernels.

3. Results

We show the application of our new segmentationmethod to four different subjects. Fig. 6 shows the coro-nal view of the MR images along with two sets of man-ual segmentation slices done by different raters. Fig. 7shows the coronal view of the automatic segmentationresults. The 3D volumes for the automatically seg-mented structures are listed in Table 1. The four casesare samples from a large neonatal study at UNC ChapelHill to assess early brain development in normal andhigh risk children (Zhai et al., 2003; Gilmore et al.,2004). We currently have over 50 datasets of neonatalMRI and will collect a total of 125, with some of themfollowed-up at the age of one year. As part of the study,we plan to measure the cortical folding and the corticalthickness of the newborn brains. Fig. 8 illustrates the 3Dview of the relevant structures for one of the subjects.Visual inspection of the results show that the myelinated

462 M. Prastawa et al. / Medical Image Analysis 9 (2005) 457–466

white matter regions are mostly distributed near thespine (central posterior) and internal capsule. We havealso observed the presence of small regions of myelin-ated white matter around the regions associated withthe sensory and motor cortex.

Images were acquired on a Siemens head-only 3Tscanner (Allegra, Siemens Medical System, Erlangen,Germany). Two structural imaging sequences wereused: a magnetization prepared rapid gradient echo(MPRAGE) T1-weighted and a turbo spin echo (TSE),dual-echo (proton density and T2 weighted). Total scantime for structural scans was approximately 10 min.The imaging parameters for the MP-RAGE sequencewere: repeat time TR = 11.1 ms, echo time TE = 4.3 ms,inversion time TI = 400 ms, slice thickness TH = 1 mm,in-plane resolution = 0.898 · 0.898 mm2. A total of 128sagittal images were acquired to cover the entire brain.The imaging parameters for the TSE sequence were:TR = 7 s, TE = 15 and 90 ms, TH = 1.95 mm, in-planeresolution 1.25 · 1.25 mm2, and 56 slices.

Validation of the automatic segmentation results aredifficult because a gold standard does not exist. Thecommon standard, manual segmentations, is difficultto obtain since highly convoluted structures in low-con-trast, noisy data are very hard to trace. In addition tothat, the myelinated white matter and the non-myelin-ated white matter have ambiguous boundaries, which

Fig. 6. The MR images along with the manually segmented labels.

From left to right: (a) T1w image, (b) T2w image, (c) color image

showing the segmentation done by the first human rater, and (d) color

image showing the segmentation done by the second human rater.

Purple is myelinated white matter, green is non-myelinated white

matter, yellow is gray matter, and blue is csf. From top to bottom:

subject 0096, 0117, 0118, and 0123.

Fig. 7. Coronal view of the 3D automatic segmentation results. From left to right: (a) the T2w image and the class posterior probabilities for (b)

myelinated white matter, (c) non-myelinated white matter, (d) gray matter, and (e) cerebrospinal fluid. From top to bottom: subject 0096, 0117, 0118,

and 0123.

M. Prastawa et al. / Medical Image Analysis 9 (2005) 457–466 463

would make manual segmentation results highly vari-able and difficult to reproduce. We have done a limitedvalidation of the segmentation results by restricting thevalidation to only a 2D coronal slice of each dataset.The human raters assign discrete labels to each voxelin the slice. The degree of myelination is not specified be-cause it is extremely difficult for the raters to consistentlyassign a continuous weight for myelination. The poster-ior probabilities that are generated by the automatic seg-mentation method are discretized following Eq. (6).

We use Cohen�s j (Cohen, 1960) to measure the seg-mentation variability. The j value given two observa-tions and N classes is defined as

j ¼

PN

i¼1agreeðCiÞ �PN

i¼1efðCiÞ

N �PN

i¼1efðCiÞ; ð7Þ

where agree(Ci) is the number of agreements betweenthe two observers for class Ci and ef(Ci) is the expectedfrequency of agreement by chance for class Ci. Given thenumber of observations of class Ci for the first and sec-ond observer, ai and bi, the expected frequency of agree-ment is efðCiÞ ¼

1Naibi. This reliability measure places

equal weight on the samples of each class, so classes withlarger number of observations will have more influenceon the final result. The j value is normalized, 0 indicatesindependence and 1 indicates complete agreement. j val-ues above 0.7 is generally interpreted to reflect a satisfac-tory level of reliability. The j values comparing the twomanual segmentations and the manual segmentationagainst the automatic segmentation is shown in Table 2.

We also measure the overlap of the segmentations ofeach class using the Dice similarity coefficient (Dice,1945). For two segmentations A and B, the Dice similar-ity coefficient is defined as 2jA \ Bj=ðjAj þ jBjÞ. Thisoverlap measure is normalized, where 0 indicates com-plete dissimilarity and 1 indicates complete agreement.The overlap values reflecting inter-rater variability isshown in Table 3. The overlap comparison betweenthe manual raters and the automatic segmentationmethod is shown in Tables 4 and 5. Since the validationis only done on 2D slices presenting complex foldingstructures, the number of samples is low and conse-quently leads to low overlap values.

The j values show that there is insufficient level ofreliability for the two manual segmentations. The non-myelinated white matter and gray matter classes havehigher number of observations compared to the otherclasses and therefore dominate the j measurements.The j values are low because the segmentations of thebrain tissue classes tend to be ambiguous. The overlapmeasures show that the automatic segmentation methodhas similar level of variability to the two manual seg-mentations. The overlap values for csf for the automaticmethod are generally lower due to misclassifications inthe partial volume regions.

Table 2

The j coefficients that measure the level of agreement between manual

raters, first manual rater against the automatic method, and second

manual rater against the automatic method

Subject Rater 1

vs rater 2

Rater 1 vs

automatic

Rater 2 vs

automatic

0096 0.658 0.604 0.558

0117 0.627 0.577 0.587

0118 0.603 0.561 0.500

0123 0.625 0.626 0.542

Table 1

The volumes of the segmented structures for the four subjects

Subject ICV Myelinated

WM

Non-myelinated

WM

Gray

matter

CSF

0096 504,724 15,353 157,160 289,133 43,078

0117 527,885 12,678 234,706 250,161 30,340

0118 514,760 11,480 193,307 255,849 54,124

0123 499,775 28,487 170,227 252,056 49,005

These include the intra cranial volume (ICV) and the volumes of the

individual structures (myelinated white matter, non-myelinated white

matter, gray matter, and cerebrospinal fluid). All volumes are mea-

sured in cubic millimeters.

Fig. 8. Surface renderings of the segmented structures of subject 0123.

From left to right: (a) intra cranial volume, (b) gray matter, (c) non-

myelinated white matter, and (d) myelinated white matter.

Table 3

The Dice similarity values that measure the overlap between the two

manual segmentations

Subject Myelinated

white matter

Non-myelinated

white matter

Gray

matter

CSF

0096 0.715 0.767 0.777 0.738

0117 0.760 0.771 0.741 0.662

0118 0.683 0.738 0.752 0.696

0123 0.787 0.757 0.750 0.639

Table 4

The Dice similarity values that measure the overlap between the

segmentation results of the first human rater and the automatic

method

Subject Myelinated

white matter

Non-myelinated

white matter

Gray

matter

CSF

0096 0.634 0.676 0.809 0.681

0117 0.637 0.725 0.776 0.491

0118 0.681 0.661 0.782 0.598

0123 0.777 0.724 0.790 0.569

464 M. Prastawa et al. / Medical Image Analysis 9 (2005) 457–466

4. Discussion

We have presented an atlas-based automatic segmen-tation method for multi-channel newborn brain MRI.The method uses graph clustering and robust estimationto obtain initial distribution estimates from the noisydata. These estimates are then used to generate spatialposterior probabilities for correcting the intensity inho-mogeneity inherent in the image. The segmentation is fi-nally refined through the use of non-parametric kerneldensity estimates.

The use of a probabilistic atlas or a template such asthe one used in Warfield et al. (2000) is crucial to over-come the intensity contrast limitations. A probabilisticbrain atlas that captures the variability of the large pop-ulation is essential for the proposed method, such as theone described in Evans et al. (1993) for adult brain MRI.The creation of a true newborn brain atlas requires thesegmentations of a large set of representative data. Also,with the high level of brain shape variability in infants, itis likely that a non-linear registration will be required fora reliable atlas formation. These factors make the crea-tion of a newborn brain atlas highly challenging. Thecurrent atlas that is built from a small set of data seemssufficient for our data. However, we are working on animproved atlas that shows better population variability.

Visual inspection of the results shows that the majorstructures are segmented consistently. The segmenta-tions of regions largely affected by partial voluming isstill insufficient and is an inherent problem with voxel-based classification. Illustrations of segmentations ofthe four cases demonstrate that the new method cancope with variable brain shapes. Also, location andshape of the early myelination structures across the sub-jects seem quite similar. The new segmentation tech-nique is currently applied to the whole database ofover 50 neonates (age range is 42.7 ± 1.8 weeks of gesta-tional age) to study volume and structure of brain tissueat this early age. The reproducibility of the results isoptimal since the method is fully automatic.

Due to the lack of a gold standard, we have per-formed only a limited validation of our results. The j

coefficient values and the volume overlap measures showthat our segmentation results have similar level of vari-

ability to the inter-rater variability for manual segmen-tations. A significant challenge in newborn brain MRIsegmentation is the construction of a gold standard forvalidation. To our knowledge, there is no standard data-set available to the community to measure and comparethe performance of segmentation methods for neonatalMRI. This problem is solved for adult brain MRI byusing web-based archives with simulated datasets (Coco-sco et al., 1997; Collins et al., 1998) and a large collec-tion of manually segmented real datasets (MGH, 2004).

Acknowledgments

We acknowledge Koen van Leemput for providingthe MATLAB code that helped with the developmentof the bias correction software and the Insight Toolkitcommunity (Insight Consortium, 2004) for providingthe software framework for the segmentation algorithm.We wish to thank the anonymous reviewers for theirconstructive suggestions which benefited the revisionof this paper.

References

Boardman, J.P., Bhatia, K., Counsell, S., Allsop, J., Kapellou, O.,

Rutherford, M.A., Edwards, A.D., Hajnal, J.V., Rueckert, D.,

2003. An evaluation of deformation-based morphometry in the

developing human brain and detection of volume changes associ-

ated with preterm birth. In: Proceedings of the MICCAI.

Cocosco, C.A., Kollokian, V., Kwan, R.K.-S, Evans, A.C., 1997.

BrainWeb: online interface to a 3D MRI simulated brain database.

NeuroImage 5 (4).

Cocosco, C.A., Zijdenbos, A.P., Evans, A.C., 2003. A fully automatic

and robust brain MRI tissue classification method. Medical Image

Analysis 7, 513–527.

Cohen, J., 1960. A coefficient of agreement for nominal scales.

Educational and Psychological Measurements 20, 37–46.

Collins, D.L., Zijdenbos, A.P., Baare, W.F.C., Evans, A.C., 1999.

ANIMAL+INSECT: Improved cortical structure segmentation.

In: Kuba, A., Samal, M., Todd-Pokropek, A. (Eds.), Infor-

mation Processing in Medical Imaging, vol. 1613 of Lecture

Notes in Computer Science. Springer, Heidelberg, Germany,

pp. 210–223.

Collins, D.L., Zijdenbos, A.P., Kollokian, V., Sled, J.G., Kabani, N.J.,

Holmes, C.J., Evans, A.C., 1998. Design and construction of a

realistic digital brain phantom. IEEE Transactions of the Medical

Imaging 17 (3), 463–468.

Cormen, T.H., Leiserson, C.E., Rivest, R.L., Stein, C., 2001. Intro-

duction to Algorithms, second ed. MIT Press, Cambridge, MA.

Dice, L.R., 1945. Measures of the amount of ecologic association

between species. Ecology 26 (3), 297–302.

Duda, R.O., Hart, P.E., Stork, D., 2001. Pattern Classification, second

ed. Wiley.

Evans, A.C., Collins, D.L., Mills, S.R., Brown, E.D., Kelly, R.L.,

Peters, T.M., 1993. 3D statistical neuroanatomical models from

305 MRI volumes. In: Proceedings of the IEEE Nuclear

Science Symposium and Medical Imaging Conference, pp.

1813–1817.

Gerig, G., Kubler, O., Kikinis, R., Jolesz, F., 1992. Nonlinear

anisotropic filtering of MRI data. IEEE TMI 11, 221–232.

Table 5

The Dice similarity values that measure the overlap between the

segmentation results of the second human rater and the automatic

method

Subject Myelinated

white matter

Non-myelinated

white matter

Gray

matter

CSF

0096 0.774 0.649 0.777 0.598

0117 0.651 0.739 0.760 0.577

0118 0.606 0.630 0.742 0.601

0123 0.719 0.694 0.732 0.478

M. Prastawa et al. / Medical Image Analysis 9 (2005) 457–466 465

Gilmore, J., Zhai, G., Wilber, K., Smith, J., Lin, W., Gerig, G., 2004.

3T magnetic resonance imaging of the brain in newborns.

Psychiatry Research Neuroimaging 132, 81–85.

Girolami, M., He, C., 2003. Probability density estimation from

optimally condensed data samples. IEEE Transactions of the

Pattern Analysis and Machine Intelligence 25 (10), 1253–1264.

Hastie, T., Tibshirani, R., Friedman, J.H., 2001. The Elements of

Statistical Learning. Springer-Verlag.

Huppi, P., Warfield, S., Kikinis, R., Barnes, P., Zientara, G., Jolesz,

F., Tsuji, M., Volpe, J., 1998. Quantitative magnetic resonance

imaging of brain development in premature and normal newborns.

Annals of Neurology 43, 224–235.

Inder, T.E., Warfield, S.K., Wang, H., Huppi, P.S., Volpe, J.J., 2005.

Abnormal cerebral structure is present at term in premature

infants. Pediatrics 115 (2), 286–294.

Insight Consortium, 2004. Insight Toolkit. Available from: <http://

www.itk.org>.

Kandel, E.R., Schwartz, J.H., Jessel, T.M., 2000. Principles of Neural

Science, fourth ed. McGraw Hill.

Lee, H.C., Cok, D.R., 1991. Detecting boundaries in a vector field.

IEEE Transactions of the Signal Processing 39 (5), 1181–1194.

Maes, F., Collignon, A., Vandermeulen, D., Marchal, G., Suetens, P.,

1997. Multimodality image registration by maximization of mutual

information. IEEE TMI 16 (2), 187–198.

Matsuzawa, J., Matsui, M., Konishi, T., Noguchi, K., Gur, R., Bilder,

W., Miyawaki, T., 2001. Age-related volumetric changes of brain

gray and white matter in healthy infants and children. Cerebral

Cortex 11 (4), 335–342.

MGH, 2004. Internet Brain Segmentation Repository. Available from:

<http://www.cma.mgh.harvard.edu/ibsr>.

Mitra, P., Murthy, C.A., Pal, S.K., 2002. Density based multiscale

data condensation. IEEE Transactions of the Pattern Analysis and

Machine Intelligence 24 (6).

Rousseeuw, P.J., Van Driessen, K., 1999. A fast algorithm for the

minimum covariance determinant estimator. Technometrics 41 (3),

212–223.

Rutherford, M. (Ed.), 2002. MRI of the Neonatal Brain. W.B.

Saunders.

Scholkopf, B., Platt, J., Shawe-Taylor, J., Smola, A., Williamson, R.,

2001. Estimating the support of a high-dimensional distribution.

Neural Computation 13, 1443–1471.

Scott, D.W., Sheather, S.J., 1985. Kernel density estimation with

binned data. Communications and Statistics – Theory and Meth-

ods 14, 1353–1359.

Sled, J.G., Zijdenbos, A.P., Evans, A.C., 1998. A nonparametric

method for automatic correction of tissue nonuniformity in MRI

data. IEEE Transactions of the Medical Imaging 17 (1), 87–97.

Styner, M., Brechbuhler, C., Szekely, G., Gerig, G., 2000. Parametric

estimate of intensity inhomogeneities applied to MRI. IEEE TMI

19 (3), 153–165.

Van Leemput, K., Maes, F., Vandermeulen, D., Suetens, P., 1999a.

Automated model-based bias field correction of MR images of the

brain. IEEE Transactions of the Medical Imaging 18, 885–896.

Van Leemput, K, Maes, F., Vandermeulen, D., Suetens, P., 1999b.

Automated model-based tissue classification of MR images of the

brain. IEEE Transactions of the Medical Imaging 18, 897–908.

Warfield, S.K., Kaus, M., Jolesz, F.A., Kikinis, R., 2000. Adaptive,

template moderated, spatially varying statistical classification.

Medical Image Analysis 4 (1), 43–55.

Wells, W.M., Kikinis, R., Grimson, W.E.L., Jolesz, F., 1996. Adaptive

segmentation of MRI data. IEEE Transactions of the Medical

Imaging 15, 429–442.

Zhai, G., Lin, W., Wilber, K., Gerig, G., Gilmore, J.H., 2003.

Comparison of regional white matter diffusion in healthy neonates

and adults using a 3T head-only MR scanner. Radiology 229, 673–

681.

466 M. Prastawa et al. / Medical Image Analysis 9 (2005) 457–466