statistical representation of high-dimensional deformation ... · statistics of deformation ﬁelds...

www.elsevier.com/locate/media

Medical Image Analysis 10 (2006) 740–751

Statistical representation of high-dimensional deformation fieldswith application to statistically constrained 3D warping

Zhong Xue *, Dinggang Shen, Christos Davatzikos

Section of Biomedical Image Analysis (SBIA), Department of Radiology, University of Pennsylvania, 3600 Market Street, Suite 380,

Philadelphia, PA 19104, United States

Received 14 January 2006; received in revised form 3 May 2006; accepted 23 June 2006Available online 2 August 2006

Abstract

This paper proposes a 3D statistical model aiming at effectively capturing statistics of high-dimensional deformation fields and thenuses this prior knowledge to constrain 3D image warping. The conventional statistical shape model methods, such as the active shapemodel (ASM), have been very successful in modeling shape variability. However, their accuracy and effectiveness typically drop dramat-ically in high-dimensionality problems involving relatively small training datasets, which is customary in 3D and 4D medical imagingapplications. The proposed statistical model of deformation (SMD) uses wavelet-based decompositions coupled with PCA in each wave-let band, in order to more accurately estimate the pdf of high-dimensional deformation fields, when a relatively small number of trainingsamples are available. SMD is further used as statistical prior to regularize the deformation field in an SMD-constrained deformableregistration framework. As a result, more robust registration results are obtained relative to using generic smoothness constraints ondeformation fields, such as Laplacian-based regularization. In experiments, we first illustrate the performance of SMD in representingthe variability of deformation fields and then evaluate the performance of the SMD-constrained registration, via comparing a hierarchi-cal volumetric image registration algorithm, HAMMER, with its SMD-constrained version, referred to as SMD+HAMMER. ThisSMD-constrained deformable registration framework can potentially incorporate various registration algorithms to improve robustnessand stability via statistical shape constraints.� 2006 Elsevier B.V. All rights reserved.

Keywords: Statistical shape model; Active shape model; Shape statistics; Wavelet packet transform; Deformable registration

1. Introduction

Representing prior statistical knowledge of high-dimen-sional scalar or vector fields is of fundamental importancein a variety of scientific areas including computationalanatomy, shape analysis, pattern recognition, and hypoth-esis testing applied to images or their deformations (Cooteset al., 1994, 1995; Staib and Duncan, 1992; Miller et al.,1997). For instance, statistical study of deformations canbe used to provide voxel-based morphological (VBM)characterization of different groups (Ashburner and Fris-ton, 2000); to incorporate prior knowledge of deformations

1361-8415/$ - see front matter � 2006 Elsevier B.V. All rights reserved.

doi:10.1016/j.media.2006.06.007

* Corresponding author.E-mail address: [email protected] (Z. Xue).

from training samples into image segmentation and regis-tration algorithms (Twining et al., 2005); to provide an effi-cient way of synthesizing new deformation fields forvalidation of registration and segmentation methods (Xueet al., 2005); to regularize deformations according to priorknowledge of sample deformations; and to estimate themissing parts of a deformation from the parts that areobserved. In fact, all these applications and the plethoraof automated methods for deformable registration of brainimages have necessitated the construction of a statisticalmodel that effectively captures the prior distribution ofhigh-dimensional deformation fields, in order to representthe true and full range of anatomical variability.

The goal of this paper is therefore to construct astatistical model of deformation fields from a limited num-ber of training samples. Although many statistical shape

mailto:[email protected]

Z. Xue et al. / Medical Image Analysis 10 (2006) 740–751 741

modeling methods have been proposed in literature, theyare often designed for 2D or 3D shapes that can be repre-sented by a relatively small number of landmarks or outlinepoints (Cootes et al., 1994, 1995). However, the highdimensionality of a variety of image warping methods of3D structural images renders global PCA-based methodsunable to properly estimate the statistics of deformationfields from the typically limited number of training samples(Mallat, 1998). This is especially true for the finer localdetail of a deformation field. In this paper, we propose astatistical model of deformation (SMD) that captures thestatistics of deformation fields between different individu-als. The basic premise throughout the paper is that a num-ber of deformation fields defined in the standard templateimage domain are available to be used for training, i.e.for estimation of the probability density function (pdf) ofa high-dimensional deformation field. Defining trainingsamples is not an easy issue, and its full treatment is beyondthe scope of this paper. For the purposes of demonstratingand testing our methodology, we used a number of defor-mations that were obtained using the high-dimensionaldeformable registration method, called HAMMER (Shenand Davatzikos, 2002). This inevitably biases the generateddeformations toward the family of deformations that canbe generated by this particular warping algorithm. This isin agreement with our goal here, which is to construct effec-tive statistical priors for constraining a deformable regis-tration mechanism, i.e. a procedure that can be appliedin conjunction with any registration algorithm. We showthat the performance of this registration algorithm is fur-ther improved after constraining it by this statistical model;the same could turn out to be true for other registrationalgorithms. Moreover, training samples can ultimately begenerated from first extensively labeling and landmarkinga number of images (Boesen et al., 2005), and then apply-ing the high-dimensional warping algorithms to theseimages, where these labels and landmarks act as the con-straints of the deformation fields. Such adequately con-strained warping algorithms are likely to generatedeformations that are close enough to a gold standard,and therefore appropriate for training.

Estimating the pdf of deformation fields from a limitednumber of training samples is a very challenging task. Oneof the most popular methods has been the application ofthe principal component analysis (PCA) (Cootes et al.,1995, 1998; Moghaddam and Pentland, 1997), in order toestimate a number of principal components that are fre-quently called principal modes of variation. However, thisapproach fails dramatically when applied to 3D densedeformation fields, due to under-training in practicalsettings. For example, accurately estimating a dense 3Ddeformation field of the entire brain could require tens ofthousands of training deformations, if not more. Thedramatic failure of standard methods for estimating covari-ance matrices and the associated pdfs has been well-knownin the signal estimation literature (Mallat, 1998). Accord-ingly, in the 1990s, methods based on scale-space decompo-

sitions were investigated. Our approach, referred to asSMD, builds upon the methods described in Davatzikoset al. (2003a,b), Coifman and Wickerhauser (1992),Mohamed and Davatzikos (2004); which uses wavelet-based decompositions coupled with PCA in each waveletband, in order to more accurately estimate pdfs of high-dimensional deformation fields, when only a relativelysmall number of training samples are available (e.g. tens,or in the order of 100).

After obtaining SMD, we use it as prior knowledge toconstrain the deformable registration. Compared to con-ventional registration methods which assume some genericsmoothness of deformation fields, the SMD-constraineddeformable registration can achieve more robust perfor-mance, because the regularization constraints reflect therelatively complex nature of the respective deformationfields. Our experiments first compare SMD with the con-ventional global PCA method. Then, the performance ofthe SMD-constrained registration is evaluated by compar-ing the high-dimensional deformable registration method,HAMMER, with its SMD-constrained version, referredto as SMD+HAMMER, via registering simulated and realMR brain images.

The rest of the paper is organized as follows: Section 2describes the statistical model of deformations and theSMD-constrained registration framework in detail. Exper-iments are carried out in Section 3. At last, Section 4 con-cludes the results.

2. Methods

In this section, we first describe how to use statisticalmodel of deformation (SMD) to estimate the statistics ofhigh-dimensional deformation fields that reflect inter-indi-vidual variability of brain structures. Then the frameworkof SMD-constrained registration is introduced in detail.

2.1. Statistical model of deformation

2.1.1. Description of SMD

Denoting f(x) as the deformation field defined in thetemplate image domain Xt, x 2 Xt, the objective is to esti-mate the pdf of f, i.e. p(f), from a relatively small numberof training samples. The commonly used PCA method(e.g. Cootes et al., 1994; Miller et al., 1997) fails miserablywhen f is of very high dimensionality. This is because a glo-bal PCA model is able to capture mainly global size andshape characteristics that are of limited interest and value,especially for capturing the variability of complex deforma-tions. In order to capture finer and more localized varia-tions of f, we follow and extend the framework proposedin Davatzikos et al. (2003a), which is referred to as thewavelet-based PCA (W-PCA) model. The W-PCA modeldecomposes f using the wavelet-packet transform (WPT)and subsequently captures within-scale statistics via hierar-chically organized PCA models. These PCA models areestimated from statistical distributions that are both of

Fig. 1. Jacobian determinants, reflecting volumetric changes, could beconceptually expected to be less variable than the displacement fields. Forexample, the displacement fields of the precentral gyri are very different forthese two brains, whereas the volumes of these gyri are very similar,leading to much less variable Jacobian determinants.

742 Z. Xue et al. / Medical Image Analysis 10 (2006) 740–751

lower dimensionality, and more compact due to correla-tions among variables (e.g. the PCA model derived froma high-scale representation of f represents a very compactdistribution due to the smoothing and down-samplingapplied at each level of the wavelet-packet decomposition;the distribution of high frequency detail within a local win-dow is also easier to estimate due to its low dimensionalityowing to the small window size). Performing PCA withineach band at a given scale is important, due to correlationsamong wavelet coefficients corresponding to adjacent loca-tions, something which is particularly prominent in smoothelastic-type of deformations, in contrast to, for example,acoustic signals in which wavelet coefficients are typicallyassumed to be statistically independent. The fundamentalassumption in W-PCA is that the wavelet-based rotationrenders the covariance matrix of f close to block-diagonal,thereby enabling a more accurate estimation from a limitedset of examples, compared to the usual sample covarianceestimation.

In theory, if the W-PCA model described above capturesthe statistics of deformation f accurately, we can just use itas the statistical model. In practice, however, the assump-tion that the covariance matrix of f is block-diagonal inthe wavelet-packet basis does not hold exactly. Althoughit is well-known that for broad classes of signals, correla-tions across different scales diminish rapidly, they are none-theless non-negligible for adjacent scales. In order toalleviate this problem, we observe that additional con-straints imposed on the estimated deformation fields canbe used to define subspaces in which the deformation mustbelong to. Therefore, we require that a valid deformationfield simultaneously satisfies all available constraints, i.e.it belongs to the intersection of a number of subspaces,each of which satisfies some constraints on the deforma-tion. The W-PCA model applied to the deformation fieldspecifies one such subspace.

In order to describe the second subspace, we use theW-PCA model of the Jacobian determinants of the defor-mation fields, since they reflect local volumes of anatomicalstructures, which are important from the perspective ofspatial distribution of the amount of brain tissue. Fig. 1highlights two precentral gyri from different subjects, whichdiffer quite dramatically in shape, but not in volume. Moregenerally, it would be reasonable to assume that, althoughthe cortical folding patterns can vary wildly across individ-uals, the need of different cortical structures to occupy cer-tain tissue volume renders the Jacobian determinant, whichis directly related to tissue volume, relatively little variableacross individuals, and therefore it would be easier to esti-mate the statistics of Jacobian determinants from a limitednumber of samples. It is worth noting that the complemen-tary property of the statistical models of deformation fieldsand those of the Jacobian determinants allows us to com-bine them together by requiring that a valid deformationfield be within the intersection of the subspaces definedby them. Therefore, a valid deformation field has to be iter-atively constrained by both of the statistical models. We

note, here, that converting a constraint on the Jacobiandeterminant to a constraint on the deformation field, i.e.finding the displacement field that satisfies certain condi-tions on the Jacobian determinant is not straightforward,and it does not have a unique solution. Herein we useKaracali and Davatzikos (2004), which utilizes an iterativeprojection scheme that minimally, according to some dis-tance criteria, modifies a given displacement field so thatit satisfies certain conditions on the Jacobian determinant.This algorithm is used to realize the projection of a givendisplacement field to the subspace of ‘‘valid Jacobians’’.

The third subspace is represented by a nested Markovrandom field (MRF) regularization, which imposes spatialsmoothness at different scales in conjunction with theinverse WPT. The purpose of MRF regularization is toeliminate potential discontinuities emanating from theassumption of independence across wavelet bands.

In the following subsections, we first describe theW-PCA model, which can be used to effectively capturethe statistics of both deformation fields and their Jacobiandeterminants. Then, we introduce the MRF regularizationin detail. Finally, we summarize the algorithm to regularizedeformation fields using SMD.

2.1.2. The wavelet-PCA (W-PCA) models for estimating the

pdfs of deformation fields and their Jacobian determinants

The W-PCA model is used to estimate the pdf of defor-mation fields, i.e. p(f), using N samples. It first applies anL-level WPT to f and then constructs a PCA model ofthe wavelet coefficients of each wavelet band at level L,and finally it combines the pdfs of difference wavelet bandstogether. Fig. 2 illustrates the structure of 2-level 1D WPT.For 3D WPT, the wavelet coefficients at level l are repre-sented by w(l,b), b = 0,1, . . .,Bl � 1, where Bl = 8l andl = 1,2, . . .,L. At each level, w(l,0) always represents thelow-pass wavelet coefficients. For simplicity, f is alsoreferred to as w(0,0).

After L-level WPT, f can be represented by all the wave-let coefficients at level L, i.e. w(L,b). Assuming that different

Fig. 2. Wavelet-packet transform (WPT).


bands in the wavelet subspaces are independent, W-PCAestimates the pdf of f as

pWPCAðfÞ ¼YBL�1

b¼0

pðwðL;bÞÞ: ð1Þ

The pdf of each band (L,b), p(w(L,b)), can be estimated byapplying PCA to the wavelet coefficients of N deformationfield samples in that wavelet band, denoted as wðL;bÞs ,s = 1,2, . . .,N. After performing PCA, we obtain the meanof the wavelet coefficients �wðL;bÞ and the matrix U(L,b)

formed by the K(L,b) eigenvectors of the covariance matrixof these wavelet coefficients corresponding to the K(L,b)

largest eigenvalues kðL;bÞj , j = 1, . . .,K(L,b), of that covariance

matrix. Therefore, w(L,b) can be represented by its projectedvector or feature vector v(L,b) in the space spanned by theK(L,b) eigenvectors,

vðL;bÞ ¼ UðL;bÞTðwðL;bÞ � �wðL;bÞÞ: ð2Þ

Then, the pdf of f in Eq. (1) is represented by

pWPCAðfÞ ¼YBL�1

b¼0

cðL;bÞ exp �XKðL;bÞj¼1

vðL;bÞ2

j

2kðL;bÞj

( ); ð3Þ

where c(L,b) is the normalization coefficient and vðL;bÞj is thejth element of feature vector v(L,b).

This W-PCA model can not only be applied to deforma-tion fields, but also to the Jacobian determinants of defor-mations, thus, we can capture the statistics of deformationfields via their Jacobian determinants,

pJ ðfÞ ¼ pWPCAðJðfÞÞ; ð4Þwhere JðfðxÞÞ ¼ ofðxÞ

oxT

�� represents the operator to calculatethe Jacobian determinant of a deformation field f.

We stress the importance of using PCA within eachwavelet band, which is in contrast to the commonly usedindependence assumption for wavelet coefficients. In par-ticular, PCA is known to be the optimal linear expansion,provided that a good estimate of the covariance matrix isavailable. Although sample covariance is a very inaccurateestimate of the covariance of f (Davatzikos et al., 2003a),the sample covariance at various scales provides a muchbetter estimate of the covariance at that scale, because ofthe relatively low dimensionality of each wavelet band(see discussion in Section 2.1.1). As a result, the W-PCAmodel can capture correlations between adjacent spatiallocations at a given scale.

2.1.3. Hierarchical MRF regularization

As mentioned in Section 2.1.1, one W-PCA model ofdeformation fields does not fully capture the region of validdeformation fields: if deformations are synthesized directlyusing the W-PCA model, some unrealistic discontinuitiesemanating from the assumption of independence acrosswavelet bands may occur. In order to eliminate such poten-tial discontinuities, a nested Markov random field (MRF)regularization scheme that imposes spatial smoothness atdifferent scales is applied in conjunction with the inverseWPT.

Suppose L-level WPT is performed, and let level L bethe lowest resolution level, in inverse WPT, all the waveletbands at level L are used to reconstruct the wavelet coeffi-cients at the higher level L � 1. Then, the newly recon-structed low-pass coefficient at level L� 1; wðL�1;0Þ, isregularized using MRF, which eliminates the independen-cies of the wavelet coefficients of different bands at levelL. Similarly, all the wavelet coefficients at level L � 1 willbe used to reconstruct the wavelet coefficients at levelL � 2 by performing inverse WPT, after which the newlyreconstructed low-pass wavelet coefficient at levelL� 2; wðL�2;0Þ, is regularized using MRF. This eliminatesthe independencies of the wavelet coefficients of differentbands at level L � 1. In this way, we repeat the same pro-cess from the lowest resolution to the finest resolution.That is, at each level l (l = L � 1, L � 2, . . ., 0), after recon-structing all the wavelet coefficients, the low-pass waveletcoefficient at level l; wðl;0Þ, is regularized by performingMRF to eliminate some discontinuity caused by the inde-pendencies of those different wavelet bands at level l + 1.Finally, the regularizations at different resolution levelspropagate to the finest resolution, thus results a hierarchi-cally smoothed deformation field. In the following para-graph, we describe how MRF regularization is performedto a low-pass wavelet coefficient.

Denoting the input low-pass coefficient as w (which canbe any of wðl;0Þ; l ¼ L� 1; . . . ; 0), the MRF regularizationestimates a ‘‘true’’ wr, by assuming that the low-pass wave-let coefficient forms a MRF and w is a degraded observa-tion of wr (w ¼ wr þ n, where n is the disturbanceassumed to be zero-mean Gaussian noise with standarddeviation (std) rN), and by using the Maximum a posteriori

(MAP) framework (Geman and Geman, 1984; Shen andIp, 1998),

wr ¼ argmaxwfpðwjwÞg¼ argmaxwfpðwjwÞpðwÞ=pðwÞg: ð5Þ

Assuming the prior p(w) and the likelihood pðwjwÞ areGaussian distributions, we have p(w) � exp{�W(w)} and

pðwjwÞ / exp � 12r2

Njjw� wjj2

n o, where WðwÞ ¼ 1

2ðw� �wÞT

v�1ðw� �wÞ. �w and v refer to the mean and the covariancematrix of w, respectively, and the structure of v meets theMRF property. Thus, wr is solved by minimizing an energyfunction, Er(w),

ConventionalRegistration

SMDRegularization

Result

Template Image

Subject Image

Image Deformation Field

Fig. 3. The structure of the SMD-constrained deformable registration,wherein the deformation from the template to the subject image generatedby a conventional registration algorithm is iteratively regularized usingSMD.


ErðwÞ ¼1

2r2N

jjw� wjj2 þWðwÞ: ð6Þ

We use a simplified approach similar to Shen and Ip (1998)and Belge et al. (2000) to minimize Er(w). First, we estimatep(w) as a product of all the local (marginal) pdfs across thelocations x, i.e. p(w) = �xG(wx,lx,rx), where G(,,) repre-sents a single Gaussian distribution with mean lx and stan-dard deviation rx. Then W(w) in Eq. (6) is estimated by

WðwÞ ¼P

xjjwx�lxjj2

2r2x

n o, where lx ¼ 1

jdðxÞjP

y2dðxÞwy and

r2x ¼ 1

jdðxÞj�1

Py2dðxÞjjwy � lxjj

2. d(x) refers to a neighbor-hood centered on x but not including x, and |d(x)| is thecardinality of d(x). Therefore, the regularized wavelet coef-ficients wr can be obtained by minimizing Eq. (6) using theNewton’s method.

2.1.4. Summary of SMD

SMD combines the W-PCA models of deformationfields and their Jacobian determinants, as well as theMRF regularization together, and requires that a validdeformation field locates inside the intersection of the threesubspaces defined by these models. Therefore, given aninput deformation field, we can use the following steps toiteratively project it onto each of the three subspaces, andfinally generate the SMD-regularized deformation fieldaccording to the priors. This SMD regularization algo-rithm is summarized as follows,

Step 1. Project the deformation field onto the W-PCAmodel of valid deformation fields.

Step 2. Project the Jacobian of the deformation field ontothe W-PCA model of valid Jacobian determinants.

Step 3. Find new deformation field whose Jacobianmatches the one generated in Step 2, using Kara-cali and Davatzikos (2004).

Step 4. Apply the nested MRF regularization to imposespatial smoothness on the deformation at allscales.

Step 5. Go to step 1 and iterate until the smoothed defor-mation field belongs to the subspaces of valid Jac-obians and deformations.

2.2. SMD-constrained deformable registration

After estimating the statistical model of deformationfields, we can use this prior knowledge to constrain thedeformation fields during image registration. The struc-ture of the SMD-constrained deformable registration isshown in Fig. 3. Unlike the traditional registration algo-rithms, the SMD-constrained registration improves therobustness and stability of the registration results sinceprior statistical information about the variability of defor-mation fields has been embedded into the registrationprocedure.

Particularly, the SMD-constrained deformable registra-tion is achieved by iteratively regularizing the results of a

conventional registration method. Compared to the conven-tional registration methods that only utilize the smoothnessconstraint of deformation term and the image-similarityterm, the SMD-constrained registration uses a new statisti-cal regularization on the deformation field so that it con-forms to the prior knowledge defined by SMD. In the firstiteration, we use the registration algorithm to estimate anew deformation field between the template image and thesubject image, and then regularize the deformation fieldusing SMD (the SMD regularization algorithm is describedin Section 2.1.4); in the subsequent iterations, the regular-ized deformation field is used as the input of the registrationalgorithm in order to iteratively refine the registration result.The SMD-constrained registration terminates until the dif-ference between the resultant deformation fields of two sub-sequent iterations drops below a certain threshold. Sincesmoothness of deformation field has been imposed fromthe SMD, in the traditional registration algorithm, a smallerweight for smoothness constraint is chosen, to prevent fromover smoothing the deformation field.

In the experiments, we use the HAMMER registrationalgorithm as the registration part and SMD as the regular-ization part. This approach is referred to as SMD+HAM-MER. Although SMD+HAMMER may not minimize theoverall energy function globally, we could generate morerobust registration results conforming to the prior knowl-edge of deformation fields by iteratively regularizing thedeformation field estimated by HAMMER, without chang-ing its mechanism. In other words, we treat the family of allpossible deformation fields generated by HAMMER as thefourth subspace. Then, the registration result ofSMD+HAMMER is obtained by iteratively projectingthe current deformation field onto each of the four sub-spaces until the resultant deformation field locates in theintersection of all these subspaces.

3. Experimental results

In this section, we first show that SMD is more effectivein capturing the statistics of high-dimensional deformationfields than the conventional global PCA used by activeshape models and related approaches. Evaluation is thenperformed to illustrate that SMD+HAMMER is morerobust than HAMMER in registering both simulated andreal MR brain images, without decrease in registrationaccuracy.


3.1. Performance of SMD in capturing the statistics of

deformations

The ability of SMD to capture the statistical variation ofdeformation fields resulting from brain image warping isfirst tested by simulating realistic brain images, as well asby measuring generalization ability, i.e. ability to representnew samples.

Simulating brain images. Simulated deformations aregenerated using random sampling from the pdf estimatedby SMD. Training was achieved using 158 brain imagesfrom the Baltimore longitudinal study of aging (BLSA)(Resnick et al., 2000). Since PCA is performed for eachwavelet band in the W-PCA model, we simulate imagesby randomly selecting the PCA coefficients of differentwavelet bands according to their estimated distributions.Fig. 4 shows examples of the images derived from simu-lated deformations. In this case, the feature vectors of eachwavelet band are randomly sampled to be around ±2 timesof the corresponding variances. These simulated imagesallow us to visually appreciate the characteristics of thedeformations captured by SMD.

Generalization. In order to quantitatively evaluate theperformance of SMD, we use a generalization measure(Styner et al., 2003) to compare SMD with the conven-tional global PCA method, by using a leave-one-outcross-validation. First, a statistical model is constructedby leaving one deformation field out of the training set.Then, the left-out deformation field is projected onto theconstructed statistical model by using the SMD regulariza-tion algorithm in Section 2.1.4, thus obtaining a new defor-mation field in the statistical space of deformations andalso the difference between these two deformation fields.Finally, by averaging all differences on all leave-one-outcases, the generalization error is finally obtained. Low gen-eralization error means good generalization and alsoimplies that the statistical model is not only able to accu-rately represent the training samples, but also to accuratelyrepresent other testing samples. Experimental results showthat the average and the standard deviation of the general-ization errors are 2.3 and 0.6 mm, respectively, for SMD,which is much smaller than those using conventionalPCA (Xue et al., 2005). Therefore, SMD can capture the

Fig. 4. Examples of randomly si

statistics of high-dimensional deformation fields moreaccurately than the conventional global PCA method.

3.2. Performance of SMD-constrained deformable

registration

In this subsection, the performance of the SMD-con-strained registration (SMD+HAMMER) is evaluated onboth simulated and real MR brain images, and it is com-pared with that of HAMMER. In the first experiment,we examine the ability of the two registration methods todetect morphological differences (atrophy) simulated intwo groups of brain scans, by examining quantitiesextracted from the deformation fields linking a templatewith each image in these two groups, i.e. using a computa-tional anatomy approach (Davatzikos et al., 2001). More-over, other measurements, such as smoothness ofdeformation fields, are also compared. Two additionalexperiments are performed on real data. First, the registra-tion accuracy of each registration method is measured by anumber of expert-defined anatomical landmarks. Second,serial images of the same subjects are aligned onto the tem-plate space, and the temporal consistency/smoothness ofthe registration results is measured quantitatively and alsovisually for each registration method. The premise in thelast experiment is that the transformations that yield tem-porally consistent results are likely to be more accurate,since the input images are acquired from the same individ-ual in consecutive years, and they differ very little from yearto year.

3.2.1. Registration of simulated atrophy images

In this experiment, we use both HAMMER andSMD+HAMMER to register the MR brain images of dif-ferent subjects with and without simulated atrophy. 10T1-weighted MR brain images of 10 different subjects areused, referred to as the group of original brains. For eachoriginal image, we simulate the atrophy on both precentralgyrus and superior temporal gyrus (Davatzikos et al.,2001). Therefore, we obtain 10 images with simulated atro-phy, referred to as the group of simulated brains. Fig. 5shows an example of the original MR brain image and itssimulated atrophy image.

mulated images using SMD.


All of these 20 images, i.e. 10 in the original group and10 in the simulated group, are then registered onto the tem-plate image by using HAMMER and SMD+HAMMER,respectively. In order to evaluate the performance of theregistration results, we perform the following quantitativecomparisons.

3.2.1.1. Image intensity difference and smoothness of defor-

mation field. We have used the average image intensity dif-ference and the smoothness of deformation field toevaluate the performance of the registration, since thesevalues give us an idea about the goodness of registration.It should be noted that good registration results obtainedfrom different registration methods should have similarvalues for both image intensity difference and the smooth-ness of deformation field. In this paper, the average imageintensity difference between the registered subject imageand the template image is calculated by

eðI t; Is; fÞ ¼1

jXtjXx2Xt

ðI tðxÞ � IsðfðxÞÞÞ2 ð7Þ

and the smoothness of deformation field is evaluated usingthe histogram of the Jacobian determinants, as well as thehistogram of the Laplacian magnitudes of that deforma-tion field.

Fig. 6 shows the average image intensity differencebetween the template image and each registered subjectimage. It can be seen that the average image intensity dif-ference is similar for both HAMMER and SMD+HAM-MER registration results, and there is no significantdifference between the results of these two registrationmethods. That means the registration accuracy is not

Fig. 5. Examples of the original MR brain image, the segmented image andtemporal gyrus. (a) Original image; (b) segmented image; (c) segmented i(e) segmented image; (f) segmented image with simulated atrophy on superior

decreased by using SMD as additional constraints fordeformation field. In addition, it can be observed that theaverage image intensity difference of affine registrationmethod is always the largest for each subject, as expected.Notice that the average image intensity difference is stilllarge even for the HAMMER and SMD+HAMMER algo-rithms. This is because the image intensities are not glob-ally normalized between the template and each subjectimage, when calculating the average image intensitydifference.

Figs. 7 and 8 compare the histograms of Jacobian deter-minants and the histograms of the Laplacian magnitudes ofthe deformation fields generated by HAMMER andSMD+HAMMER, respectively. It can be seen that forSMD+HAMMER, the Jacobian determinants are moretightly distributed around one, and the Laplacian magni-tudes are relatively small, compared to HAMMER.

The resultant deformation fields can also be visuallyobserved in Fig. 9. We can see that HAMMER andSMD+HAMMER generate similar deformation fields forthe same template and subject image pair, but theSMD+HAMMER result is relatively smooth. Moreover,we can also see that in some locations pointed by thearrows, the deformation field is clearly regularized bySMD+HAMMER, as compared with the HAMMERresult.

All of these results indicate that by incorporating SMDinto the registration procedure, we can obtain relativelysmooth deformation fields, without decreasing registrationaccuracy. Further evaluation of registration accuracy isalso performed by measuring the accuracy in aligning man-ual landmarks in real image as provided in Section 3.2.2.

the simulated image with local atrophy on precentral gyrus and superiormage with simulated atrophy on precentral gyrus; (d) original image;temporal gyrus.

Comparison of Average Image Intensity Differences

0

5

10

15

20

25

30

35

1 2 3 4 6 7 95 8 10 11 12 13 14 15 16 17 18 19 20Image

Ave

rage

Int

ensi

ty D

iffe

renc

e

HAMMER

SMD+HAMMERAffine Transformation

Fig. 6. Comparison of average image intensity difference between the template image and the registered subject images.


3.2.1.2. Ability of detecting simulated atrophy. We then useda computational anatomy method to detect group differ-ences between the brains with atrophy and the originalones. In particular, for each brain image, we can calculatethe tissue density maps for different brain tissues in thetemplate image space, i.e. the RAVENS maps (Resnicket al., 2000; Goldszal et al., 1998; Davatzikos, 1998) of graymatter (GM), white matter (WM), and the cerebrospinalfluid inside the ventricle (VN), from the deformation fieldthat registers this brain onto the template space. These tis-sue density maps are created in a volume-preserving way,so they directly reflect the regional volumetric structureof the respective brains. For example, if an individual’sventricles are deformed into conformation with a tem-

Comparison of Histograms of Jacobian Determinants

0.0%

0.5%

1.0%

1.5%

2.0%

0 1 2 3 4Jacobian Determinant

Perc

enta

ge

HAMMER

SM D+ HAMMER

Fig. 7. Comparison of Jacobian determinants of deformation fields.

0 1 2 3 4 5 6 7

Comparison of Histograms of Laplacian Magnitudes

0.0%

0.1%

0.2%

0.3%

0.4%

0.5%

Laplacian Magnitude

Perc

enta

ge

HAMMER

SMD+HAMMER

Fig. 8. Comparison of the Laplacian magnitude of deformation fields.

plate’s ventricles, local expansion or contraction changesthe local density of CSF. Local contraction, i.e. trying tomatch big subject ventricle to the small template ventricle,increases the local density. This is also true for GM andWM structures, as well as for arbitrary subdivisions ofthem. Since the original information about volumes ofbrain structures and any arbitrary partitions of them isconverted into tissue densities, and since these tissue den-sity maps are registered, local differences or changes in vol-umes can be quantified by respective changes in the tissuedensity maps. Fig. 10 shows the examples of the brain tis-sue density maps for GM, WM, and VN, calculated fromHAMMER and SMD+HAMMER results, respectively.It can be seen that relatively smooth tissue density mapswere obtained by SMD+HAMMER. Our assumption isthat these smooth maps would remove unwanted ‘‘noise’’and would allow us to better detect the atrophy.

In order to test for group differences, we performed apaired t-test on the combined brain density maps of thetwo groups (each group includes 10 brain density maps cal-culated from 10 respective deformation fields), using thestatistical parametric mapping (SPM) software package.A smaller p-value or a larger t-value of a paired t-test willindicate better separation ability. Table 1 shows the statis-tical measures for the two clusters detected in the locationsof the superior temporal gyrus and the precentral gyrus,respectively. It can be seen that smaller p-values (both ofpFWE-corr and pFDR-corr) and larger t-values are obtainedfor the SMD+HAMMER algorithm. In fact, as shown inFig. 10, the brain density maps of GM, WM, and VN cal-culated from SMD+HAMMER results are smoother thanthose from HAMMER results, which also agrees with theresults shown in Figs. 8 and 9. Therefore, SMD+HAM-MER in these experiments was much more powerful indetecting group differences.

3.2.2. Registration of real MR brain data3.2.2.1. Measuring registration accuracy by manual land-

marks. We used 18 MR brain images from different

Fig. 9. Examples of the deformation fields generated by HAMMER and SMD+HAMMER, respectively. (a) HAMMER; (b) SMD+HAMMER.


subjects, each of which had 20 landmarks manuallymarked by experts. After registering these 18 images, wetransformed all the manual landmarks onto the templatedomain and thus obtained 20 groups of correspondingpoints in the template domain, and each group consistsof 18 points of the same landmark from 18 different subjectimages. For each group, we calculated the mean and std ofthe distances between all the 18 points and their averagepoint; for ideal registration, these should be zero. Thesemean and std for each landmark group are shown in

Fig. 10. Examples of the RAVENS maps for GM, WM and VN, calculated frorespectively. (a) HAMMER; (b) SMD+HAMMER.

Fig. 11. It can be seen that all the means are below5 mm, and similar registration accuracy was achieved forHAMMER and SMD+HAMMER (p-value for a pairedt-test is 0.023). Although the results of HAMMER andSMD+HAMMER are comparable and they have nosignificant difference, the registration accuracy on mostmanual landmarks for HAMMER is slightly higher thanthat of SMD+HAMMER, this may be because that theSMD trained could not completely represent new deforma-tions, and the resultant deformation fields for new subject

m the deformation fields generated by HAMMER and SMD+HAMMER,

Table 1Paired t-test results: p-values and t-values at the peaks of clusterscorresponding to the superior temporal gyrus and the precentral gyrus

HAMMER SMD+HAMMER

Cluster 1 (in thesuperior temporal gyrus)

pFWE�corr = 0.342 pFWE�corr = 0.026pFDR�corr = 0.057 pFDR�corr = 0.003T = 12.91 T = 17.50

Cluster 2 (in theprecentral gyrus)

pFWE�corr = 0.272 pFWE�corr = 0.003pFDR�corr = 0.057 pFDR�corr = 0.003T = 13.26 T = 22.29


images have to be subjected to SMD regularization.Finally, we notice that these results are comparable tothe inter-rater errors in placing landmarks, i.e. the meanand std are 5.6 and 3.6 mm, respectively, for the sametesting data (with voxel size of 1 mm · 1 mm · 2.2 mm)(Xue et al., 2004).

3.2.2.2. Registering serial MR brain images. In this experi-ment, serial images of six different subjects from the BLSAproject (Resnick et al., 2000) are registered onto the tem-plate image by HAMMER and SMD+HAMMER, respec-tively, to compare the goodness of registration. In theexperiment, each subject has eight serial images capturedfrom eight consecutive years. For accurately measuringthe subtle longitudinal changes, it is important to evaluatethe temporal consistency of the registration results on the

Mean and std in Aligning Manual Landmarks

0

2

4

6

8

10

1 2 3 4 6 75 8 9 10 11 12 13 14 15 16 17 18 19 20Landmark

Dis

tanc

e (i

n m

m)

HAMMER

SMD+HAMMER

Fig. 11. Comparison of the registration accuracy on manual landmarks.

Fig. 12. Comparison of temporal smoothness of serial deformation fields. TSMD+HAMMER by the TS map of HAMMER. (a) HAMMER; (b) SMD+image.

serial images of each subject. Therefore, the temporalsmoothness (TS) of the T serial deformation fields, usedto register the serial images of the same subject to the tem-plate, is measured,

TSðxÞ ¼ 1

T � 2

XT�1

t¼2

f tðxÞ �f tþ1ðxÞ þ f t�1ðxÞ

2

��: ð8Þ

A smaller TS value at x means the deformation fields alongthe corresponding voxels of x in the serial images are tem-porally smooth, while a large TS value means that defor-mation fields are not temporally smooth. Notice thattemporal consistency or temporal smoothness is an impor-tant indicator of method robustness in many studies, inwhich relatively small brain changes are known to have oc-curred between consecutive scans. Here, we exploit this factto test the stability and robustness of our registration meth-od. Fig. 12 gives an example of the TS map. Fig. 12(a)shows the TS map calculated from the serial deformationfields generated by using HAMMER, and Fig. 12(b) givesthe TS map by using SMD+HAMMER. It can be seenthat the TS map in Fig. 12(b) has smaller values than thatin Fig. 12(a), e.g. smaller peaks and lower peak values areobserved. We can also calculate the difference between theTS map of SMD+HAMMER and that of HAMMER, andoverlay this difference of TS maps upon the template im-age, as shown in Fig. 12(c). According to the color bar,we can observe that SMD+HAMMER can achieve betterresults within the red regions and similar results within theyellow regions.

In fact, most values in the difference of TS maps are neg-ative, indicating that SMD+HAMMER generates tempo-rally smoother serial deformation fields for the serialimages than HAMMER. Alternatively, we can calculatethe histogram of the difference of TS maps, to see whetherSMD+HAMMER improves the temporal smoothness.Fig. 13 shows six such histograms for six different subjects.It can be seen that for each subject, most values of thedifference of the TS maps are below zero, thus most TSvalues of SMD+HAMMER are smaller than those of

he difference of TS maps is defined as the subtraction of the TS map ofHAMMER; (c) overlaying the difference of TS maps upon the template

Histograms of the Difference of TS Maps for Different Subjects

0%

10%

20%

30%

-4 -3 -2 -1 0 0 0

Difference of TS Maps

Perc

enta

ge

Subject 1

Subject 2

Subject 3

Subject 4

Subject 5

Subject 6

Zero

Fig. 13. Histograms of the difference of TS maps for six different subjects.

Fig. 14. Warping consistency on eight-year serial images of the same subject. (aserial images by SMD+HAMMER.


HAMMER. Thus, compared to HAMMER,SMD+HAMMER produces temporally smoother registra-tion results for serial images.

The temporally consistent registration results bySMD+HAMMER can be also visually observed via 3Drendering. Fig. 14 shows the template image and the regis-tered serial images of the same subject in the templatespace, using a 3D rendering, for HAMMER andSMD+HAMMER, respectively. White contours are iden-tically placed in each image, for facilitating the visualinspection. It can be seen that the shapes of the deformedgyri for the year 1, year 3 and year 5 of the HAMMER reg-istration results (indicated by black arrows in Fig. 14(b))

) Template image; (b) registered serial images by HAMMER; (c) registered


are quite different from those in the other years. In con-trast, the shapes of the registered gyri by SMD+HAM-MER are not only more temporally consistent, but alsomore similar with the template as shown in Fig. 14(c).

4. Conclusion

A statistical model of deformation (SMD) is proposed inthis paper to capture the statistics of high-dimensionaldeformation fields effectively. SMD uses the W-PCA modelto estimate the pdf of high-dimensional deformation fieldsand represents the space of valid deformation fields as anintersection of three subspaces that reflect different aspectsof deformation fields, i.e. W-PCA model of deformationfields, W-PCA model of Jacobian determinants of defor-mation fields, and a nested MRF smoothness regulariza-tion. Compared to the conventional statistical shapemodels, such as the active shape model (ASM), SMD cancapture the statistics more accurately and effectively, espe-cially for high-dimensional data and small sample sets.SMD is further used as the prior knowledge to regularizethe deformation fields in an SMD-constrained registrationframework. Experiments demonstrate that more robustregistration results are obtained by the SMD-constrainedregistration, without decreasing the registration accuracy.Therefore, SMD can potentially be incorporated into var-ious registration algorithms to improve their robustnessand stabilities via statistically based regularization.

References

Ashburner, J., Friston, K.J., 2000. Voxel-based morphometry: themethods. Neuroimage 11, 805–821.

Belge, M., Kilmer, M., Miller, E., 2000. Wavelet domain image restorationwith adaptive edge-preserving regularization. IEEE Trans. ImageProcess. 9, 597–608.

Boesen, K., Frey, S., Huang, J., Germann, J., Stern, J., Collins, D.L.,Evans, A.C., Rottenberg, D., 2005. Inter-rater Reproducibility of 3DCortical and Subcortical Landmark Points. In: 11th Annual Meetingof the Organization for Human Brain Mapping, Toronto, Canada,2005.

Coifman, R.R., Wickerhauser, M.V., 1992. Entropy-based algorithms forbest basis selection. IEEE Trans. Inf. Theory 38, 713–718.

Cootes, T.F., Hill, A., Taylor, C.J., Haslam, J., 1994. Use of active shapemodels for locating structures in medical images. Image VisionComput. 12, 355–365.

Cootes, T.F., Taylor, C., Cooper, D., Graham, J., 1995. Active shapemodels – their training and application. Comput. Vis. Image Und. 61,38–59.

Cootes, T., Edwards, G., Taylor, C., 1998. Active appearance models. In:Fifth European Conference on Computer Vision, Freiburg, Germany,1998.

Davatzikos, C., 1998. Mapping of image data to stereotaxic spaces.Human Brain Mapp. 6, 334–338.

Davatzikos, C., Genc, A., Xu, D., Resnick, S.M., 2001. Voxel-basedmorphometry using the RAVENS maps: methods and validation usingsimulated longitudinal atrophy. NeuroImage 14, 1361–1369.

Davatzikos, C., Tao, X., Shen, D.G., 2003a. Hierarchical active shapemodels using the wavelet transform. IEEE Trans. Med. Imaging 22,414–423.

Davatzikos, C., Tao, X., Shen, D., 2003. Applications of wavelets inmorphometric analysis of medical images. In: SPIE Wavelets X, SanDiego, CA, 2003.

Geman, S., Geman, D., 1984. Stochastic relaxation Gibbs distributionsand Bayesian restoration of images. IEEE Trans. Patt. Anal. Mach.Intell. PAMI-6, 721–741.

Goldszal, A.F., Davatzikos, C., Pham, D., Yan, M., Bryan, R.N.,Resnick, S.M., 1998. An image processing protocol for the analysis ofMR images from an elderly population. J. Comput. Assist. Tomo. 22,827–837.

Karacali, B., Davatzikos, C., 2004. Estimating topology preserving andsmooth displacement fields. IEEE Trans. Med. Imaging 23, 868–880.

Mallat, S., 1998. A Wavelet Tour of Signal Processing. Academic Press.Miller, M., Banerjee, A., Christensen, G., Joshi, S., Khaneja, N.,

Grenander, U., Matejic, L., 1997. Statistical methods in computationalanatomy. Stat. Method Med. Res. 6, 267–299.

Moghaddam, B., Pentland, A., 1997. Probabilistic visual learning for objectrepresentation. IEEE Trans. Patt. Anal. Mach. Intell. 19, 696–710.

Mohamed, A., Davatzikos, C., 2004. Shape representation via bestorthogonal basis selection. In: Medical Image Computing and Com-puter Assisted Intervention(MICCAI), St. Malo, France, 2004.

Resnick, S.M., Goldszal, A., Davatzikos, C., Golski, S., Kraut, M.A.,Metter, E.J., Bryan, R.N., Zonderman, A.B., 2000. One-year agechanges in MRI brain volumes in older adults. Cereb. Cortex 10, 464–472.

Shen, D., Davatzikos, C., 2002. HAMMER: hierarchical attributematching mechanism for elastic registration. IEEE Trans. Med.Imaging 21, 1421–1439.

Shen, D., Ip, H., 1998. Markov random field regularization models foradaptive binarization of nonuniform images. IEE Proc. Vis. ImageSignal Process. 145, 322–332.

Staib, L.H., Duncan, J.S., 1992. Boundary finding with parametricallydeformable models. IEEE Trans. Patt. Anal. Mach. Intell. 14, 1061–1075.

Styner, M.A., Rajamani, K.T., Nolte, L.P., Zsemlye, G., Szkely, G.,Taylor, C.J., Davies, R.H., 2003. Evaluation of 3D correspondencemethods for model building. In: Information Processing in MedicalImaging, Ambleside, UK, 2003.

Twining, C.J., Cootes, T., Marsland, S., Petrovic, V., Schestowitz, R.,Taylor, C.J., 2005. A unified information-theoretic approach togroupwise non-rigid registration and model building. In: InformationProcessing in Medical Imaging, Glenwood Springs, CO, 2005.

Xue, Z., Shen, D., Davatzikos, C., 2004. Determining correspondence in3D MR brain images using attribute vectors as morphologicalsignatures of voxels. IEEE Trans. Med. Imaging 23, 1276–1291.

Xue, Z., Shen, D., Karacali, B., Davatzikos, C., 2005. Statisticalrepresentation and simulation of high-dimensional deformations:application to synthesizing brain deformations. In: Medical ImageComputing and Computer Assisted Intervention (MICCAI 2005),Palm Springs, CA, USA, 2005.

statistical representation of high-dimensional deformation ... · statistics of deformation ﬁelds...

Documents