extraction of 3d anatomical point landmarks based on - kogs

Pattern Recognition 32 (1999) 3—15

Extraction of 3d anatomical point landmarks basedon invariance principles

Karl Rohr

Universitat Hamburg, Fachbereich Informatik, Arbeitsbereich Kognitive Systeme, Vogt-Kolln-Str. 30, 22527 Hamburg, Germany

Received 28 October 1997, in revised form 20 April 1998; received for publication 12 February 1998

Abstract

We describe 3D operators for extracting anatomical landmarks which are based on only first-order partial derivativesof an image. To improve the predictability of the extraction results we analyze certain properties of the operators. First,we provide a statistical interpretation in terms of the Cramer—Rao bound representing the minimal localizationuncertainty. Second, we show that the operators can be derived on the basis of invariance principles. It turns out that theoperators form a complete set of principal invariants. Third, we analyze the detection performance using a certain type ofperformance visualization and a scalar performance measure. Experimental results are presented for 3D tomographicimages of the human brain. ( 1999 Pattern Recognition Society. Published by Elsevier Science Ltd. All rights reserved.

Keywords: Point-based registration; Medical image analysis; Landmark extraction; 3D differential operators; Invarianceprinciples; Uncertainty lower bound; Detection performance

1. Introduction

The accuracy and reliability of point-based image reg-istration in general strongly depends on the detectionand accurate localization of landmarks. In particular,this holds true in the case of 3D medical image data, e.g.magnetic resonance (MR) and X-ray computed tomo-graphy (CT) images or digital atlases. In this context, thetask is to accurately match multimodality images suchthat the complementary information of the differentdata sets can be combined. In either case, the goal is toincrease the accuracy of localizing anatomical structuresin 3D space which is of fundamental importance incomputer-assisted neurosurgery. To match different datasets, often corresponding prominent points, also denotedas landmarks, are identified and used as features tocompute the transformation between the data sets [1—4].In comparison to fiducial markers placed outside the

human head, anatomical landmarks have the advantagethat they can be located within the relevant inner brainparts. Therefore, the registration accuracy is generallyincreased in these regions. However, usually 3D ana-tomical landmarks are localized manually which istime-consuming and often lacks accuracy. Instead, semi-automatic or automatic procedures promise to improvethis situation.

Currently, there are only a few computational ap-proaches for extracting point landmarks in 3D to-mographic images [5,6]. Thirion [5], for example, hasintroduced 3D differential operators that consist ofpartial derivatives of an image up to the third order todetect certain points on 3D ridge lines. However, thecomputation of high-order partial derivatives generallyis very sensitive to noise. Therefore, additional stepsare necessary to diminish these instabilities [7]. Relatedoperators based on isocontour curvature properties and

0031-3203/99/$19.00#0.00 ( 1999 Pattern Recognition Society. Published by Elsevier Science Ltd. All rights reserved.PII: S 0 0 3 1 - 3 2 0 3 ( 9 8 ) 0 0 0 8 8 - 0

using partial derivatives up to the second order can befound in Ref. [8] (see also Ref. [9]). In Ref. [6] we haveintroduced 3D differential operators which are generaliz-ations of existing 2D corner detectors. These operatorsemploy either only first-order partial derivatives or first-and second-order partial derivatives of an image. There-fore, these operators are computationally efficient andthey do not suffer from instabilities of computing high-order partial derivatives. Recently, the 2D versions ofthese operators have been evaluated using 2D to-mographic images of the human brain (see Ref. [10]). Itturned out that the operators based on only first-orderderivatives yield better results than those based on first-and second-order derivatives.

In this contribution, we consider 3D operators whichemploy only first-order partial derivatives of an image.Whereas in Ref. [6], these operators have been derivedstraightforwardly in accordance with the 2D case, we hereprovide a sound statistical interpretation of the operators.This interpretation is given in terms of the minimal local-ization uncertainty defined by the Cramer—Rao bound.Moreover, we show that the operators can be derived onthe basis of invariance principles. It turns out that theoperators form a complete set of principal invariants w.r.t.the covariance matrix of the position estimate. Applicationof the operators to 3D tomographic images reveals thata large number of important 3D anatomical landmarkscan be detected with these operators; however, there arealso a number of false detections. In this contribution, wetherefore analyze the detection performance in more de-tail. We introduce and apply a certain type of performancevisualization and also use a scalar quantity to measure thedetection capability. Experimental results are presentedfor 3D tomographic images of the human brain.

2. Statistical interpretation

2D differential corner detectors have usually been de-signed to extract grey-value corners; however, in manyapplications they have been used to find general points ofhigh-intensity variations. Therefore, we can summarize thegeneral extraction criterion of these operators as finding‘‘points with high intensity variations’’. This criterion mo-tivated the 3D generalizations in Ref. [6] and their use forlocalizing anatomical point landmarks in 3D tomographicimages. Whereas the criterion is rather general and un-specific, in this section we provide a statistical interpreta-tion of the 3D operators. We consider the following threeoperators which are based only on first-order partialderivatives of an image and which represent 3D general-izations of the 2D corner detectors in Refs. [11] and [12]:

Op3"detC

gtr C

g

, Op3@"1

tr C~1g

, Op4"detCg,

(1)

where

Cg"+g (+g)T (2)

is a symmetric 3]3 matrix which represents the averageddyadic product of the 3D intensity gradient +g"(g

x, g

y, g

z)T, g (x, y, z) is the image function, and det and tr

denote the determinant and the trace of a matrix, respec-tively. If C

ghas full rank, then C

gis positive-definite and

all eigenvalues of Cg

as well as those of it’s inverse arelarger than zero. Therefore, the operators in Eq. (1) yieldpositive values, and landmark points can be found bycomputing local maxima of the expressions in Eq. (1).

In the following, let p2n

denote the variance of additivewhite Gaussian image noise and m the number of voxelsin a local 3D window. Then, we can relate the matrix C

g,

which captures the intensity variations inside the win-dow, to the minimal localization uncertainty of the centerof the window x"(x, y, z). The minimal localizationuncertainty is given by the Cramer—Rao bound [13] andis represented by the covariance matrix

Rg"

p2n

mC~1

g. (3)

We see that Rgis proportional to the inverse of C

g. From

Eq. (3) we can derive the 3D error ellipsoid of the positionestimate with semi-axes p

x, p

y, and p

z. A quantitative

measure for the localization uncertainty of a landmark isthe volume of the 3D error ellipsoid which is defined as

»"

4

3nJdetR

g, (4)

where

detRg"p2

xp2yp2z. (5)

The smaller the value of det Rgthe smaller is the localiza-

tion uncertainty. Thus, we can formulate the followingcriterion for localizing 3D point landmarks: Find thosepoints with ‘‘minimal localization uncertainty’’, i.e. min-imal volume of the 3D error ellipsoid. This requirementcan be stated as

detRgPmin. (6)

Since detRg"1/detR~1

gand R

gis the inverse of C

g(up to

a factor) we see that Eq. (6) is equivalent to

detCgPmax, (7)

which indeed is the operator Op4 in Eq. (1). Thus, thisoperator extracts 3D points with ‘‘minimal localizationuncertainty’’, i.e. ‘‘highest possible localization pre-cision’’. Note that C

gand thus R

gcan directly be com-

puted from the image data. Analogous, with Eqs. (1) and(3), we see that the operator Op3@ minimizes tr R

gwhich is

the sum of the squared semi-axes of the 3D error ellipsoid(for the 2D case see also Ref. [11]),

trRg"p2

x#p2

y#p2

z. (8)

4 K. Rohr / Pattern Recognition 32 (1999) 3—15

If the semi-axes are small, then generally also the errorellipsoid is small and vice versa. Therefore, this operatorcan be seen as an approximate measure for the volume ofthe ellipsoid. Finally, Op3 is equivalent to (see also Sec-tion 3 below)

12

((trRg)2!trR2

g)"p2

xp2y#p2

xp2z#p2

yp2z, (9)

which can further be written as

1

n2((np

xpy)2#(np

xpz)2#(np

ypz)2). (10)

Now we can see, that the three terms in the paren-theses represent the areas of three (2D) ellipses withcorresponding semi-axes, and the total expression is thesum of the squared areas of the ellipses. Actually, theseellipses are the three orthogonal sections of the 3Derror ellipsoid determined through the directions of theeigenvectors of R

g. Thus, also Op3 represents an approxi-

mation to the volume of the 3D error ellipsoid. In sum-mary, all three operators Op3, Op3@, and Op4 can beinterpreted as measures for the size of the 3D errorellipsoid, using either the sum of (squared) sectionalellipse areas, the sum of the (squared) lengths of thesemi-axes, or the (squared) volume of the ellipsoid, re-spectively.

As an example, in Fig. 1, we show the estimated 3Derror ellipsoid for the landmark ‘genu of corpus callosum’within a 3D MR data set of the human brain. The 3Derror ellipsoid has been displayed by three orthogonal

Fig. 1. Estimated 3D error ellipsoid (enlarged by a factor of 30)for the landmark genu of corpus callosum within a 3D MR dataset of the human brain (orthogonal views: sagittal, axial, co-ronal).

Fig. 1. (Continued).

views (sagittal, axial, coronal) and corresponds to the 1pconfidence regions. Note that the ellipsoid has been en-larged by a factor of 30 for visualization purposes. Wehave used a region-of-interest (ROI) of 21]21]21voxels centered around the detected landmark position,and p2

n"25 has been chosen for the variance of the

image noise. In Fig. 1, we see within the sagittal planethat the localization uncertainty along the edge of theobject is much larger than the localization uncertaintyperpendicular to the edge. This is what we expectfrom the local intensity structure of this landmark. Theaxial and coronal views of the 3D error ellipsoid indi-cate an approximately isotropic localization uncertainty,

K. Rohr / Pattern Recognition 32 (1999) 3—15 5

where the uncertainty within the coronal plane is largerthan in the axial plane. Another example is given inFig. 2 which shows the estimated 3D error ellipsoid forthe landmark ‘‘lower cusp between pons and medullaoblongata’’.

3. Relation to principal invariants

In this section, we show that the 3D differentialoperators described above can be derived by applyinginvariance principles. Invariance (absolute or relative

Fig. 2. Same as Fig. 1 but for the landmark lower cusp betweenpons and medulla oblongata.

Fig. 2. (Continued).

invariance) is an important property which indicatespreservation under certain types of transformations(e.g. rotations). Our analysis is based on the co-variance matrix R

gas introduced above and which rep-

resents the minimal positional uncertainty (Cramer—Raobound). We will consider the principal invariants of thismatrix.

3.1. Principal invariants of a matrix

Generally, the principal invariants of a d]d matrixB"(b

ij) are the coefficients ı

1,2 , ı

d, also denoted

ı1(B), ..., ı

d(B) if we wish to make the dependence on

B explicit. These coefficients appear in the characteristicpolynomial of B:

det(B!jI)"(!1)djd#(!1)d~1ı1(B)jd~1#2

!ıd~1

(B)j#ıd(B), (11)

where I is a d]d unity matrix, such that ı1(B)"trB,

ıd~1

(B)"tr (cofB) ("detB trB~1 if B is invertible), andıd(B)"det B [14]. The complete set of the principal in-

variants can be abbreviated as ıB"(ı1(B), ..., ı

d(B)). The

ıi(B) are invariant under similarity transformations

(translation, rotation, scale) and they are independent ofeach other.

In the case of a 3]3 matrix we have the coefficientsıB"(ı

1(B), ı

2(B), ı

3(B)) which appear in the characteristic

polynomial

det(B!jI)"!j3#ı1(B)j2!ı

2(B)j#ı

3(B). (12)


If we denote the eigenvalues of B by j1, j

2, j

3then the

following relations can be deduced:

ı1(B)"trB"j

1#j

2#j

3,

ı2(B)"1

2((tr B)2!tr B2)"j

1j2#j

2j3#j

1j3,

ı3(B)"detB"j

1j2j3. (13)

Other examples of invariants are trB2 or trB3; however,these invariants are not principal invariants. In the 2Dcase, i.e. for a 2]2 matrix, we have det(B!jI)"j2!ı

1(B)j#ı

2(B) with the two principal invariants

ı1(B)"trB and ı

2(B)"det B.

3.2. Interpretation of the 3D differential operators

The covariance matrix Rg

given in Eq. (3) transformsunder similarity transformations xJ "Ax as R3

g"AR

gAT.

In the 3D case, according to Eq. (13), the three principalinvariants are

ı1(R

g)"tr R

g, ı

2(R

g)"1

2((trR

g)2!tr R2

g),

ı3(R

g)"detR

g. (14)

Regarding the first principal invariant and knowing fromEq. (3) that C

gis the inverse of R

g(up to a factor) it

follows that ı1(R

g) is equivalent to Op3@"1/trC~1

gin Eq.

(1). Note that for localizing landmarks, we compute localmaxima of the operator values. Thus, constant factorsof the operators are not relevant. Also note, that maxi-mizing an expression is equivalent to minimizing theinverse of it. Considering ı

2(R

g) in order to derive a

relation to our 3D operators, we first rewrite the sec-ond principal invariant as ı

2(R

g)"detR

gtr R~1

g. Since

detRg"1/detR~1

gwe have ı

2(R

g)"trR~1

g/detR~1

g.

Thus, ı2(R

g) is equivalent to Op3"det C

g/trC

g. Finally,

again since det Rg"1/detR~1

gand due to Eq. (3), it

follows that the third principal invariant ı3(R

g) is equiva-

lent to Op4"detCg. In summary, the 3D operators Op3,

Op3@, and Op4 in Eq. (1) are equivalent to the principalinvariants of the covariance matrix R

g. The operators

represent the complete set of principal invariants of thismatrix. With J denoting proportional or inverse pro-portional, we can write

ı1(R

g)J

1

tr C~1g

, ı2(R

g)J

detCg

trCg

, ı3(R

g)Jdet C

g.

(15)

Note, that ı1(R

g) can also be expressed exclusively in

terms of the matrix Cg

as ı1(R

g)J2 detC

g/((trC

g)2

!tr C2g).

If instead, we take the matrix Cg

as the basis of ouranalysis and consider the principal invariants of thismatrix, then we obtain

ı1(C

g)"trC

g, ı

2(C

g)"1

2((trC

g)2!tr C2

g),

ı3(C

g)"detC

g. (16)

We immediately see that ı3(C

g) is equivalent to ı

3(R

g),

while the other two invariants are different from theinvariants of R

g. Thus, we have two additional invariants

and in total there are five different principal invariants.All invariants are given in terms of the matrix C

gwhich is

easier to implement in comparison to using the matrixRg. However, since only R

ghas a sound statistical inter-

pretation in terms of the minimal uncertainty of theposition estimate, we favor those operators which arebased on this matrix, i.e. ı

1(R

g), ı

2(R

g), and ı

3(R

g).

In the 2D case, we have the principal invariantsı1(R

g)"tr R

gand ı

2(R

g)"det R

gwhich are indeed equiv-

alent to the 2D corner detectors in Refs. [11] and [12]:

ı1(R

g)J

detCg

trCg

, ı2(R

g)Jdet C

g. (17)

Considering the matrix Cgwe have

ı1(C

g)"trC

g, ı

2(C

g)"detC

g, (18)

where ı2(C

g) is equivalent to ı

2(R

g). Thus, overall we have

three different principal invariants of Rgand C

gin the 2D

case.As an example for the 3D case, we show an application

of the invariant ı2(R

g) which is equivalent to the operator

Op3. We have used a 3D MR image of the human brainconsisting of 68 slices of 165]210 resolution. The totalnumber of detected points is about 1000. The result fora number of slices is shown in Fig. 3 (slices 7, 20, 36, 37,38, and 48). It can be seen that significant structures havebeen detected. For example, in slices 7, 20, and 48 tips ofthe ventricular horns (see the dark elongated structure inthe middle of the brain) and certain characteristic pointson the sulci have been found. Additional detected land-marks are, for example, in slice 36 the anterior commis-sure and the genu of the corpus callosum, in slice 37 thetips between the midbrain and the pons, the posteriorcommissure, and the splenium of the corpus callosum,and in slice 38 the inferior colliculus (for the names ofthese landmarks see a standard book on human anat-omy, e.g., Ref. [15]). Note, that each of these landmarkshave been detected only in single slices which indicatesthat we have actually determined 3D points (to clearlysee this behavior was the reason why we selected thethree adjacent slices 36, 37, and 38). On the other hand,we also obtain some points which we cannot directlyrelate to points of anatomical relevance, e.g. in slices 37and 38 in the upper and right part of the brain. Here, theproblem may be the gap between the two hemispheres ofthe brain which is partly visible as dark areas in theseslices. Anyway, in general the detection performance isvery important, since the reliability and accuracy of theregistration result strongly depends on this criterion.Therefore, we analyze the detection performance of theoperators in more detail below.


Fig. 3. Detected point landmarks for a 3D MR image of a human brain using the 3D operator Op3 (slices 7, 20, 36, 37, 38, and 48).

4. Detection performance

To localize anatomical landmarks in 3D tomographicimages, we use a semi-automatic procedure on the basisof the 3D differential operators described above. In com-

parison to a fully automatic scheme such a semi-auto-matic approach has the advantage that the user has thepossibility to control the results, which is importantin clinical scenaria. In our case, we rely on the follow-ing semi-automatic procedure for localizing a certain


landmark (see also Ref. [16]. First, the user specifiesa region-of-interest (ROI) together with an approximateposition (e.g. the center of the ROI), and second, a 3Doperator is applied yielding landmark candidates withinthe selected ROI. Ideally, there should be only one (cor-rect) landmark candidate. However, due to noise andinteraction effects with neighboring anatomical struc-tures there are generally also false detections. In eithercase, the user should have the possiblility to accept orreject detected candidates.

4.1. Performance visualization

The described semi-automatic procedure can be ap-plied most reliably and efficiently if the number offalse detections is small and moreover if the operatorresponse of the correct detection is much larger thanthe operator responses corresponding to the false de-tections. Note that landmark candidates with very lowoperator responses can be excluded by applying a lowthreshold. However, since still there would be falsedetections left such a procedure would not principallysolve the problem. Also note that since we deal with 3Dimages which are displayed on a slice-by-slice basis inpractice, it is generally very difficult to distinguish falsedetections from correct detections. In the 2D case, theselection is generally no problem since, for example, wecan mark all detections in the original image and displaythe whole result on the screen. To improve the situationfor 3D images, we suggest to use a certain performancevisualization which gives an indication of the detectionperformance. With this visualization we first compute theEuclidean distances Ex

i!x

ROIE2

between the positionsxi

of the detected candidates and the specified centerof the ROI abbreviated by x

ROI. Then, the computed

distances are represented together with the operatorresponses (cf. Figs. 4—6). Thus, this representation com-bines the information of the number of false detectionswith the significance of the detections. Also, we obtain anindication of how the locations of the detections aredistributed w.r.t. x

ROI. However, a disadvantage is that

we lose the full information of the distribution in 3Dspace.

4.2. A performance measure

Whereas the performance visualization describedabove gives a visual impression, we are also interestedin quantitative measures for the detection performance.One measure is the number of false detections; however,with this measure the strength of the operator responsesis not taken into account. Instead, we suggest usinganother measure. Suppose we have a number of n detec-tions within a ROI and the operator responses are de-noted by R

i"Op (x

i), i"1,2 , n, where the maximal

operator response is Rmax

"Op (xmax

). The candidatesmay or may not be the result after applying a threshold

)"MxiDR

i*eR

maxN, (19)

where e is user-defined, e.g. e"0.01. For this set we cancompute the following measure:

t"G0, n"0,

n+i/1

Ri

Rmax

n*1.(20)

If there are no detections at all then we have t"0.Alternatively, if there is only one detection we obtaint"1, and additional false detections with low operatorresponses yield a value of t+1. In this case, the correctdetection can clearly be distinguished from the falsedetections. On the other hand, if there are operatorresponses with similar values as the maximal operatorresponse, then t is much larger than 1. Thus, in summarywe here have a scalar quantity which gives an indicationof the detection performance. Additionally, we can em-ploy the mean value tM "t/n. In the case of severalexclusively similar operator values, where we have tA1,we obtain tM +1. In the case of one strong response anda large number of low responses which also sum up totA1, we instead obtain tM @1. Note, that for our oper-ators the responses R

iare always larger or equal to zero.

Thus, when computing t for other operators which pos-sibly yield positive as well as negative responses, thenabsolute values of R

ishould be used.

5. Experimental results

In this section, we report on experimental results con-cerning the detection performance and the registrationaccuracy.

5.1. Detection performance

The plots on the left side of Fig. 4 show an applicationof the detection performance visualization described inSection 4.1 above. We have used three different anatom-ical landmarks within a 3D MR image of the humanbrain (on the right side are the results for another 3D MRdata set). The landmarks are located on the ventricularsystem and on the skull base. We have used the operatorOp3 and Gaussian derivative filters of p"1.5 to com-pute the partial derivatives of the image. The size of theROI has been chosen to 21]21]21 voxels and thecenter of the ROI is the position due to best manuallocalization. Thus, we take the manually selected posi-tion as ‘‘ground truth’’, although we know that thisposition may be prone to error. In our case, we havealleviated this problem by using landmark positions


Fig. 4. Operator responses of Op3 for landmarks external occipital protuberance (top), left frontal ventricular horn (middle), and leftoccipital ventricular horn (bottom) in two different 3D MR data sets (left and right).

which have been specified in agreement with the judge-ment of three to four persons. To further reduce thesubjectivity in our experiments we have used no thre-sholds at all. For the landmark on the top left of Fig.

4 (external occipital protuberance) we see that we havea detected landmark candidate close to the manuallyspecified position. However, we also have a number offalse detections with relatively large responses. Thus, the


Fig. 5. Same as Fig. 4 but for operator Op3@.

detection performance for this landmark is relatively badand it will be difficult and time-consuming for a userto select the correct candidate. Instead, for the land-mark in the middle left of Fig. 4 (tip of left frontal

ventricular horn) we have a perfect detection result withonly one strong candidate within the whole 3D ROI.For the landmark on the bottom left of Fig. 4 (tip ofleft occipital ventricular horn), we essentially have two


Fig. 6. Same as Fig. 4 but for operator Op4.

strong operator responses of similar values, where oneof them is closer to the manually specified position.Inspection of the original image reveals that the doubledetection due to the individual anatomy of the imaged

person. Here, the occipital ventricular horn of this personactually is a ‘double horn’ consisting of two tips. There-fore, in this case both landmarks are suited for use inregistration and the user has to decide which of the


landmarks should be used. In Fig. 4 on the right are theresults for the same landmarks as before but for a differ-ent 3D MR data set. Here, we see that for the firstlandmark (top right of Fig. 4) the detection result ismuch better in comparison to the previous data set.We have one strong detection close to the manuallyspecified position. The additional false detections havelower operator responses and are farer away from thelandmark position. For the second landmark (middleright) the detection result is good but it is worse incomparison to the previous data set, since here we haveone false detection with a relatively large operator re-sponse. For the third landmark (bottom right), we havea perfect detection result up to some false detections withextremely low operator responses. Comparison with theprevious data set shows that we here have a unique tip ofthe occipital ventricular horn corresponding to ‘‘normal’’anatomy.

We have also applied the operators Op3@ and Op4 forthe same landmarks and the same data sets as above (seeFigs. 5 and 6, respectively). It turns out that the results ofOp3@ are somewhat worse than those of Op3. For thelandmark on the top left in Fig. 5 there is no strongdetection close to the manually selected position, and forthe landmark on the bottom left the detection closest tothe manually selected position has a lower operator re-sponse than the one farer away. Additionally, for Op3@ theoperator responses of the false detections with relativelylow reponses are larger than those for Op3. Thus, thediscrimination power of Op3@ is worse (e.g. compare theresults on the bottom right of the Figs. 4 and 5). Consid-ering the results of Op4 in Fig. 6 we see that this operatoryields similar results as Op3. We have nearly the samestrong detections and also the discrimination power be-tween strong and weak detections is comparable.

For all examples above we have also computed theperformance measure t as defined in Eq. (20). The resultsin Tables 1 and 2 correspond to the left and right sides ofthe Figs. 4—6, respectively. It turns out that the computedquantities are in well agreement with our observations inFigs. 4—6. First of all, we see that the detection perfor-mance for the first landmark is much worse in compari-son to the other landmarks. Second, the operator Op3@generally yields larger values of t than Op3 as well asOp4 and thus the detection performance of this operatoris worse. Third, it can also be seen that Op3 and Op4 yieldvery similar results. We also note, that the measure tis a much better performance characterization than thenumber of false detections alone. For example, for Op3 inTable 2 we have for the first and third landmark exactlythe same number of false detections (7 false detections).However, the detection performance for the third land-mark is much better than that for the first one, since forthe third landmark the false detections have much loweroperator reponses. This fact is clearly quantified by thecorresponding values of t which are in accordance with

Table 1Performance measure t for landmarks in a 3D MR data set

Landmark Op3 Op3@ Op4

External occipital protuberance 4.82 6.38 5.17Frontal ventricular horn 1.01 1.31 1.04Occipital ventricular horn 2.14 2.93 1.81

Table 2Same as Table 1 but for a different 3D MR data set

Landmark Op3 Op3@ Op4

External occipital protuberance 2.58 4.84 1.93Frontal ventricular horn 1.56 1.60 1.63Occipital ventricular horn 1.06 1.62 1.01

the performance visualizations on the top and bottomright of Fig. 4.

5.2. Registration accuracy

We have also used the semi-automatically localizedlandmarks as features for the registration of 3D MR andCT images of the human brain. In the following experi-ment, we have applied the operator Op3 and Gaussianderivative filters of p"1.5. The investigated MR data setconsists of 120 axial slices of 256]256 resolution, and theCT data set consists of 87 axial slices of 320]320 resolu-tion. The images represent the brain of the same person.We have used the following anatomical point landmarks:topmost concavity of fourth ventricle roof, external oc-cipital protuberance, tip of frontal ventricular horn, tipof occipital ventricular horn, tip of temporal ventricularhorn, saddle point at zygomatic bone, as well as saddlepoint at processus mastoideus. The last five landmarkscan be found in both hemispheres of the human brain.Note, that for the used CT data set the tip of the temporalventricular horn was siginificantly worse pronounced incomparison to the other landmarks, therefore, we did notinclude this landmark. In total, we thus have a numberof 11 homologous landmarks. To assess the registrationaccuracy of the localized landmarks we have applied anoptimal affine transformation model. We hereby assumethat an affine transformation model well approximatesthe true transformation. Note also that the registrationaccuracy only checks the global consistency of the land-marks according to the applied transformation model.Anyway, using 11 landmarks and an optimal affine trans-formation model we obtained a good registration resultwith a mean Euclidean error at the landmarks of eN+1.5voxels, where the individual errors at the landmarksvaried between 0.4 and 2.6 voxels.


6. Summary and future work

We have investigated 3D differential operators forextracting anatomical point landmarks in 3D tomo-graphic images of the human brain. These operators onlyemploy first-order partial derivatives of an image and aretherefore computationally efficient and do not sufferfrom instabilities of computing high order partial deriva-tives. To improve the predictability of the operatorswe have analyzed certain properties of them. It has beenshown that the operators can be interpreted in terms ofthe Cramer—Rao bound which represents the minimallocalization uncertainty. All operators can be related tothe geometry of the 3D error ellipsoid and they essential-ly represent the size of the ellipsoid. Moreover, we haveshown that the operators can be derived on the basis ofinvariance principles. It turned out, that the operatorsform a complete set of principal invariants under sim-ilarity transformations. As a basis of our analysis we haveused the covariance matrix of the position estimate dueto the Cramer—Rao bound. Taking the inverse of thatmatrix as a basis, we additionally obtained two invari-ants in the 3D case and one in the 2D case.

We have also analyzed the detection performance ofthe operators. We have introduced and applied a certaintype of performance visualization which combines thenumber of false detections with the significance of thedetections. Also, we have suggested a scalar measureof the detection performance. It turned out, that thismeasure gives a good indication of the detection capabil-ity and that it is more appropriate than using the numberof false detections alone as a detection performance cri-terion. Note that both the performance visualization aswell as the performance measure can be applied withina semi-automatic procedure to guide the user in selectingthe correct landmark candidates. Investigation of thedifferent operators revealed that two operators, namelyOp3 and Op4, yield superior results. We also saw that thedetection performance is generally different for differentlandmarks.

Besides the detection performance, the localization ac-curacy is of primary interest. Recently, an extension ofour operators has been reported in Ref. [17] whichallows for subvoxel localization using multi-step differen-tial approaches. The extracted landmarks serve as fea-tures for a point-based registration scheme. Note, thatthe information of the landmark error ellipsoids can alsobe incorporated as input for registration [18]. In futurework, it will be important to analyze in detail the influ-ence of landmark extraction on the registration result.

Acknowledgement

I thank S. Frantz and H.S. Stiehl for discussionsand critical comments on the manuscript. This work has

been supported by Philips Research Hamburg, projectIMAGINE (IMage- and Atlas-Guided Interventions inNEurosurgery). The original images have kindly beenprovided by W. P. Th. M. Mali, L. Ramos, and C. W. M.van Veelen (Utrecht University Hospital) via ICS-ADof Philips Medical Systems Best and Philips ResearchHamburg.

References

[1] G.T.Y. Chen, M. Kessler, S. Pitluck, Structure transferbetween sets of three dimensional medical imaging data,Proc. 6th Annual Conf. on Computer Graphics, Vol.6 (1985) pp. 171—177.

[2] F.L. Bookstein, Principal warps: thin-plate splines andthe decomposition of deformations, IEEE Trans. PatternAnal. Machine Intell. 11(6) (1989) 567—85.

[3] A.C. Evans, W. Dai, L. Collins, P. Neelin, S. Marrett,Warping of a computerized 3-D atlas to match brainimage volumes for quantitative neuroanatomical and func-tional analysis, in Medical Imaging V: Image Processing,San Jose, California/USA, Proc. SPIE 1445, ed. M.H.Loew, 1991, pp. 236—246.

[4] D.L.G. Hill, D.J. Hawkes, J.E. Crossman, M.J. Gleeson,T.C.S. Cox, E.E.C.M. Bracey, A.J. Strong, P. Graves, Reg-istration of MR and CT images for skull base surgeryusing point-like anatomical features, Br. J. Radiol. 64 (767)(1991) 1030—1035.

[5] J.-P. Thirion, Extremal points: definition and applicationto 3D image registration, Proc. IEEE Conf. on ComputerVision and Pattern Recognition, Seattle/Washington, USA,587—592, June 1994, 21—23.

[6] K. Rohr, On 3D differential operators for detecting pointlandmarks, Image and Vision Computing 15(3) (1997)219—233.

[7] O. Monga, R. Lengagne, R. Deriche, Extraction of thezero-crossings of the curvature derivatives in volumic3D medical images: a multi-scale approach, Proc. IEEEConf. on Computer Vision and Pattern Recognition,Seattle/Washington, U.S.A., June 21—23, 1994, pp.852—855.

[8] W. Beil, K. Rohr, H.S. Stiehl, Investigation of approachesfor the localization of anatomical landmarks in 3Dmedical images, Proc. Computer Assisted Radiology andSurgery (CAR’97), Berlin, Germany, eds H.U. Lemke,M.W. Vannier, K. Inamura June 25—28 (1997),Elsevier, Amsterdam, 1997, pp. 265—270.

[9] J.B.A. Maintz, P.A. van den Elsen, M.A. Viergever, Com-parison of feature-based matching of CT and MRbrain images Proc. 1st Internat. Conf. Computer Vision,Virtual Reality and Robotics in Medicine (CVRMed’95),Nice/France, April (1995), Lecture Notes in ComputerScience 905, ed. N. Ayache, Springer, Berlin, 1995,pp. 219—228.

[10] T. Hartkens, K. Rohr, H.S. Stiehl, Evaluierung von Differ-entialoperatoren zur Detektion charakteristischer Punktein tomographischen Bildern, 18. DAGM-SymposiumMustererkennung, 11—13. September 1996, Heidelberg,Germany, Informatik aktuell, eds B. Jahne, P. Geibler,


About the Author—KARL ROHR received his Diploma degree in electrical engineering from the University of Karlsruhe, Germany, in1987, and his Ph. D. degree in computer science from the University of Hamburg, Germany, in 1994. From 1988 to 1991 he was with theDepartment of Computer Science, University of Karlsruhe, and currently he is with the Department of Computer Science, University ofHamburg, Germany. In 1990 Dr. Rohr was awarded a DAGM prize for his paper on model-based recognition of grey-value corners. In1995 he received the Springer Best Paper Award KI-95 for his joint work on model-based recognition and natural language descriptionof human movements. His research interests include computer vision, medical image analysis, and computer graphics.

H. Haubecker, F. Hering, Springer, Berlin, 1996,pp. 637—644.

[11] W. Forstner, A feature based correspondence algorithmfor image matching, Int. Arch. Photogrammetry RemoteSensing 26(3/3) (1986) 150—166.

[12] K. Rohr, Untersuchung von grauwertabhangigen Trans-formationen zur Ermittlung des optischen Flusses inBildfolgen, Diplomarbeit, Institut fur Nachrichtensysteme,Universitat Karlsruhe, 1987.

[13] H.L. van Trees, Detection, Estimation, and ModulationTheory, Part I, Wiley, New York, 1968.

[14] P. G. Ciarlet, Mathematical Elasticity, Vol. I: Three-Dimensional Elasticity. North-Holland Amsterdam 1988.

[15] J. Sobotta, Atlas der Anatomie des Menschen, 1. Band:Kopf, Hals, Obere Extreminta( ten, eds H. Ferner andJ. Staubesand. Urban und Scharzenberg, Munchen, 1982.

[16] K. Rohr, H.S. Stiehl, R. Sprengel, W. Beil, T.M. Buzug,J. Weese, M.H. Kuhn, Point-based elastic registration of

medical image data using approximating thin-platesplines, Proc. 4th Int. Conf. Visualization in BiomedicalComputing (VBC’96), Hamburg, Germany, 22—25 Septem-ber, 1996, Lecture Notes in Computer Science Vol. 1131,eds K.H. Hohne, R. Kikinis, Springer, Berlin, 1996,pp. 297—306.

[17] S. Frantz, K. Rohr, H.S. Stiehl, On the localization of 3Danatomical point landmarks in medical imagery usingmulti-step differential approaches, 19. DAGM-Sympo-sium Mustererkennung, Braunschweig, Germany, 15—17September (1997), Informatik aktuell, eds E. Paulus, F.M.Wahl, Springer, Berlin, 1997 pp. 340—347.

[18] K. Rohr, R. Sprengel, H.S. Stiehl, Incorporation of land-mark error ellipsoids for image registration based on ap-proximating thin-plate splines, Proc. Computer AssistedRadiology and Surgery (CAR’97), Berlin, Germany,25—28 June (1997), eds H.U. Lemke, M.W. Vannier andK. Inamura, Elsevier Amsterdam, 1997, pp. 234—239.


extraction of 3d anatomical point landmarks based on - kogs

Documents