Hybrid multiscale registration of planning CT

images and daily CBCT images for adaptive and

image-guided radiation therapy

Dana Paquin∗, Doron Levy†, and Lei Xing‡

May 10, 2007

Abstract

Adaptive radiation therapy (ART) is the incorporation of daily images inthe radiotherapy treatment process so that the treatment plan can be eval-uated and modified to maximize the amount of radiation dose to the tumorwhile minimizing the amount of radiation delivered to healthy tissue. De-formable registration between the planning images and the daily images isthus an important component of ART. In this paper, we report our researchon deformable registration of planning CT images the daily CBCT images.The registration algorithm is based on a multiscale approach in which thecoarse scales of the images are registered with one another using a landmark-based registration method, and the resulting transformation is fine-tuned bydeformably registering the next scales with one another using the landmark-based transformation as a starting point. This procedure is then iterated, ateach stage using the transformation computed by the previous scale registra-tion as the starting point for the current scale registration. We present theresults of studies of head-neck and prostate CT-CBCT registration, and vali-date our registration method with comparisons of similarity measures beforeand after registration and with visual evaluation of the registration results. Wealso demonstrate that the multiscale method is more accurate for CT-CBCTregistration than ordinary (non-multiscale) B-splines deformable registration.

1 Introduction

Image-guided radiation therapy (IGRT) is the use of patient imaging before andduring treatment to increase the accuracy and efficacy of radiation treatment. The

∗Department of Mathematics, Stanford University, Stanford, CA 94305-2125;paquin@math.stanford.edu

†Department of Mathematics, Stanford University, Stanford, CA 94305-2125;dlevy@math.stanford.edu

‡Department of Radiation Oncology, Stanford University, Stanford, CA 94305-5847;lei@reyes.stanford.edu

1

goals of IGRT are to increase the radiation dose to the tumor, while minimizing theamount of healthy tissue exposed to radiation. As imaging techniques and externalbeam radiation delivery methods have advanced, IGRT (used in conjunction withintensity-modulated radiation therapy) has become increasingly important in treat-ing cancer patients. Numerous clinical studies and simulations have demonstratedthat such treatments can decrease both the spread of cancer in the patient and reducehealthy tissue complications [7], [8], [24].

IGRT is typically implemented in the following way. CT images are obtainedseveral days or weeks prior to treatment, and are used for planning dose distribu-tions, patient alignment, and radiation beam optimization. Immediately prior totreatment, CBCT images are obtained in the treatment room and are used to adjustthe treatment parameters to maximize the radiation dose delivered to the tumor.This enables the practitioner to adjust the treatment plan to account for patientmovement, tumor growth or movement, and deformation of the surrounding organs.To adjust the patient position and radiation beam angles based on the informationprovided by the CBCT images, the CBCT images must first be registered with theplanning CT images. Ideally, an adaptive radiotherapy treatment (ART) will even-tually be implemented in which the patient alignment and/or radiation beam anglesare continually updated in the treatment room to maximize radiation dose to thetumor and minimize radiation to healthy tissue. Such a treatment program wouldrequire real-time multi-modality registration of images obtained during treatmentwith planning images obtained prior to treatment. Thus, registration of images ac-quired from different machines at different times is an important step in the adaptivetreatment process, and incorporation of daily images into the radiotherapy treatmentprocess is an essential step in ART [17], [26].

In a conventional CT imaging system, a motorized table moves the patientthrough a circular opening in the imaging device. See Figure 1 for an example.The machine depicted in Figure 1 is a Hitachi Medical Systems CXR-16 CT imagingsystem (image courtesy of Brooks Stevens Design and Hitachi Medical Systems). Asthe patient passes through the CT system, a source of x-rays rotates around the in-side of the circular opening. The x-ray source produces a narrow, fan-shaped beamof x-rays used to irradiate a section of the body. As x-rays pass through the body,they are absorbed or attenuated at different levels, and image slices are reconstructedbased on the attenuation process. Three-dimensional images are constructed usinga series of two-dimensional slices taken around a single axis of rotation.

In a CBCT imaging system, on the other hand, a cone-shaped beam is rotatedaround the patient, acquiring images incrementally at various angles around thepatient. See Figure 2 for an example. The CBCT machine depicted in Figure 2is an on-board imager (OBI) integrated in a TrilogyTM medical linear accelerator(image courtesy of Varian Medical Systems, Palo Alto, CA). The reconstructed dataset is a three-dimensional image without slice artifacts, which can then be sliced onany plane for two-dimensional visualization. CBCT images contain low frequencycomponents that are not present in CT images (similar to inhomogeneity relatedcomponents in magnetic resonance images). One of the challenges in CT-CBCTimage registration is thus to account for artifacts and other components that appear

2

Figure 1: A conventional spiral CT imaging system.

Figure 2: A CBCT imaging system.

in one of the modalities but not in the other.Numerous efficient and accurate methods for deformable image registration have

been published [1], [12], [19], [20], [25]. See [3], [5] for a survey of deformable im-age registration techniques. Most image registration algorithms can be classifiedas either landmark-based or intensity-based. Landmark-based techniques [18] relyon the identification of corresponding points or features in the images to be regis-tered. Although such methods are computationally easy to implement, the detectionand mathematical characterization of anatomical landmarks is not fully automated.Thus, the landmarks must be user-supplied, and this can be a time-consuming anddifficult process. Intensity-based methods, on the other hand, are fully automaticand do not require extraction or identification of physical features in the images.Mean squares and normalized correlation similarity measures are typically used forimages of the same modality, and mutual information [4], [21] is used for registrationof images of different modalities. In general, fast and fully automatic multi-modalitydeformable registration is still a challenging problem and is a subject of current

3

research [6], [11].In our previous work [16], we developed a fast, accurate, and robust hybrid im-

age registration technique which combines features of landmark-based registrationand free-form deformable registration techniques. Our technique was motivated bythe idea that correspondence between certain structures in images, such as bonystructures, can be easily identified visually, while the correspondence between otherregions, such as soft tissue, is less easy to see. Bony structures undergo rigid trans-formations, while soft tissue and muscular structures undergo deformable transfor-mations. Thus, mapping of the two regions can be approached using different tech-niques. We have shown that the accuracy of our hybrid technique is independentof the precise correspondence of landmarks, and thus provides a significant improve-ment over ordinary landmark-based techniques. Additionally, we have demonstratedthat our technique is robust when the images to be registered contain a substantialamount of noise. In this current work, we register planning CT and daily CBCTimages using our hybrid technique. The motivation for applying our hybrid registra-tion algorithm to CT-CBCT registration is that artifacts that appear, for example,in CBCT images but not in CT images can be treated as noise. Since our hybridalgorithm accurately registers extremely noisy images, we expect that it will alsoprovide accurate registration of CT and CBCT images.

The structure of this paper is as follows. In Section 2, we discuss our multiscaleregistration algorithm. In Section 3, we present several examples of CT-CBCT reg-istration to demonstrate the accuracy of the hybrid registration technique. Section4 concludes.

2 Materials and Methods

We study registration of planning CT and daily CBCT images obtained from theStanford University Medical Center. The CT images were acquired using a GEDiscovery-ST Scanner (GE Medical Systems, Miluakee, WI), and the CBCT imageswere acquired using an on-board imager (OBI) integrated in a TrilogyTM medicallinear accelerator (Varian Medical Systems, Palo Alto, CA). The data set includedone head-neck patient case and one prostate patient case. The head-neck case con-sists of 80 planning CT images and 120 daily CBCT images, and the prostate caseconsists of 80 planning CT images and 60 daily CBCT images. To study the prob-lem of deformable registration of planning CT and daily CBCT images, the followingmethods are investigated in this paper. In the following discussion, A represents thefixed image (in these examples, the CT image) and B represents the moving image(CBCT).

2.1 Hybrid multiscale registration

Our hybrid registration algorithm is implemented as follows.

1. Decompose the images to be registered into a hierarchical series of coarse andfine scales using the multiscale (BV, L2) image decomposition of Tadmor et al.

4

.

.

.

Coarsest Scale of

(Fixed Image)

Original Image B

(Moving Image)

Original Image A

C1(A) Image A

Coarsest Scale of

C2(A)

Finest Scale of

Image A

C1(B) Image B

C2(B)

Image B

Cm(A)

Cm(B)Finest Scale of

Figure 3: The multiscale image decomposition.

3

4

3

4

2

1

2

1

C1(A) C1(B)

LandmarkRegistration

φlandmark

Figure 4: The coarse-scale landmark-based registration step of the hybrid multiscaleregistration algorithm.

[23]. See Figure 3. This decomposition separates an image into a series of scalesin which the first scale consists only of the main shapes and general featuresof the image, and each additional scale includes increasingly finer amounts ofdetail and texture (until the final scale is a close approximation to the originalimage). In this paper, as in our previous works, we decompose each image into12 scales. We note that the decomposition can be implemented in either 2 or3 dimensions. For the details of the 2D implementation, see [23]. For detailsof the 3D implementation, see [15].

2. Register the first coarse scales C1(A) and C1(B) of the images with one an-other using a standard landmark-based registration algorithm [18]. See Figure4. The landmarks are taken as bony structures. Let φlandmark denote the result-ing transformation. Since bony structures undergo rigid transformations, werequire φlandmark to be an affine transformation (consisting only of translation,rotation, scaling, and shearing). Since the coarse scales contain only the mainshapes and general features of each image, the identification of landmarks inthe coarse scales is a relatively easy task, and can potentially be implementedautomatically (for example, using the methods of [22]).

3. Next, use φlandmark as the starting point to deformably register the next coarsescales C2(A) and C2(B) with one another. In this work, we use φlandmark as the

5

Cm(B)

φlandmark

DeformableRegistration

φ1

C1(A) C1(B)

Deformable

RegistrationC2(A)

φ1

φ2

C2(B)

Deformable

Registration

φm

φm−1

Cm(A)

Figure 5: The hybrid multiscale registration algorithm.

input transformation for a B-splines free-form deformable registration [2], [10]algorithm. Let φ2 denote the resulting transformation.

4. Use φ2 as the starting point to deformably register C3(A) and C3(B) with oneanother. Let φ3 denote the transformation obtained upon registering C3(A)with C3(B). Iterate this method, at each stage using the transformation com-puted by the previous scale registration algorithm as the starting point forthe current registration. See Figure 5. For all of the B-splines deformableregistration steps, we use a spline of order 3 and a grid size of 8×8.

2.2 Evaluation

To visually evaluate the accuracy of our hybrid multiscale technique, we will comparecheckerboard comparisons of corresponding slices of the CT and CBCT images afterordinary B-splines deformable registration and after hybrid multiscale registration.

To quantitatively evaluate the accuracy of our hybrid multiscale registration al-gorithm for registration of CT and CBCT images, we shall compare the mutual in-formation measure between the images before registration, after ordinary deformableregistration, and after hybrid multiscale registration. Recall that mutual informationis an information-theoretic metric which computes the amount of information thatone random variable (here, image intensity) gives about another random variable(here, intensity values of another image). The mutual information of two images A

and B is defined as follows:

MI(A, B) = h(A) + h(B) − h(A, B), (1)

where h(x) = −∫

p(x) ln p(x) dx is the entropy of the random variable x, h(x, y) =∫p(x, y) ln p(x, y) dx dy is the joint entropy of the random variables x and y, p(x)

6

is the probability density of the random variable x, and p(x, y) is the joint proba-bility density function of x and y. Mutual information has been studied extensivelyas a method of comparing images originating from multiple modalities in [4], [21].The mutual information of two images increases when the images are geometricallyaligned.

If x and y are independent, then p(x, y) = p(x)p(y), so h(x, y) = h(x) + h(y).However, However, if there is any dependency (as would be the case if x and y areintensity values of images of the same object), then h(x, y) < h(x)+h(y). Note thatMI(A, A) = h(A), so that the maximum possible mutual information between twoimages is the entropy of one of the images.

3 Results

In each of the following examples, we present the results of our hybrid multiscaleregistration algorithm for CT-CBCT head-neck and prostate registration. In eachexample, we illustrate a slice of the CT image (upper left), the corresponding slice ofthe CBCT image (upper right), a checkerboard comparison of the images after ordi-nary B-splines deformable registration (lower left), and a checkerboard comparisonof the images after hybrid multiscale registration (lower right). We have highlightedareas of misregistration in the checkerboard images after ordinary registration witharrows. In particular, we notice misalignment on the edges and in the bony struc-ture regions. To demonstrate the accuracy and applicability of our method, we haveillustrated example slices which contain different anatomical features.

3.1 Head-neck CT-CBCT registration

In Figures 6, 8, and 9, we present three head-neck registration examples. To highlightthe misregistration obtained in the bony structures after ordinary deformable regis-tration, in Figure 7 we present a magnified view of the misaligned areas for example1 after ordinary deformable registration and after hybrid multiscale registration.

3.2 Prostate CT-CBCT registration

In Figures 10, 12, 13, and 14, we present four prostate registration examples. To high-light the misregistration obtained in the bony structures after ordinary deformableregistration, in Figure 11 we present a magnified view of the misaligned areas forexample 1 after ordinary deformable registration and after hybrid multiscale regis-tration.

3.3 Mutual information similarity measures

In Figures 15 and 16, we present the mutual information similarity measures betweenthe planning CT’s and daily CBCT’s before registration (circles), after ordinary B-splines deformable registration (squares), and after hybrid multiscale registration

7

Figure 6: Head and neck example 1. Planning CT image (upper left), daily CBCTimage (upper right), checkerboard comparison after ordinary B-splines deformableregistration (lower left), checkerboard comparison after hybrid multiscale registration(lower right).

8

Figure 7: Head and neck example 1. A magnified view of the checkerboard compar-ison of the bony structures after ordinary deformable registration (upper) and afterhybrid multiscale registration (lower).

9

Figure 8: Head and neck example 2. Planning CT image (upper left), daily CBCTimage (upper right), checkerboard comparison after ordinary B-splines deformableregistration (lower left), checkerboard comparison after hybrid multiscale registration(lower right).

10

Figure 9: Head and neck example 3. Planning CT image (upper left), daily CBCTimage (upper right), checkerboard comparison after ordinary B-splines deformableregistration (lower left), checkerboard comparison after hybrid multiscale registration(lower right).

11

Figure 10: Prostate example 1. Planning CT image (upper left), daily CBCT image(upper right), checkerboard comparison after ordinary B-splines deformable registra-tion (lower left), checkerboard comparison after hybrid multiscale registration (lowerright).

12

Figure 11: Prostate example 1. A magnified view of the checkerboard comparison ofthe bony structures after ordinary deformable registration (upper) and after hybridmultiscale registration (lower).

13

Figure 12: Prostate example 2. Planning CT image (upper left), daily CBCT image(upper right), checkerboard comparison after B-splines deformable registration (lowerleft), checkerboard comparison after hybrid multiscale registration (lower right).

14

Figure 13: Prostate example 3. Planning CT image (upper left), daily CBCT image(upper right), checkerboard comparison after B-splines deformable registration (lowerleft), checkerboard comparison after hybrid multiscale registration (lower right).

15

Figure 14: Prostate example 4. Planning CT image (upper left), daily CBCT image(upper right), checkerboard comparison after B-splines deformable registration (lowerleft), checkerboard comparison after hybrid multiscale registration (lower right).

16

0 1 0 2 0 3 0 4 0 5 000 . 20 . 40 . 60 . 8 11 . 2 A f t e r m u l t i s c a l e r e g i s t r a t i o nA f t e r o r d i n a r y r e g i s t r a t i o nB e f o r e r e g i s t r a t i o n

Figure 15: The mutual information similarity measures between the head-neck plan-ning CT’s and daily CBCT’s before registration, after B-splines deformable registra-tion, and after hybrid multiscale registration

(crosses) for all slices considered for the head-neck and prostate registration examples.For all cases, the similarity measures increased after hybrid multiscale registration.

4 Conclusions

We have demonstrated the accuracy of our hybrid multiscale registration algorithm[16] for registration of planning CT images and daily CBCT images with severalhead-neck and prostate registration examples. The hybrid multiscale algorithm ismotivated by the observation that different anatomical structures in the images to beregistered undergo different types of transformations, and thus mapping of the dif-ferent regions should be approached differently. We use a multiscale implementationto increase computational efficiency and accuracy and to prevent the registrationalgorithm from reaching local minima.

References

[1] R. Bajcsy and S. Kovacic, Multiresolution elastic matching, Computer Vi-sion Graphics and Image Processing, 46, pp. 1–21, 1989.

[2] F. L. Bookstein, Principal Warps: Thin-Plate Splines and the Decomposition

of Deformations, IEEE Transactions on Pattern Analysis and Machine Intelli-gence, 11, no. 6, pp. 567–585, 1989.

17

0 1 0 2 0 3 0 4 0 5 00 . 20 . 30 . 40 . 50 . 60 . 70 . 80 . 9 A f t e r m u l t i s c a l e r e g i s t r a t i o nA f t e r o r d i n a r y r e g i s t r a t i o nB e f o r e r e g i s t r a t i o n

Figure 16: The mutual information similarity measures between the prostate plan-ning CT’s and daily CBCT’s before registration, after B-splines deformable registra-tion, and after hybrid multiscale registration

[3] L. Brown, A Survey of Image Registration Techniques, ACM Computing Sur-veys, 24, no. 4, pp. 325–376, 1992.

[4] A. Collignon, D. Vadermeulen, P. Suetens, and G. Marchal, 3d

multi-modality medical image registration based on information theory, Compu-tational Imaging and Vision, vol. 3, pp. 263-274, 1995.

[5] W. R. Crum, T. Hartkens, and D. L. G. Hill, Non-rigid image registra-

tion: theory and practice, The British Journal of Radiology, 77, pp. 140–153,2004.

[6] J.C. Gee, A. Maintz, and M.W. Vannier, Biomedical Image Registration:

Revised Papers, Second International Workshop, WBIR 2003 (Philadelphia, PA,USA: Springer).

[7] C.W. Hurkmans, B.C.J. Cho, E. Damen, et al., Reduction of cardiac

and lung complication probabilities after breast irradiation using conformal ra-

diotherapy with or without intensity modulation, Radiotherapy Oncology, vol.62, pp. 163–171, 2002.

[8] R. Jacob, A. Hanlon, E. Horowitz, et al., The relationship of increas-

ing radiotherapy dose to reduced distant metastases and mortality in men with

prostate cancer, Cancer, vol. 100, pp. 538–543, 2004.

18

[9] S. Lee, G. Wolberg, K.-Y. Chwa, and S. Y. Shin, Image metamorpho-

sis with scattered feature constraints, IEEE Transactions on Visualization andComputer Graphics, 2, pp. 337–354, 1996.

[10] S. Lee, G. Wolberg, and S. Y. Shin, Scattered data interpolation with mul-

tilevel B-splines, IEEE Transactions on Visualization and Computer Graphics,3, pp. 228–244, 1997.

[11] B. Likar and F. Pernus, A hierarchical approach to elastic registration based

on mutual information, Image and Vision Computing, 19, pp. 33–44, 2001.

[12] W. Lu, M. Chen, G.H. Olivera, K. Ruchala and T.R. Mackie, Fast

free-form deformable registration via calculus of variations, Physics in Medicineand Biology, 49, pp. 3067–3087, 2004.

[13] J. Modersitzki, Numerical Methods for Image Registration, Oxford, 2004.

[14] D. Paquin, D. Levy, E. Schreibmann, and L. Xing, Multiscale image

registration, Mathematical Biosciences and Engineering, 3, no. 2, pp. 389–418,2006.

[15] D. Paquin, D. Levy, and L. Xing, Multiscale deformable registration, IEEETransactions on Medical Imaging, submitted.

[16] D. Paquin, D. Levy, and L. Xing, Hybrid multiscale landmark and de-

formable image registration, Mathematical Biosciences and Engineering, ac-cepted for publication.

[17] H. Rehbinder, C. Forsgren, and J. Lof, Adaptive radiation therapy for

compensation of errors in patient setup and treatment delivery, Medical Physics,31, pp. 3363–3371, 2004.

[18] K. Rohr, Landmark-based image analysis, Computational Imaging and Vision,Kluwer Academic, Dordrecht, 2001.

[19] E. Schreibmann and L. Xing, Narrow band deformable registration of

prostate magnetic resonance imaging, magnetic resonance spectroscopic imag-

ing, and computed tomography studies, International Journal of Radiation On-cology Biology Physics, 62, pp. 595–605, 2005.

[20] sc J.P. Thirion, Image matching as a diffusion process: an analogy with

Maxwell’s demons, Medical Image Analysis, 2, pp. 243–260, 1998.

[21] P. Viola, W. Wells, H. Atsumi, S. Nakajima, and R. Kikinis, Multi-

modal volume registration by maximization of mutual information, Medical Im-age Analysis, vol. 1, no. 1, pp. 35-51, 1995.

[22] E. Schreibmann and L. Xing, Image registration with auto-mapped control

volumes, Medical Physics, 33, no. 4, pp. 1165–1179, 2006.

19

[23] E. Tadmor, S. Nezzar, and L. Vese, A multiscale image representation us-

ing hierarchical (BV, L2) decompositions, Multiscale Modeling and Simulations,2, no. 4, pp. 554–579, 2004.

[24] M. Tubiana and F. Eschwege, Conformal radiotherapy and intensity-

modulated radiotherapy: Clinical data, Acta Oncology, vol. 39, pp. 555-567,2000.

[25] H. Wang, L. Dong, J. O’Daniel, R. Mohan, A.S. Garden, K.K. Ang,

D.A. Kuban, M. Bonnen, J.Y. Chang, and R. Cheung, Validation of

an accelerated demons algorithm for deformable image registration in radiation

therapy, Physics in Medicine and Biology, 50, pp. 2887–2905, 2005.

[26] D. Yan, F. Vicini, J. Wong, and A. Martinez, Adaptive radiation therapy,Physics in Medicine and Biology, 42, pp. 123–132, 1997.

20