clinical prospects and technological challenges for
TRANSCRIPT
![Page 1: Clinical Prospects and Technological Challenges for](https://reader030.vdocuments.mx/reader030/viewer/2022012721/61b3c1bbb6801f63353e9f57/html5/thumbnails/1.jpg)
Clinical Prospects and Technological Challenges for Multimodality Imaging Applications in
Radiotherapy Treatment Planning
Clinical Prospects and Technological Challenges for Multimodality Imaging Applications in
Radiotherapy Treatment Planning
Issam El Naqa, PhD
Assistant ProfessorDepartment of Radiation Oncology
Washington University, School of Medicine, St. Louis, MO
SWAAPM Austin, TX, Spring 2008
Issam El Naqa, PhD
Assistant ProfessorDepartment of Radiation Oncology
Washington University, School of Medicine, St. Louis, MO
SWAAPM Austin, TX, Spring 2008
![Page 2: Clinical Prospects and Technological Challenges for](https://reader030.vdocuments.mx/reader030/viewer/2022012721/61b3c1bbb6801f63353e9f57/html5/thumbnails/2.jpg)
• Motivation– Increase usage of multimodality imaging
(CT,PET,MRI,MRS,US) in diagnostic, image-guided intervention, daily localization
– Complementary effect• Anatomical, physiological, soft tissue structures
– Improved target definition (Bradley et al ’04, Rasch et al ’05,
Zangheri et al ’05, Milker-Zabel et al ’06)
• Objectives– Integrate multiple information streams from different
imaging technologies to automatically define ‘biophysical’ target (normal structure) volume
• Motivation– Increase usage of multimodality imaging
(CT,PET,MRI,MRS,US) in diagnostic, image-guided intervention, daily localization
– Complementary effect• Anatomical, physiological, soft tissue structures
– Improved target definition (Bradley et al ’04, Rasch et al ’05,
Zangheri et al ’05, Milker-Zabel et al ’06)
• Objectives– Integrate multiple information streams from different
imaging technologies to automatically define ‘biophysical’ target (normal structure) volume
Why Multimodality Image Analysis?Why Multimodality Image Analysis?
![Page 3: Clinical Prospects and Technological Challenges for](https://reader030.vdocuments.mx/reader030/viewer/2022012721/61b3c1bbb6801f63353e9f57/html5/thumbnails/3.jpg)
Multimodality Image IntegrationMultimodality Image Integration
Anatomical Imaging Functional Imaging
MRI
CT
PETSPECT
MRS
PET/CT
Biophysical Target
Biophysical Target= ( , , ,...)f CT PET MRI
US
![Page 4: Clinical Prospects and Technological Challenges for](https://reader030.vdocuments.mx/reader030/viewer/2022012721/61b3c1bbb6801f63353e9f57/html5/thumbnails/4.jpg)
H&N Example: CT/MRI/PET H&N Example: CT/MRI/PET
Milker-Zabel et al., IJROBP ‘06
![Page 5: Clinical Prospects and Technological Challenges for](https://reader030.vdocuments.mx/reader030/viewer/2022012721/61b3c1bbb6801f63353e9f57/html5/thumbnails/5.jpg)
Prostate Example: CT/MRI/3D-TRUS Prostate Example: CT/MRI/3D-TRUS
Smith et al., IJROBP ‘07
![Page 6: Clinical Prospects and Technological Challenges for](https://reader030.vdocuments.mx/reader030/viewer/2022012721/61b3c1bbb6801f63353e9f57/html5/thumbnails/6.jpg)
Clinical Application ChallengesClinical Application Challenges• Increased acquisition time• Efficiency and automated delineation• Co-registration and fusion of different imaging data
– PET/CT, but how about other modalities?
• Increased acquisition time• Efficiency and automated delineation• Co-registration and fusion of different imaging data
– PET/CT, but how about other modalities?
![Page 7: Clinical Prospects and Technological Challenges for](https://reader030.vdocuments.mx/reader030/viewer/2022012721/61b3c1bbb6801f63353e9f57/html5/thumbnails/7.jpg)
Image RegistrationImage Registration
![Page 8: Clinical Prospects and Technological Challenges for](https://reader030.vdocuments.mx/reader030/viewer/2022012721/61b3c1bbb6801f63353e9f57/html5/thumbnails/8.jpg)
Image registrationImage registration
• Single modality deformable– In PET/CT registration of transmission images instead of
emission images
• Multimodality– Rigid
• PET to CT using normalized mutual information (NMI)
– Deformable Multimodality Registration• Feature based
• Volume Intensity based
• Single modality deformable– In PET/CT registration of transmission images instead of
emission images
• Multimodality– Rigid
• PET to CT using normalized mutual information (NMI)
– Deformable Multimodality Registration• Feature based
• Volume Intensity based
![Page 9: Clinical Prospects and Technological Challenges for](https://reader030.vdocuments.mx/reader030/viewer/2022012721/61b3c1bbb6801f63353e9f57/html5/thumbnails/9.jpg)
Deformable Registration (Level set)Deformable Registration (Level set)
Yang et al., SPIE ‘07
642
531
![Page 10: Clinical Prospects and Technological Challenges for](https://reader030.vdocuments.mx/reader030/viewer/2022012721/61b3c1bbb6801f63353e9f57/html5/thumbnails/10.jpg)
Improved Optical Flow Deformable RegistrationImproved Optical Flow Deformable Registration
Yang et al., ICCR 07
Multigrid
Multipass
![Page 11: Clinical Prospects and Technological Challenges for](https://reader030.vdocuments.mx/reader030/viewer/2022012721/61b3c1bbb6801f63353e9f57/html5/thumbnails/11.jpg)
Deformable Registration (Optical flow)Deformable Registration (Optical flow)
Yang et al., AAPM ‘07
Before registration After registration
![Page 12: Clinical Prospects and Technological Challenges for](https://reader030.vdocuments.mx/reader030/viewer/2022012721/61b3c1bbb6801f63353e9f57/html5/thumbnails/12.jpg)
Deformable Registration ToolDeformable Registration Tool
![Page 13: Clinical Prospects and Technological Challenges for](https://reader030.vdocuments.mx/reader030/viewer/2022012721/61b3c1bbb6801f63353e9f57/html5/thumbnails/13.jpg)
NMI Rigid Registration of Multimodality ImagesNMI Rigid Registration of Multimodality Images
MAX(NMI)
( ) ( )where
( , )
and ( ) is image entropy
H A H BNMI
H A B
H
+=
⋅
![Page 14: Clinical Prospects and Technological Challenges for](https://reader030.vdocuments.mx/reader030/viewer/2022012721/61b3c1bbb6801f63353e9f57/html5/thumbnails/14.jpg)
Example of NMI Registration (MR/CT)Example of NMI Registration (MR/CT)
![Page 15: Clinical Prospects and Technological Challenges for](https://reader030.vdocuments.mx/reader030/viewer/2022012721/61b3c1bbb6801f63353e9f57/html5/thumbnails/15.jpg)
Surface matching and FEM (Finite Element Method)
Surface matching and FEM (Finite Element Method)
FEM-based multi-organ deformable image registration (Brock et al., IJROBP ‘05)
![Page 16: Clinical Prospects and Technological Challenges for](https://reader030.vdocuments.mx/reader030/viewer/2022012721/61b3c1bbb6801f63353e9f57/html5/thumbnails/16.jpg)
Intensity RemappingIntensity Remapping
• Define the intensity mapping functionT(i) = f (s(i))+η(i)
• Finding function f through regressionf(s) = a0+a1*s+a2*s2+a3*s3+…+an*sn
• Bi-functional dependence: allow to remap one intensity value to two intensity values in the second image
• Define the intensity mapping functionT(i) = f (s(i))+η(i)
• Finding function f through regressionf(s) = a0+a1*s+a2*s2+a3*s3+…+an*sn
• Bi-functional dependence: allow to remap one intensity value to two intensity values in the second image
![Page 17: Clinical Prospects and Technological Challenges for](https://reader030.vdocuments.mx/reader030/viewer/2022012721/61b3c1bbb6801f63353e9f57/html5/thumbnails/17.jpg)
Multimodality optical flowMultimodality optical flow
• For any image registration:
• J(h) measure the distance (difference) between the moving image andthe fixed image. R(h) measure the variations of the motion field
• General solution:
• Similarity metrics– Mutual information– Cross-correlation– Correlation ratio
• For any image registration:
• J(h) measure the distance (difference) between the moving image andthe fixed image. R(h) measure the variations of the motion field
• General solution:
• Similarity metrics– Mutual information– Cross-correlation– Correlation ratio
![Page 18: Clinical Prospects and Technological Challenges for](https://reader030.vdocuments.mx/reader030/viewer/2022012721/61b3c1bbb6801f63353e9f57/html5/thumbnails/18.jpg)
Adaptive Radiotherapy Application: KVCT-MVCT Registration
Adaptive Radiotherapy Application: KVCT-MVCT Registration
Yang, Chaudhari, Goddu, Khullar,. Deasy, El Naqa
![Page 19: Clinical Prospects and Technological Challenges for](https://reader030.vdocuments.mx/reader030/viewer/2022012721/61b3c1bbb6801f63353e9f57/html5/thumbnails/19.jpg)
KVCTMVCT
Registered w/o correction
Registered w/ correction
![Page 20: Clinical Prospects and Technological Challenges for](https://reader030.vdocuments.mx/reader030/viewer/2022012721/61b3c1bbb6801f63353e9f57/html5/thumbnails/20.jpg)
Validation of Deformable Registrationusing a ‘biomechanical’ phantom
Validation of Deformable Registrationusing a ‘biomechanical’ phantom
Courtesy Deshan Yang (AAPM ‘07)
![Page 21: Clinical Prospects and Technological Challenges for](https://reader030.vdocuments.mx/reader030/viewer/2022012721/61b3c1bbb6801f63353e9f57/html5/thumbnails/21.jpg)
Image SegmentationImage Segmentation
![Page 22: Clinical Prospects and Technological Challenges for](https://reader030.vdocuments.mx/reader030/viewer/2022012721/61b3c1bbb6801f63353e9f57/html5/thumbnails/22.jpg)
ExamplesExamples
MR cardiac classification
(Unsupervised learning) (Zheng, El Naqa, ’05)
Microcalcification detection
(Supervised Machine learning)
(El Naqa et al. ’02)
PET/CT NSCLC delineation
(Active Contours)
Coronary stenosis detection (Edge detection and linking)
(El Naqa et al ’96)
![Page 23: Clinical Prospects and Technological Challenges for](https://reader030.vdocuments.mx/reader030/viewer/2022012721/61b3c1bbb6801f63353e9f57/html5/thumbnails/23.jpg)
Methods I: ClusteringMethods I: Clustering
Zheng et al., MRI ‘05
![Page 24: Clinical Prospects and Technological Challenges for](https://reader030.vdocuments.mx/reader030/viewer/2022012721/61b3c1bbb6801f63353e9f57/html5/thumbnails/24.jpg)
II. Active Contour Deformable ModelsII. Active Contour
Deformable Models
• Definition: Geometric representations for curves or surfaces that are defined explicitly or implicitly in the imaging domain. These models move (deform) under the influence of internal forces, which are defined within the curve or surface itself, and external forces, which are computed from the image data
• Pros– Boundary smoothness (continuity)– Subpixel accuracy – Prior information (atlas-based)– Mathematically tractable (PDE)– 2D curves are easily generalized to 3D surfaces
• Cons– PDE! (Numerical instability)
• Definition: Geometric representations for curves or surfaces that are defined explicitly or implicitly in the imaging domain. These models move (deform) under the influence of internal forces, which are defined within the curve or surface itself, and external forces, which are computed from the image data
• Pros– Boundary smoothness (continuity)– Subpixel accuracy – Prior information (atlas-based)– Mathematically tractable (PDE)– 2D curves are easily generalized to 3D surfaces
• Cons– PDE! (Numerical instability)
![Page 25: Clinical Prospects and Technological Challenges for](https://reader030.vdocuments.mx/reader030/viewer/2022012721/61b3c1bbb6801f63353e9f57/html5/thumbnails/25.jpg)
• Problems– non-convex optimization problem in (2)
– sensitivity to contour initialization
– dependency on parameterization
– inability to account for topological adaptation
• Problems– non-convex optimization problem in (2)
– sensitivity to contour initialization
– dependency on parameterization
– inability to account for topological adaptation
Parametric models--contParametric models--cont
![Page 26: Clinical Prospects and Technological Challenges for](https://reader030.vdocuments.mx/reader030/viewer/2022012721/61b3c1bbb6801f63353e9f57/html5/thumbnails/26.jpg)
Contour = cross section at L = 0 (i.e., {(x,y,z) | Φ (x,y,z;t) = 0})
L=0
L=+1
L=-1
L=+1
L=0
L=-1
Evolution in the normal direction
Geometric modelsGeometric models
![Page 27: Clinical Prospects and Technological Challenges for](https://reader030.vdocuments.mx/reader030/viewer/2022012721/61b3c1bbb6801f63353e9f57/html5/thumbnails/27.jpg)
PET Segmentation Examples
PET Segmentation Examples
![Page 28: Clinical Prospects and Technological Challenges for](https://reader030.vdocuments.mx/reader030/viewer/2022012721/61b3c1bbb6801f63353e9f57/html5/thumbnails/28.jpg)
Active Contour Segmentation (Synthetic Data)
Active Contour Segmentation (Synthetic Data)
Gradient-based Region-based
![Page 29: Clinical Prospects and Technological Challenges for](https://reader030.vdocuments.mx/reader030/viewer/2022012721/61b3c1bbb6801f63353e9f57/html5/thumbnails/29.jpg)
Active Contour Segmentation (Clinical Data)
Active Contour Segmentation (Clinical Data)
Gradient-based Region-based
El Naqa et al., ICCR ‘04
![Page 30: Clinical Prospects and Technological Challenges for](https://reader030.vdocuments.mx/reader030/viewer/2022012721/61b3c1bbb6801f63353e9f57/html5/thumbnails/30.jpg)
3D Active Contour Segmentation 3D Active Contour Segmentation
![Page 31: Clinical Prospects and Technological Challenges for](https://reader030.vdocuments.mx/reader030/viewer/2022012721/61b3c1bbb6801f63353e9f57/html5/thumbnails/31.jpg)
Multimodality Image AnalysisMultimodality Image Analysis
![Page 32: Clinical Prospects and Technological Challenges for](https://reader030.vdocuments.mx/reader030/viewer/2022012721/61b3c1bbb6801f63353e9f57/html5/thumbnails/32.jpg)
Algorithm to Apply to MultimodalityAlgorithm to Apply to Multimodality
![Page 33: Clinical Prospects and Technological Challenges for](https://reader030.vdocuments.mx/reader030/viewer/2022012721/61b3c1bbb6801f63353e9f57/html5/thumbnails/33.jpg)
Pre-processing: Motion-based Compensation in PETPre-processing: Motion-based Compensation in PET
Motion Blur
Deconvolution-corrected
45
67
8
2
4
6
8
100
2
4
6
8
10
12
Lateral (mm)Anterior-Posterior (mm)
Sup
erio
r-In
ferio
r (m
m)
El Naqa et al., Med Phys. ‘06
![Page 34: Clinical Prospects and Technological Challenges for](https://reader030.vdocuments.mx/reader030/viewer/2022012721/61b3c1bbb6801f63353e9f57/html5/thumbnails/34.jpg)
(b) (c) (d)
GTV-CTGTV-PETGTV-PET/CT
GTV-CTGTV-PETGTV-PET/CTInitializationMVLS
(a)CT
PET
Method II: Active Contours
![Page 35: Clinical Prospects and Technological Challenges for](https://reader030.vdocuments.mx/reader030/viewer/2022012721/61b3c1bbb6801f63353e9f57/html5/thumbnails/35.jpg)
(b) (c) (d)
GTV-PET (40% SUVmax)InitializationMVLS
CT
PET
![Page 36: Clinical Prospects and Technological Challenges for](https://reader030.vdocuments.mx/reader030/viewer/2022012721/61b3c1bbb6801f63353e9f57/html5/thumbnails/36.jpg)
(a)
(b)
(c)
![Page 37: Clinical Prospects and Technological Challenges for](https://reader030.vdocuments.mx/reader030/viewer/2022012721/61b3c1bbb6801f63353e9f57/html5/thumbnails/37.jpg)
![Page 38: Clinical Prospects and Technological Challenges for](https://reader030.vdocuments.mx/reader030/viewer/2022012721/61b3c1bbb6801f63353e9f57/html5/thumbnails/38.jpg)
(a) (b)
(c) (d) (e)
![Page 39: Clinical Prospects and Technological Challenges for](https://reader030.vdocuments.mx/reader030/viewer/2022012721/61b3c1bbb6801f63353e9f57/html5/thumbnails/39.jpg)
El Naqa et al., AAPM ‘06
![Page 40: Clinical Prospects and Technological Challenges for](https://reader030.vdocuments.mx/reader030/viewer/2022012721/61b3c1bbb6801f63353e9f57/html5/thumbnails/40.jpg)
(a)
Phantom Validation of Multimodality Concurrent
Segmentation I
Phantom Validation of Multimodality Concurrent
Segmentation I
Courtesy Sasa Mutic
![Page 41: Clinical Prospects and Technological Challenges for](https://reader030.vdocuments.mx/reader030/viewer/2022012721/61b3c1bbb6801f63353e9f57/html5/thumbnails/41.jpg)
CT
PET
MR
Phantom Validation of Multimodality Concurrent
Segmentation II
Phantom Validation of Multimodality Concurrent
Segmentation II
![Page 42: Clinical Prospects and Technological Challenges for](https://reader030.vdocuments.mx/reader030/viewer/2022012721/61b3c1bbb6801f63353e9f57/html5/thumbnails/42.jpg)
Phantom Validation of Multimodality Concurrent Segmentation III
Phantom Validation of Multimodality Concurrent Segmentation III
0
0.2
0.4
0.6
0.8
1
1 2 3 4
Balls
Ove
rlap
Ind
ex
PET/CT/MR CT only
0
2
4
6
8
1 2 3 4
Balls%
Err
or
in v
olu
me
esti
mat
e
PET/CT/MR CT only
El Naqa et al, ICIP ‘07
![Page 43: Clinical Prospects and Technological Challenges for](https://reader030.vdocuments.mx/reader030/viewer/2022012721/61b3c1bbb6801f63353e9f57/html5/thumbnails/43.jpg)
GUI Screen shot of the software tool. (1) image selector, (2) manual registration control, (3) window level control, (4) zoom control (5) 3D slice number control, (6) status information, (7) the working image panel, (8) ROI region contour, (9) not confirmed segmentation result, (10) right mouse click context menu, (11) menu, (12) the result display panel, zoomed in to ROI, (13) confirmed segmented regions, (14) separated 3D rendering window.
Multimodality Image Analysis ToolMultimodality Image Analysis Tool