automatic segmentation and statistical shape modeling of the...
TRANSCRIPT
Automatic segmentation and statistical shape modeling of the paranasal
sinuses to estimate natural variations
Ayushi Sinhaa, Simon Leonarda, Austin Reitera, Masaru Ishiib, Russell H. Taylora and Gregory D. Hagera
aDept. of Computer Science, The Johns Hopkins University, Baltimore, MD 21218, USA bDept. of Otolaryngology-Head and Neck Surgery, Johns Hopkins Medical Institutions, Baltimore, MD 21287, USA
Introduction
• Functional endoscopic sinus surgery (FESS) is a routine operation performed by an otolaryngologist• Between 200,000 and 600,000 endoscopic interventions
per year in the USA[1][2][3]
• Navigation during surgery can be improved using pre-operative CT• Reduces likelihood of potential complications
• Enhances patient safety and outcome
[1]Hosemann W, Draf C. Danger points, complications and medico-legal aspects in endoscopic sinus
surgery. GMS Current Topics in Otorhinolaryngology, Head and Neck Surgery. 2013;12:Doc06.[2]Hepworth EJ, Bucknor M, Patel A, Vaughan WC. Nationwide survey on the use of image-guided functional
endoscopic sinus surgery. Otolaryngol Head Neck Surg. 2006 Jul;135(1):68–73.[3]Psaltis AJ, Soler ZM, Nguyen SA, Schlosser RJ. Changing trends in sinus and septal surgery, 2007 to 2009. Int
Forum Allergy Rhinol. 2012 Sep-Oct;2(5):357–361.
Nasal Cycle
• Alternating partial congestion and decongestion of the nasal cavities due to the expansion and contraction of the inferior, middle, and superior turbinates[4]
• Each cycle can span between ~50 minutes to several hours[5]
• Need to compensate for this regularly deforming topology
[4]Hasegawa M, Kern EB, The human nasal cycle. Mayo Clinic Proceedings. May 1977;51:28-34[5]Atanasov AT. Length of Periods in the Nasal Cycle during 24-Hours Registration. Open Journal of Biophysics.
2014;4:93-96
Can we estimate this deformation?
• Lack of longitudinal studies
• But, plenty of head CTs from different patients
• Can we characterize this deformation from head CTs of a large population?
Can we estimate this deformation?
• Hypothesis: Given CTs of 𝑛 individuals, it is likely that the turbinates of each individual are at a different state in the nasal cycle than all others. Therefore, a statistical model of the turbinates built from these 𝑛 CTs should also reflect natural variation.
Statistical Shape Model (SSM)Patient CTs
Template
Method
Deformed Template
Deformably Register
Deform Template PCA
Template Creation[6]
[6]BB Avants, P Yushkevich, J Pluta, D Minko, M Korczykowski, J Detre, JC Gee, “The optimal template effect in hippocampus studies of diseased populations," NeuroImage 49(3), p. 2457, 2010.
1. Automatic SegmentationTemplate CreationDeformable RegistrationSegmentation Improvement
2. Statistical Shape ModelingPrincipal Component AnalysisCorrespondence Improvement
Automatic Segmentation[7]
Template Mesh
Deformation Fields Deformed Meshes
[7]BB Avants, NJ Tustison, . Song, PA Cook, A Klein, and JC Gee, “A reproducible evaluation of ANTs similarity metric performance in brain image registration," NeuroImage 54(3), pp. 2033-2044, 2011.
1. Automatic SegmentationTemplate CreationDeformable RegistrationSegmentation Improvement
2. Statistical Shape ModelingPrincipal Component AnalysisCorrespondence Improvement
Deformable Registration (DR) [7]
[7]BB Avants, NJ Tustison, . Song, PA Cook, A Klein, and JC Gee, “A reproducible evaluation of ANTs similarity metric performance in brain image registration," NeuroImage 54(3), pp. 2033-2044, 2011.
1. Automatic SegmentationTemplate CreationDeformable RegistrationSegmentation Improvement
2. Statistical Shape ModelingPrincipal Component AnalysisCorrespondence Improvement
Gradient Vector Flow (GVF) [10][11]
[10] C. Xu and J. L. Prince, “Gradient vector ow: A new external force for snakes," in Computer Vision andPattern Recognition, IEEE Computer Society Conference on, pp. 66-71, 1997.
[11] C. Xu and J. Prince, “Snakes, shapes, and gradient vector flow," Image Processing, IEEE Transactionson 7, pp. 359-369, Mar 1998.
1. Automatic SegmentationTemplate CreationDeformable RegistrationSegmentation Improvement
2. Statistical Shape ModelingPrincipal Component AnalysisCorrespondence Improvement
Segmentation ResultsLeft Maxillary Sinus Right Maxillary Sinus
DR GVF DR GVF
Front
Back
Segmentation Results
Errors (mm) as compared to hand segmented ground truth
Segmentation Results
Errors (mm) as compared to hand segmented ground truth
Statistical Shape Model (SSM) [8][9]
PCA
[8] T. Cootes, C. Taylor, D. Cooper, and J. Graham, “Active shape models-their training and application,“ Computer Vision and Image Understanding 61(1), pp. 38-59, 1995.
[9] G. Chintalapani, L. M. Ellingsen, O. Sadowsky, J. L. Prince, and R. H. Taylor, “Statistical atlases of bone anatomy: construction, iterative improvement and validation," in Medical Image Computing and Computer-
Assisted Intervention, pp. 499-506, 2007.
1. Automatic SegmentationTemplate CreationDeformable RegistrationSegmentation Improvement
2. Statistical Shape ModelingPrincipal Component AnalysisCorrespondence Improvement
Statistical Shape Model (SSM)Right Maxillary Sinus Left Maxillary SinusInferior TurbinateMiddle Turbinate
1st Mode
2nd Mode
Correspondence Improvement [12]
∗
[12] Seshamani S, Chintalapani G, Taylor RH, Iterative refinement of point correspondences for 3d statistical shape models. MICCAI. 2011;417-425.
1. Automatic SegmentationTemplate CreationDeformable RegistrationSegmentation Improvement
2. Statistical Shape ModelingPrincipal Component AnalysisCorrespondence Improvement
Correspondence Improvement [12]
[12] Seshamani S, Chintalapani G, Taylor RH, Iterative refinement of point correspondences for 3d statistical shape models. MICCAI. 2011;417-425.
1. Automatic SegmentationTemplate CreationDeformable RegistrationSegmentation Improvement
2. Statistical Shape ModelingPrincipal Component AnalysisCorrespondence Improvement
PCA
∗
Leave-one-out Analysis
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
0 10 20 30 40 50 60
Mea
n V
erte
x Er
ror
(mm
)
# modes
Middle Turbinate: Vertex Error
Iter_0
Iter_1
Iter_2
Iter_3
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0 10 20 30 40 50 60
Res
idu
al S
urf
ace
Erro
r (m
m)
# modes
Middle Turbinate: Residual Surface Error
Iter_0
Iter_1
Iter_2
Iter_3
Natural Variation
• Hypothesis: Given CTs of 𝑛 individuals, it is likely that the turbinates of each individual are at a different state in the nasal cycle than all others. Therefore, a statistical model of the turbinates built from these 𝑛 CTs should also reflect natural variation.
Natural Variation
• Experiment
• Built separate models for skull and inferior turbinates
• We expect inferior turbinates to change, but the skull to not change. This should be reflected in the mode weights when pre-op and post-op models are projected onto our statistical model.
PATIENT X
Pre-op CT
Post-op CT
𝑏𝑖𝐼1 = 𝑚𝑖
𝑇 𝑉𝐼1 − 𝑉
𝑉𝐼1∗ = 𝑉 +
𝑖=1
𝑛𝑠
𝑏𝑖𝐼1𝑚𝑖
𝑏𝑖𝐼2 = 𝑚𝑖
𝑇 𝑉𝐼2 − 𝑉
𝑉𝐼2∗ = 𝑉 +
𝑖=1
𝑛𝑠
𝑏𝑖𝐼2𝑚𝑖
Compare mode weights
Natural Variation
-4
-3
-2
-1
0
1
2
3
4
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Mo
de
wei
ghts
mode
Mode Weights: Bone
P2a P2b
-4
-3
-2
-1
0
1
2
3
4
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Mo
de
wei
ghts
mode
Mode Weights: Inferior Turbinate
P2a P2b
Natural Variation
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Dif
fere
nce
mode
Difference in Mode Weights
Bone
IT
Natural Variation
Natural Variation
Population variation vs Natural Variation
Summary
• We have built an initial statistical shape model (SSM) of the paranasal sinuses from CT scans of 53 different patients.
• SSMs of erectile tissue in the sinuses reflect variations due to the nasal cycle, which are captured in the modes of our PCA models.
• A preliminary experiment with a single same-patient pre-op/post-op CT image pair suggests that certain statistical modes are more sensitive than others in characterizing this variation.
Future Work
• We are currently working on constructing a larger statistical atlas of the sinuses based on CT scans of 500 patients.
• We hope to extend our exploration of the nasal cycle using a larger number of same-patient longitudinal studies.
• We are also working to incorporate our results into ongoing research on intraoperative video-CT registration.
References[1] Hosemann W, Draf C. Danger points, complications and medico-legal aspects in endoscopic sinus surgery. GMS Current Topics in Otorhinolaryngology,Head and Neck Surgery. 2013;12:Doc06.[2] Hepworth EJ, Bucknor M, Patel A, Vaughan WC. Nationwide survey on the use of image-guided functional endoscopic sinus surgery. Otolaryngol HeadNeck Surg. 2006 Jul;135(1):68–73.[3] Psaltis AJ, Soler ZM, Nguyen SA, Schlosser RJ. Changing trends in sinus and septal surgery, 2007 to 2009. Int Forum Allergy Rhinol. 2012 Sep-Oct;2(5):357–361.[4] Hasegawa M, Kern EB, The human nasal cycle. Mayo Clinic Proceedings. 1977 May;51:28-34[5] Atanasov AT. Length of Periods in the Nasal Cycle during 24-Hours Registration. Open Journal of Biophysics. 2014;4:93-96[6] Avants BB, Yushkevich P, Pluta J, Minko D, Korczykowski, M, Detre J, Gee JC. The optimal template effect in hippocampus studies of diseased populations.NeuroImage;49(3):2457-2010.[7] Avants BB, Tustison NJ, Song G, Cook PA, Klein A, and Gee JC. A reproducible evaluation of ANTs similarity metric performance in brain image registration.NeuroImage. 2011;54(3):2033-2044. [8] Xu C, Prince JL. Gradient vector flow: A new external force for snakes. Computer Vision and Pattern Recognition, IEEEComputer Society Conference on. 1997;66-71.[8] Cootes T, Taylor C, Cooper D, Graham J. Active shape models-their training and application. Computer Vision and Image Understanding. 1995;61(1):38-59.[9] Chintalapani G, Ellingsen LM, Sadowsky O, Prince JL, and Taylor RH, Statistical atlases of bone anatomy: construction, iterative improvement andvalidation. MICCAI. 2007;499-506.[10] C. Xu and J. L. Prince, “Gradient vector ow: A new external force for snakes," in Computer Vision and Pattern Recognition, IEEE Computer SocietyConference on, pp. 66-71, 1997.[11] Xu C, Prince JL. Snakes, shapes, and gradient vector flow. Image Processing, IEEE Transactions on. 1998 Mar;7:359-369.[12] Seshamani S, Chintalapani G, Taylor RH, Iterative refinement of point correspondences for 3d statistical shape models. MICCAI. 2011;417-425.[13] Lorensen WE, Cline HE, Marching cubes: A high resolution 3d surface construction algorithm. SIGGRAPH. 1987;63-169. [14] Delgado-Gonzalo R,Chenouard N, Unser M. Spline-based deforming ellipsoids for interactive 3D bioimage segmentation. Image Processing, IEEE Transactions on. 2013Oct;22:3926-3940.[14] Weiler K, Edge-based data structures for solid modeling in curved-surface environments. Computer Graphics and Applications, 1985 Jan;5:21-40.
Thank you!
Questions?
Acknowledgement: This work is funded by NIH R01-EB015530: Enhanced Navigation for Endoscopic Sinus Surgery through Video Analysis.