Clustering on Local AppearanceClustering on Local Appearance for Deformable Model Segmentation for Deformable Model Segmentation

Clustering on Local AppearanceClustering on Local Appearance for Deformable Model Segmentation for Deformable Model Segmentation

Joshua V. Stough, Robert E. Broadhurst,Joshua V. Stough, Robert E. Broadhurst,Stephen M. Pizer, Edward L. ChaneyStephen M. Pizer, Edward L. Chaney

Joshua V. Stough, Robert E. Broadhurst,Joshua V. Stough, Robert E. Broadhurst,Stephen M. Pizer, Edward L. ChaneyStephen M. Pizer, Edward L. Chaney

MIDAG, UNC-Chapel HillMIDAG, UNC-Chapel Hill

ISBI 2007, SA-AM-OS2bISBI 2007, SA-AM-OS2b

April 14, 2007April 14, 2007

MIDAG, UNC-Chapel HillMIDAG, UNC-Chapel Hill

ISBI 2007, SA-AM-OS2bISBI 2007, SA-AM-OS2b

April 14, 2007April 14, 2007

Deformable Model SegmentationDeformable Model Segmentation Deformable Model SegmentationDeformable Model Segmentation

Problem: Bladder and Prostate in CTProblem: Bladder and Prostate in CT Problem: Bladder and Prostate in CTProblem: Bladder and Prostate in CT

Low contrastLow contrastVariability across Variability across

days.days.

Low contrastLow contrastVariability across Variability across

days.days.

Clustering on Local AppearanceClustering on Local Appearance for Deformable Model Segmentation for Deformable Model Segmentation

Clustering on Local AppearanceClustering on Local Appearance for Deformable Model Segmentation for Deformable Model Segmentation

Bayesian Deformable Model SegmentationBayesian Deformable Model SegmentationAppearance / Image MatchAppearance / Image MatchClustering on Local AppearanceClustering on Local AppearanceResults on bladders and prostates in CTResults on bladders and prostates in CTConclusionsConclusions

Bayesian Deformable Model SegmentationBayesian Deformable Model SegmentationAppearance / Image MatchAppearance / Image MatchClustering on Local AppearanceClustering on Local AppearanceResults on bladders and prostates in CTResults on bladders and prostates in CTConclusionsConclusions

Deformable Model Segmentation (DMS) Deformable Model Segmentation (DMS) for Radiation Treatment Planningfor Radiation Treatment Planning

Deformable Model Segmentation (DMS) Deformable Model Segmentation (DMS) for Radiation Treatment Planningfor Radiation Treatment Planning

Segmentation is Segmentation is challenging and high challenging and high cost for clinicians.cost for clinicians.

Bayesian DMSBayesian DMS

Image match trained Image match trained on expert on expert segmentationssegmentations

Segmentation is Segmentation is challenging and high challenging and high cost for clinicians.cost for clinicians.

Bayesian DMSBayesian DMS

Image match trained Image match trained on expert on expert segmentationssegmentations

Deformable Model and LocalityDeformable Model and Locality Deformable Model and LocalityDeformable Model and Locality

M-rep provides: M-rep provides: Boundary rep. with normalsBoundary rep. with normals CorrespondenceCorrespondence Statistical deformation Statistical deformation pp(m)(m)

pp((I I | | mm),), i image matchmage match We consider appearance at We consider appearance at

local patches.local patches.

M-rep provides: M-rep provides: Boundary rep. with normalsBoundary rep. with normals CorrespondenceCorrespondence Statistical deformation Statistical deformation pp(m)(m)

pp((I I | | mm),), i image matchmage match We consider appearance at We consider appearance at

local patches.local patches.

Atom grid Implied Surface

[Pizer et al., IJCV 55 (2) 2003][Fletcher et al., TMI 23 (8) 2004]

Image Match Image Match pp((II || mm): Related Work): Related Work Image Match Image Match pp((II || mm): Related Work): Related Work

Profile-based, voxel-scale correspondenceProfile-based, voxel-scale correspondence AAMAAM, , [Cootes et al., CVIU [Cootes et al., CVIU 6161 (1) 1995] (1) 1995] AAM +AAM +, , [Scott et al., IPMI 2003][Scott et al., IPMI 2003] Intensity Profile ClusteringIntensity Profile Clustering, , [Stough et al., ISBI 2004][Stough et al., ISBI 2004]

Region-basedRegion-based Intensity RangesIntensity Ranges, , [Zhu et al., PAMI [Zhu et al., PAMI 1818 (9) 1996] (9) 1996] Summary StatisticsSummary Statistics

[Tsai et al., TMI [Tsai et al., TMI 2222 (2) 2003] (2) 2003] [Chan et al., TIP [Chan et al., TIP 1010 (2) 2001] (2) 2001]

Histogram MetricsHistogram Metrics [Rubner et al., CVIU [Rubner et al., CVIU 8484 2001] 2001] [Freedman et al., TMI [Freedman et al., TMI 2424 (3) 2005] (3) 2005]

Statistics on DistributionsStatistics on Distributions, , [Broadhurst et al., ISBI 2006][Broadhurst et al., ISBI 2006]

Profile-based, voxel-scale correspondenceProfile-based, voxel-scale correspondence AAMAAM, , [Cootes et al., CVIU [Cootes et al., CVIU 6161 (1) 1995] (1) 1995] AAM +AAM +, , [Scott et al., IPMI 2003][Scott et al., IPMI 2003] Intensity Profile ClusteringIntensity Profile Clustering, , [Stough et al., ISBI 2004][Stough et al., ISBI 2004]

Region-basedRegion-based Intensity RangesIntensity Ranges, , [Zhu et al., PAMI [Zhu et al., PAMI 1818 (9) 1996] (9) 1996] Summary StatisticsSummary Statistics

[Tsai et al., TMI [Tsai et al., TMI 2222 (2) 2003] (2) 2003] [Chan et al., TIP [Chan et al., TIP 1010 (2) 2001] (2) 2001]

Histogram MetricsHistogram Metrics [Rubner et al., CVIU [Rubner et al., CVIU 8484 2001] 2001] [Freedman et al., TMI [Freedman et al., TMI 2424 (3) 2005] (3) 2005]

Statistics on DistributionsStatistics on Distributions, , [Broadhurst et al., ISBI 2006][Broadhurst et al., ISBI 2006]

Appearance: Regional Intensity Appearance: Regional Intensity Quantile Functions (RIQFs)Quantile Functions (RIQFs)

Appearance: Regional Intensity Appearance: Regional Intensity Quantile Functions (RIQFs)Quantile Functions (RIQFs)

Inverse cumulative distribution function. Inverse cumulative distribution function. Suited to PCA.Suited to PCA. Regional – local object-relative image extent.Regional – local object-relative image extent. Example: probability density and quantile function.Example: probability density and quantile function.

Inverse cumulative distribution function. Inverse cumulative distribution function. Suited to PCA.Suited to PCA. Regional – local object-relative image extent.Regional – local object-relative image extent. Example: probability density and quantile function.Example: probability density and quantile function.

[Levina, UC-Berkeley 2002] [Broadhurst et al., ISBI 2006]

Question: Which Image Match Produces Question: Which Image Match Produces Best Segmentations?Best Segmentations?Question: Which Image Match Produces Question: Which Image Match Produces Best Segmentations?Best Segmentations?

Global regionsGlobal regions

Versus geometrically Versus geometrically defined local regionsdefined local regions

Versus regions defined Versus regions defined by RIQF clusters.by RIQF clusters.

Global regionsGlobal regions

Versus geometrically Versus geometrically defined local regionsdefined local regions

Versus regions defined Versus regions defined by RIQF clusters.by RIQF clusters.

Determine Region TypesDetermine Region Types Determine Region TypesDetermine Region Types

Pool RIQFs over all regions and training Pool RIQFs over all regions and training images.images.

Fuzzy Fuzzy CC-Means Clustering -Means Clustering Example: Example: CC == 2 on bladder exterior. 2 on bladder exterior.

Pool RIQFs over all regions and training Pool RIQFs over all regions and training images.images.

Fuzzy Fuzzy CC-Means Clustering -Means Clustering Example: Example: CC == 2 on bladder exterior. 2 on bladder exterior.

[Bezdec 1981]

Partition the Boundary by Cluster TypePartition the Boundary by Cluster TypePartition the Boundary by Cluster TypePartition the Boundary by Cluster Type

Eig. 1 Z-score

Eig

. 2 Z

-sco

re

For each patch, choose most For each patch, choose most popular cluster.popular cluster.

PCA on cluster populations.PCA on cluster populations.

For each patch, choose most For each patch, choose most popular cluster.popular cluster.

PCA on cluster populations.PCA on cluster populations.

Bladder Partition for Bladder Partition for CC == 2 2 Bladder Partition for Bladder Partition for CC == 2 2

Confirming evidenceConfirming evidence Bladder: mostly fat with prostate Bladder: mostly fat with prostate

and boneand bone Prostate: dense tissue with bone Prostate: dense tissue with bone

and bladder, some fatand bladder, some fat

Confirming evidenceConfirming evidence Bladder: mostly fat with prostate Bladder: mostly fat with prostate

and boneand bone Prostate: dense tissue with bone Prostate: dense tissue with bone

and bladder, some fatand bladder, some fat

Experimental SetupExperimental Setup Experimental SetupExperimental Setup1.1. Determine local RIQF-types in training dataDetermine local RIQF-types in training data2.2. Construct Gaussian models on each typeConstruct Gaussian models on each type3.3. Build a template of optimal types Build a template of optimal types

5 patient image sets, ~16 images 5 patient image sets, ~16 images per patient.per patient.

UNC RadOnc and William UNC RadOnc and William Beaumont, Michigan.Beaumont, Michigan.

512 512 512 5120.98 0.98 0.98 0.98 3 mm 3 mm

5 patient image sets, ~16 images 5 patient image sets, ~16 images per patient.per patient.

UNC RadOnc and William UNC RadOnc and William Beaumont, Michigan.Beaumont, Michigan.

512 512 512 5120.98 0.98 0.98 0.98 3 mm 3 mm

Results Summary, Global v ClusteredResults Summary, Global v Clustered Results Summary, Global v ClusteredResults Summary, Global v Clustered

BladderBladder GlobDSCGlobDSC ClustDSCClustDSC GlobASDGlobASD ClustASDClustASD

11 91.0%91.0% 92.092.0 1.431.43mmmm 1.501.50

22 93.593.5 93.693.6 1.231.23 1.151.15

33 90.990.9 91.391.3 1.581.58 1.481.48

44 93.793.7 93.993.9 1.161.16 1.141.14

55 89.789.7 89.989.9 2.132.13 1.981.98

ProstateProstate GlobDSCGlobDSC ClustDSCClustDSC GlobASDGlobASD ClustASDClustASD

11 90.2%90.2% 91.891.8 0.98mm0.98mm 0.820.82

22 92.092.0 92.392.3 1.341.34 1.261.26

33 92.392.3 92.092.0 0.950.95 0.940.94

44 93.993.9 94.294.2 0.970.97 0.930.93

55 91.391.3 90.090.0 1.591.59 1.781.78

RIQF clustered image match compares favorably with global.

Future Directions:Future Directions: Improved clusteringImproved clustering Modeling mixturesModeling mixtures Region shiftingRegion shifting

Future Directions:Future Directions: Improved clusteringImproved clustering Modeling mixturesModeling mixtures Region shiftingRegion shifting

Conclusions:Conclusions: Local-clustered regions lead to Local-clustered regions lead to

improved segmentationsimproved segmentations Already approaching expert quality, Already approaching expert quality,

exceeding agreement between experts.exceeding agreement between experts.

Conclusions:Conclusions: Local-clustered regions lead to Local-clustered regions lead to

improved segmentationsimproved segmentations Already approaching expert quality, Already approaching expert quality,

exceeding agreement between experts.exceeding agreement between experts.

Prostate ArmProstate ArmProstate ArmProstate Arm

BowelBowelBowelBowel

A walk in the projected space of A walk in the projected space of local intensity distributionslocal intensity distributionsA walk in the projected space of A walk in the projected space of local intensity distributionslocal intensity distributions

Cluster statistics are not specific enough.Cluster statistics are not specific enough.Cluster statistics are not specific enough.Cluster statistics are not specific enough.

Evidence for modeling each Evidence for modeling each patch separately.patch separately.

Evidence for modeling each Evidence for modeling each patch separately.patch separately.