huzefa slides icip01web.eecs.umich.edu/~hero/preprints/huzefa_slides_icip01.pdfmi registration of...
TRANSCRIPT
![Page 1: huzefa slides icip01web.eecs.umich.edu/~hero/Preprints/huzefa_slides_icip01.pdfMI Registration of Gray Levels (Viola&Wells:ICCV95) X:a N N UL image (lexicographically ordered) X (k):](https://reader034.vdocuments.mx/reader034/viewer/2022051806/60018171cf65a227b95548c7/html5/thumbnails/1.jpg)
Feature coincidence trees for registration of ultrasoundbreast images
H. Neemuchwala†, A. O. Hero�†#, P. Carson†
Dept. EECS�, Dept BME†, Dept. Statistics#
University of Michigan - Ann Arbor
http://www.eecs.umich.edu/˜hero
Outline
1. Breast Imaging and Registration Background
2. α-MI Criterion
3. Higher Order Feature Selection
4. Experimental results
1
![Page 2: huzefa slides icip01web.eecs.umich.edu/~hero/Preprints/huzefa_slides_icip01.pdfMI Registration of Gray Levels (Viola&Wells:ICCV95) X:a N N UL image (lexicographically ordered) X (k):](https://reader034.vdocuments.mx/reader034/viewer/2022051806/60018171cf65a227b95548c7/html5/thumbnails/2.jpg)
(a) ImageX0 (b) ImageXi
Figure 1: A multidate 3D breast-registration example
2
![Page 3: huzefa slides icip01web.eecs.umich.edu/~hero/Preprints/huzefa_slides_icip01.pdfMI Registration of Gray Levels (Viola&Wells:ICCV95) X:a N N UL image (lexicographically ordered) X (k):](https://reader034.vdocuments.mx/reader034/viewer/2022051806/60018171cf65a227b95548c7/html5/thumbnails/3.jpg)
Background
Some statistics (US)� One out of nine women will contract breast cancer in their lifetimes
� Breast cancer is second leading cause of cancer death among women
� Diagnostic ultrasound (UL) is cheap/available screening modality
� 65% of malignant breast lesions are missed by community
practitioners
What measures are needed to improve detection?
� Routine screening exams: Serial UL studies
� Volumetric imaging to discriminate low contrast lesions from benign
microcalcifications and cysts
� Requirement:Fast and accurate volumetric image registration
3
![Page 4: huzefa slides icip01web.eecs.umich.edu/~hero/Preprints/huzefa_slides_icip01.pdfMI Registration of Gray Levels (Viola&Wells:ICCV95) X:a N N UL image (lexicographically ordered) X (k):](https://reader034.vdocuments.mx/reader034/viewer/2022051806/60018171cf65a227b95548c7/html5/thumbnails/4.jpg)
MI Registration of Gray Levels (Viola&Wells:ICCV95)� X: aN�N UL image (lexicographically ordered)
� X(k): image gray level at pixel locationk
� X0 andX1: primary and secondary images to be registered
Hypothesis: f(X0(k);Xi(k)gN2
k=1 are i.i.d. r.v.’s with j.p.d.f
f0;i(x0;x1); x0;x1 2 f0;1; : : : ;255g
Mutual Information (MI) criterion : T = argmaxTiM̂I
whereM̂I is an estimate of
MI( f0;i) =Z Z
f0;i(x0;x1) ln f0;i(x0;x1)=( f0(x0) fi(x1))dx1dx0: (1)
4
![Page 5: huzefa slides icip01web.eecs.umich.edu/~hero/Preprints/huzefa_slides_icip01.pdfMI Registration of Gray Levels (Viola&Wells:ICCV95) X:a N N UL image (lexicographically ordered) X (k):](https://reader034.vdocuments.mx/reader034/viewer/2022051806/60018171cf65a227b95548c7/html5/thumbnails/5.jpg)
(a) ImageX1 (b) ImageX0
Figure 2: Single Pixel Coincidences
5
![Page 6: huzefa slides icip01web.eecs.umich.edu/~hero/Preprints/huzefa_slides_icip01.pdfMI Registration of Gray Levels (Viola&Wells:ICCV95) X:a N N UL image (lexicographically ordered) X (k):](https://reader034.vdocuments.mx/reader034/viewer/2022051806/60018171cf65a227b95548c7/html5/thumbnails/6.jpg)
Joint Feature Histogram Scatterplots
50 100 150 200 250
50
100
150
200
250
50 100 150 200 250
50
100
150
200
250
50 100 150 200 250
50
100
150
200
250
50 100 150 200 250
50
100
150
200
250
Figure 3:MI Scatterplots. 1st Col: target=reference slice. 2nd Col: target = reference+1 slice.
6
![Page 7: huzefa slides icip01web.eecs.umich.edu/~hero/Preprints/huzefa_slides_icip01.pdfMI Registration of Gray Levels (Viola&Wells:ICCV95) X:a N N UL image (lexicographically ordered) X (k):](https://reader034.vdocuments.mx/reader034/viewer/2022051806/60018171cf65a227b95548c7/html5/thumbnails/7.jpg)
Range of UL breast Image Types
Figure 4: Three ultrasound breast scans. From top to bottom are: case 151,
case 142 and case 162.
7
![Page 8: huzefa slides icip01web.eecs.umich.edu/~hero/Preprints/huzefa_slides_icip01.pdfMI Registration of Gray Levels (Viola&Wells:ICCV95) X:a N N UL image (lexicographically ordered) X (k):](https://reader034.vdocuments.mx/reader034/viewer/2022051806/60018171cf65a227b95548c7/html5/thumbnails/8.jpg)
Limitations of Gray Level MI Registration Methods
Difficulties:
1. Gray levels are uninformative features for UL images
2. MI criterion is sub-optimal for classifying correct deformation T
Our approach:
1. Generalize gray levels to a more stable and pertinent feature set
2. Use inductive learning techniques for feature selection
3. Use newα-MI criterion in place of MI criterion
8
![Page 9: huzefa slides icip01web.eecs.umich.edu/~hero/Preprints/huzefa_slides_icip01.pdfMI Registration of Gray Levels (Viola&Wells:ICCV95) X:a N N UL image (lexicographically ordered) X (k):](https://reader034.vdocuments.mx/reader034/viewer/2022051806/60018171cf65a227b95548c7/html5/thumbnails/9.jpg)
α-MI Registration of Coincident Features� X: aN�N UL image (lexicographically ordered)
� Z = Z(X): a general image feature vector in aP-dimensional feature
space
Let fZ0(k)gKk=1 andfZi(k)gK
k=1 be features extracted fromX0 andXi at K
identical spatial locations
α-MI coincident-feature criterion
T = argmaxTiM̂Iα
whereM̂Iα is an estimate of
MIα( f0;i) =
1α�1
log
Z Z
f α0;i(z0;z1) f 1�α
0 (z0) f 1�αi (z1)dz1dz0: (2)
9
![Page 10: huzefa slides icip01web.eecs.umich.edu/~hero/Preprints/huzefa_slides_icip01.pdfMI Registration of Gray Levels (Viola&Wells:ICCV95) X:a N N UL image (lexicographically ordered) X (k):](https://reader034.vdocuments.mx/reader034/viewer/2022051806/60018171cf65a227b95548c7/html5/thumbnails/10.jpg)
Why α-MI?
Special cases:
� α-MI vs. Shannon MI
limα!1
MIα( f0;i) =Z Z
f0;i ln f0;i=( f0 fi)dz1dz0:
� α-MI vs. Hellinger Mutual Affinity
MI 12
( f0;i) = � ln�Z Z p
f0;i f0 fi dz0dz1
�2
� α-MI vs. Batthacharyya-Hellinger information
Z Z �p
f0;i �p
f0 fi
�2dz0dz1 = 2
�1�expf�MI 1
2
( f0;i)g�
10
![Page 11: huzefa slides icip01web.eecs.umich.edu/~hero/Preprints/huzefa_slides_icip01.pdfMI Registration of Gray Levels (Viola&Wells:ICCV95) X:a N N UL image (lexicographically ordered) X (k):](https://reader034.vdocuments.mx/reader034/viewer/2022051806/60018171cf65a227b95548c7/html5/thumbnails/11.jpg)
α-MI and Decision Theoretic Error Exponents
H0 : Z0(k);Zi(k) independent
H1 : Z0(k);Zi(k) o:w:
Bayes probability of error
Pe(n) = β(n)P(H1)+α(n)P(H0)
Chernoff bound
liminfn!∞
1n
logPe(n) =� supα2[0;1]
f(1�α)MIα( f0;i)g :11
![Page 12: huzefa slides icip01web.eecs.umich.edu/~hero/Preprints/huzefa_slides_icip01.pdfMI Registration of Gray Levels (Viola&Wells:ICCV95) X:a N N UL image (lexicographically ordered) X (k):](https://reader034.vdocuments.mx/reader034/viewer/2022051806/60018171cf65a227b95548c7/html5/thumbnails/12.jpg)
Gray Level α-MI Trajectories
−5 −4 −3 −2 −1 0 1 2 3 4 5−0.3
−0.25
−0.2
−0.15
−0.1
−0.05
0
0.05
ANGLE OF ROTATION (deg) −−−−−−>
α−D
IVE
RG
EN
CE
(αM
I) −
−−
−−
−>
α−DIVERGENCE v/s α CURVES FOR α ∈ [0,1] FOR SINGLE PIXEL INTENSITY
α =0 α =0.1α =0.2α =0.3α =0.4α =0.5α =0.6α =0.7α =0.8α =0.9
α=0.1
α=0.9
α=0
Figure 5:α-MI for ultra sound image registration
12
![Page 13: huzefa slides icip01web.eecs.umich.edu/~hero/Preprints/huzefa_slides_icip01.pdfMI Registration of Gray Levels (Viola&Wells:ICCV95) X:a N N UL image (lexicographically ordered) X (k):](https://reader034.vdocuments.mx/reader034/viewer/2022051806/60018171cf65a227b95548c7/html5/thumbnails/13.jpg)
Peak Curvature of Gray Level α-MI
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.90
0.005
0.01
0.015
0.02
0.025
α −−−−−−>
curv
atur
e of
α−
MI −
−−
−−
−>
CURVATURE OF: α−MI v/s α CURVE FOR SINGLE PIXEL INTENSITY
Figure 6: Curvatureα-MI as function of alpha
13
![Page 14: huzefa slides icip01web.eecs.umich.edu/~hero/Preprints/huzefa_slides_icip01.pdfMI Registration of Gray Levels (Viola&Wells:ICCV95) X:a N N UL image (lexicographically ordered) X (k):](https://reader034.vdocuments.mx/reader034/viewer/2022051806/60018171cf65a227b95548c7/html5/thumbnails/14.jpg)
Higher Order Features
1. Local tags
2. Spatial relations between local tags
3. Forests of randomized feature trees
4. Independent components analysis (ICA)
14
![Page 15: huzefa slides icip01web.eecs.umich.edu/~hero/Preprints/huzefa_slides_icip01.pdfMI Registration of Gray Levels (Viola&Wells:ICCV95) X:a N N UL image (lexicographically ordered) X (k):](https://reader034.vdocuments.mx/reader034/viewer/2022051806/60018171cf65a227b95548c7/html5/thumbnails/15.jpg)
Local Tags
(a) ImageX0 (b) ImageXi
Figure 7: Local Tag Coincidences
15
![Page 16: huzefa slides icip01web.eecs.umich.edu/~hero/Preprints/huzefa_slides_icip01.pdfMI Registration of Gray Levels (Viola&Wells:ICCV95) X:a N N UL image (lexicographically ordered) X (k):](https://reader034.vdocuments.mx/reader034/viewer/2022051806/60018171cf65a227b95548c7/html5/thumbnails/16.jpg)
Spatial Relations Between Local Tags
N
E
SE
S
W
R
(a) ImageX0
N
E
SE
S
W
R
(b) ImageXi
Figure 8: Spatial Relation Coincidences
16
![Page 17: huzefa slides icip01web.eecs.umich.edu/~hero/Preprints/huzefa_slides_icip01.pdfMI Registration of Gray Levels (Viola&Wells:ICCV95) X:a N N UL image (lexicographically ordered) X (k):](https://reader034.vdocuments.mx/reader034/viewer/2022051806/60018171cf65a227b95548c7/html5/thumbnails/17.jpg)
Feature Coincidence Tree of Local Tags
Root Node
Depth 1
Depth 2
Not examined further
Figure 9:Part of feature tree data structure.
Terminal nodes (Depth 16)
Figure 10:Leaves of feature tree data structure.
17
![Page 18: huzefa slides icip01web.eecs.umich.edu/~hero/Preprints/huzefa_slides_icip01.pdfMI Registration of Gray Levels (Viola&Wells:ICCV95) X:a N N UL image (lexicographically ordered) X (k):](https://reader034.vdocuments.mx/reader034/viewer/2022051806/60018171cf65a227b95548c7/html5/thumbnails/18.jpg)
Forests of Randomized Feature TreesRANDOMIZED TREES
Figure 11:Forest of randomized trees
Registration criterion:
T = argmaxTi
# trees
∑t=1
M̂Iα(t)
18
![Page 19: huzefa slides icip01web.eecs.umich.edu/~hero/Preprints/huzefa_slides_icip01.pdfMI Registration of Gray Levels (Viola&Wells:ICCV95) X:a N N UL image (lexicographically ordered) X (k):](https://reader034.vdocuments.mx/reader034/viewer/2022051806/60018171cf65a227b95548c7/html5/thumbnails/19.jpg)
ICA Features
Decomposition ofM�M tag imagesY(k) acquired atk= 1; : : : ;K spatial
locations
Y(k) =
P
∑p=1
akpSp
� fSkg
Pk=1: statistically independent components
� akp: projection coefficients of tagY(k) onto componentSp
� fSkg
Pk=1 andP: selected via MLE and MDL
� Feature vector for coincidence processing:
Z(k) = [a1k; : : : ;aPk]T
19
![Page 20: huzefa slides icip01web.eecs.umich.edu/~hero/Preprints/huzefa_slides_icip01.pdfMI Registration of Gray Levels (Viola&Wells:ICCV95) X:a N N UL image (lexicographically ordered) X (k):](https://reader034.vdocuments.mx/reader034/viewer/2022051806/60018171cf65a227b95548c7/html5/thumbnails/20.jpg)
ICA Basis for Breast 141
Figure 12:Estimated ICA basis set for ultrasound breast image database
20
![Page 21: huzefa slides icip01web.eecs.umich.edu/~hero/Preprints/huzefa_slides_icip01.pdfMI Registration of Gray Levels (Viola&Wells:ICCV95) X:a N N UL image (lexicographically ordered) X (k):](https://reader034.vdocuments.mx/reader034/viewer/2022051806/60018171cf65a227b95548c7/html5/thumbnails/21.jpg)
Simple Example
Figure 13: Bar images with contrast 1.02, 1.07 and 1.78. Background is
low variance white Gaussian while bar is uniform intensity.
21
![Page 22: huzefa slides icip01web.eecs.umich.edu/~hero/Preprints/huzefa_slides_icip01.pdfMI Registration of Gray Levels (Viola&Wells:ICCV95) X:a N N UL image (lexicographically ordered) X (k):](https://reader034.vdocuments.mx/reader034/viewer/2022051806/60018171cf65a227b95548c7/html5/thumbnails/22.jpg)
Single Pixel vs Feature Tag
−5 −4 −3 −2 −1 0 1 2 3 4 50.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
Degree of rotation
Mut
ual I
nfor
mat
ion
( α
~=
1 )
Single pixel based MI peak with submergence of structure in background
intensity ratio = 1.02intensity ratio = 1.07intensity ratio = 1.78
−5 −4 −3 −2 −1 0 1 2 3 4 50.6
0.7
0.8
0.9
1
1.1
1.2
Degree of rotation
Mut
ual I
nfor
mat
ion
( α
~=
1 )
Tag based MI peak with submergence of structure in background
intensity ratio = 1.02intensity ratio = 1.07intensity ratio = 1.78
Figure 14: Upper curves are single pixel based MI trajectories while lower
curves are 4�4 tag based MI trajectories for bar images.
22
![Page 23: huzefa slides icip01web.eecs.umich.edu/~hero/Preprints/huzefa_slides_icip01.pdfMI Registration of Gray Levels (Viola&Wells:ICCV95) X:a N N UL image (lexicographically ordered) X (k):](https://reader034.vdocuments.mx/reader034/viewer/2022051806/60018171cf65a227b95548c7/html5/thumbnails/23.jpg)
UL Registration Comparisons
151 142 162 151/8 151/16 151/32
pixel 0:3=0:9 0:6=0:3 0:6=0:3
tag 0:5=3:6 0:5=3:8 0:4=1:4
spatial-tag 0:99=14:6 0:99=8:4 0:6=8:3
ICA 0:7=4:1 0:7=3:9 0:99=7:7
Table 1: Numerator =optimal values ofα and Denominator = maximum
resolution of mutualα-information for registering various images (Cases
151, 142, 162) using various features (pixel, tag, spatial-tag, ICA). 151/8,
151/16, 151/32 correspond to ICA algorithm with 8, 16 and 32 basis ele-
ments run on case 151.
23
![Page 24: huzefa slides icip01web.eecs.umich.edu/~hero/Preprints/huzefa_slides_icip01.pdfMI Registration of Gray Levels (Viola&Wells:ICCV95) X:a N N UL image (lexicographically ordered) X (k):](https://reader034.vdocuments.mx/reader034/viewer/2022051806/60018171cf65a227b95548c7/html5/thumbnails/24.jpg)
Conclusions
1. Inclusion of better higher order features
2. Implementation of better registration criterion
3. Open issues:
(a) How best to estimateα-MI?
(b) How to determine bestα empirically?
(c) What are best 3D features for coarse registration vs. fine
registration?
24