image registration lecture 16: view-based registration may 3, 2005
DESCRIPTION
Image Registration Lecture 16: View-Based Registration May 3, 2005. Prof. Charlene Tsai. Overview. Retinal image registration The Dual-Bootstrap ICP algorithm Covariance matrix Covariance propagation Model selection View-based registration Software design. - PowerPoint PPT PresentationTRANSCRIPT
Image Registration Lecture 16: View-Based Registration
May 3, 2005
Image Registration Lecture 16: View-Based Registration
May 3, 2005
Prof. Charlene TsaiProf. Charlene Tsai
Image Registration Lecture 16 2
OverviewOverview
Retinal image registration The Dual-Bootstrap ICP algorithm
Covariance matrix Covariance propagation Model selection
View-based registration Software design
Retinal image registration The Dual-Bootstrap ICP algorithm
Covariance matrix Covariance propagation Model selection
View-based registration Software design
Image Registration Lecture 16 3
Retinal Image Registration: ApplicationsRetinal Image Registration: Applications
Mosaics Multimodal integration Change detection
Mosaics Multimodal integration Change detection
Image Registration Lecture 16 4
MosaicsMosaics
Image Registration Lecture 16 5
Multimodal IntegrationMultimodal Integration
Image Registration Lecture 16 6
Change VisualizationChange Visualization
Image Registration Lecture 16 7
Retinal Image Registration - PreliminariesRetinal Image Registration - Preliminaries
Features Transformation models Initialization
Features Transformation models Initialization
Image Registration Lecture 16 8
landmarks vascular centerlines
FeaturesFeatures
Vascular centerline points Discrete locations along
the vessel contours Described in terms of
pixel locations, orientations, and widths
Vascular landmarks Pixel locations,
orientations and width of vessels that meet to form landmarks
Vascular centerline points Discrete locations along
the vessel contours Described in terms of
pixel locations, orientations, and widths
Vascular landmarks Pixel locations,
orientations and width of vessels that meet to form landmarks
Image Registration Lecture 16 9
Transformation ModelsTransformation Models
)(pXq Tyxyxyx 221)( pX
Model Parameter Matrix DoF Accuracy (pixels)
Similarity 4 5.05
Affine 6 4.58
Reduced quadratic
6 2.41
Full quadratic 12 0.64
000
000
abt
bat
y
x
262524232221
161514131211
000
000
dbt
cat
y
x
ffabt
eebat
y
x
0
0
Image Registration Lecture 16 10
Initializing RegistrationInitializing Registration
Form list of landmarks in each image Form matches of one landmark from
each image The selection of these matches
will be discussed in Lectures 18 and 19
Choose matches, one at a time For each match:
Compute an initial similarity transformation in the small image region surrounding the landmarks
Apply Dual-Bootstrap ICP procedure to see if the initial alignment can be successfully grown into an accurate, image-wide alignment
End when one match leads to success, or all matches are exhausted
Form list of landmarks in each image Form matches of one landmark from
each image The selection of these matches
will be discussed in Lectures 18 and 19
Choose matches, one at a time For each match:
Compute an initial similarity transformation in the small image region surrounding the landmarks
Apply Dual-Bootstrap ICP procedure to see if the initial alignment can be successfully grown into an accurate, image-wide alignment
End when one match leads to success, or all matches are exhausted
Image Registration Lecture 16 11
Dual-Bootstrap - OverviewDual-Bootstrap - Overview
Match and refine estimate in each region
Bootstrap the model: Low-order for small
regions; High-order for large Automatic selection
Bootstrap the region: Covariance propagation
gives uncertainty
Iterate until convergence
Image Registration Lecture 16 12
Matching and Estimation in Each RegionMatching and Estimation in Each Region
Matching - standard stuff: Vascular centerline points from within current region of
moving image Mapped using current transform estimate Find closest point using Borgefors digital distance map
Estimation: Fix scale estimate Run IRLS
Matching - standard stuff: Vascular centerline points from within current region of
moving image Mapped using current transform estimate Find closest point using Borgefors digital distance map
Estimation: Fix scale estimate Run IRLS
Image Registration Lecture 16 13
Covariance Matrix of EstimateCovariance Matrix of Estimate
Measures uncertainty in estimate of transformation parameters
Basis for region growth and model selection The next few slides will give an overview of
computing an approximate covariance matrix We’ll start with linear regression
Measures uncertainty in estimate of transformation parameters
Basis for region growth and model selection The next few slides will give an overview of
computing an approximate covariance matrix We’ll start with linear regression
Image Registration Lecture 16 14
Problem Formulation in Linear RegressionProblem Formulation in Linear Regression
Independent (non-random) variable values:
Dependent (random) variable values
Linear relationship based on k+1 dimensional parameter vector a:
Independent (non-random) variable values:
Dependent (random) variable values
Linear relationship based on k+1 dimensional parameter vector a:
Image Registration Lecture 16 15
Least-Squares FormulationLeast-Squares Formulation
Least-squares error term:
Here:
Least-squares error term:
Here:
Image Registration Lecture 16 16
Estimate and Covariance MatrixEstimate and Covariance Matrix
Estimate:
Residual error variance (square of “scale”):
Parameter estimate covariance
Estimate:
Residual error variance (square of “scale”):
Parameter estimate covariance
scatter matrix for 1st order
Shape of the cov matrixmagnitude
Equation #1
Image Registration Lecture 16 17
Aside: Equation #1Aside: Equation #1
How to obtain Assuming that y is the only random variable,
independent and identically distributed, with covariance
The covariance matrix of a is
How to obtain Assuming that y is the only random variable,
independent and identically distributed, with covariance
The covariance matrix of a is
][)(][ 1 yXEXXaaE T
12
112
11
11
ˆ
ˆ
])ˆ)(ˆ[(
T
TTT
TTTT
TTTT
Ta
XX
XXXIXXX
XXXyEyyEyXEXX
XXXyEyyEyXXXE
aaaaE
I2̂
Image Registration Lecture 16 18
Aside: Line Fitting in 2DAside: Line Fitting in 2D
Form of the equation:
If the xi values are centered:
Then the parameters are independent with variances
for the linear and constant terms, respectively
Form of the equation:
If the xi values are centered:
Then the parameters are independent with variances
for the linear and constant terms, respectively
Image Registration Lecture 16 19
Hessians and CovariancesHessians and Covariances
Back to k dimensions, re-consider the objective function:
Compute the Hessian matrix:
Observe the relationship
Back to k dimensions, re-consider the objective function:
Compute the Hessian matrix:
Observe the relationship
Image Registration Lecture 16 20
Hessians and CovariancesHessians and Covariances
This is exact for linear regression, but serves as a good approximation for non-linear least-squares
In general, the Hessian will depend on the estimate (in regression it doesn’t because the problem is quadratic), so the approximate relationship is
This is exact for linear regression, but serves as a good approximation for non-linear least-squares
In general, the Hessian will depend on the estimate (in regression it doesn’t because the problem is quadratic), so the approximate relationship is
Image Registration Lecture 16 21
Hessian in RegistrationHessian in Registration
Recall the weighted least-squares objective function:
Keeping the correspondences and the weights fixed,
where Dk gives the error of the k-th correspondence
Inverting this gives the covariance approximation. This approximation is only good when the estimate is
fairly accurate
Recall the weighted least-squares objective function:
Keeping the correspondences and the weights fixed,
where Dk gives the error of the k-th correspondence
Inverting this gives the covariance approximation. This approximation is only good when the estimate is
fairly accurate
Image Registration Lecture 16 22
Back to Dual-Bootstrap ICPBack to Dual-Bootstrap ICP
Covariance is used in two ways in each DB-ICP iteration: Determining the region incorporates enough
constraints to switch to a more complex model Similarity => Affine => Reduced Quadratic => Quadratic
Determining the growth of the dual-bootstrap region: More stable transformation estimates lead to faster
growth
Covariance is used in two ways in each DB-ICP iteration: Determining the region incorporates enough
constraints to switch to a more complex model Similarity => Affine => Reduced Quadratic => Quadratic
Determining the growth of the dual-bootstrap region: More stable transformation estimates lead to faster
growth
Image Registration Lecture 16 23
Model SelectionModel Selection
What model should be used to describe a given set of data?
Classic problem in statistics, and many methods have been proposed
Most trade-off the fitting accuracy of higher-order models with the stability (or lower complexity) of lower-order models
What model should be used to describe a given set of data?
Classic problem in statistics, and many methods have been proposed
Most trade-off the fitting accuracy of higher-order models with the stability (or lower complexity) of lower-order models
Image Registration Lecture 16 24
Model Selection in DB-ICPModel Selection in DB-ICP
Use correspondence set Estimate the IRLS parameters and covariance matrices for each
model in current set
For each model (with dm parameters) this generates a set of weights and errors and a covariance matrix:
Choose the model maximizing the model selection equation (derived from Bayesian modeling):
The first two terms increase with increasingly complex models; the last term decreases
Use correspondence set Estimate the IRLS parameters and covariance matrices for each
model in current set
For each model (with dm parameters) this generates a set of weights and errors and a covariance matrix:
Choose the model maximizing the model selection equation (derived from Bayesian modeling):
The first two terms increase with increasingly complex models; the last term decreases
stabilityaccuracy
Image Registration Lecture 16 25
Region Growth in DB-ICPRegion Growth in DB-ICP
Grow each side independently
Grow is inversely proportional to uncertainty in mapping of boundary point on the center of each side
New rectangular region found from the new positions of each of the boundary points
Grow each side independently
Grow is inversely proportional to uncertainty in mapping of boundary point on the center of each side
New rectangular region found from the new positions of each of the boundary points
Image Registration Lecture 16 26
Aside: Covariance Propagation and Transfer Error
Aside: Covariance Propagation and Transfer Error
Given mapping function:
We will treat as a random variable, but not gk
Uncertainty in makes gk’ a random variable.
What then is the covariance of gk’?
We solve this using standard covariance propagation techniques:
Compute the Jacobian of the transformation, evaluated at gk:
Pre- and post-multiply to obtain the covariance of gk’
In computer vision, this is called the “transfer error”
Given mapping function:
We will treat as a random variable, but not gk
Uncertainty in makes gk’ a random variable.
What then is the covariance of gk’?
We solve this using standard covariance propagation techniques:
Compute the Jacobian of the transformation, evaluated at gk:
Pre- and post-multiply to obtain the covariance of gk’
In computer vision, this is called the “transfer error”
Image Registration Lecture 16 27
Outward Growth of a SideOutward Growth of a Side
Let k be the outward normal of the side, and let rk be the distance of the side from the center of the region
Project the transfer error covariance onto k to obtain a scalar variance k
The outward growth (along normal k) is
where controls the maximum growth rate, which occurs when k < 1
Let k be the outward normal of the side, and let rk be the distance of the side from the center of the region
Project the transfer error covariance onto k to obtain a scalar variance k
The outward growth (along normal k) is
where controls the maximum growth rate, which occurs when k < 1
Image Registration Lecture 16 28
Putting It All Together - The Example, RevisitedPutting It All Together - The Example, Revisited
Image Registration Lecture 16 29
Turning to the SoftwareTurning to the Software
A “view” is a definition or snapshot of the registration problem.
A “view” contains: An image region (current region, plus goal region) A current transformation estimate and estimator A current stage (resolution) of registration
Views work in conjunction with multistage registration
A “view” is a definition or snapshot of the registration problem.
A “view” contains: An image region (current region, plus goal region) A current transformation estimate and estimator A current stage (resolution) of registration
Views work in conjunction with multistage registration
Image Registration Lecture 16 30
View-Based Registration - ProceduralView-Based Registration - Procedural
The following is repeated for each initial estimate For each stage:
Do Match Compute weights Estimate scale For each model
Run IRLS to estimate parameters and covariances Re-estimate scale Generate next view
Choose best model Grow region
Until region has converged and highest order model used Prepare for next stage
The following is repeated for each initial estimate For each stage:
Do Match Compute weights Estimate scale For each model
Run IRLS to estimate parameters and covariances Re-estimate scale Generate next view
Choose best model Grow region
Until region has converged and highest order model used Prepare for next stage
Image Registration Lecture 16 31
ImplementationImplementation
rgrl_view Store the information about the view
rgrl_view_generator Generate the next view
rgrl_view_based_registration Mirrors rgrl_feature_based_registration
with modifications based on the outline on previous slide
Example rgrl/example/registration_retina.cxx
rgrl_view Store the information about the view
rgrl_view_generator Generate the next view
rgrl_view_based_registration Mirrors rgrl_feature_based_registration
with modifications based on the outline on previous slide
Example rgrl/example/registration_retina.cxx
Image Registration Lecture 16 32
SummarySummary
Retina registration: Models, features and initialization
DB-ICP: Matching, estimation and covariances Model selection Region growing
Generalization to view-based registration and its implementation in the toolkit.
Retina registration: Models, features and initialization
DB-ICP: Matching, estimation and covariances Model selection Region growing
Generalization to view-based registration and its implementation in the toolkit.