robust 3d shape correspondence in the spectral domain varun jain and hao (richard) zhang graphics,...
Post on 19-Dec-2015
220 views
TRANSCRIPT
Robust 3D Shape Correspondence in the Spectral Domain
Varun Jain and Hao (Richard) Zhang
Graphics, Usability, and Visualization (GrUVi) LabSchool of Computing Science
Simon Fraser UniversityBurnaby, BC Canada
June 15, 2006
2
The correspondence problem Given two shapes represented by triangle meshes, find
a meaningful correspondence between their vertices
Not a (continuous) parameterization problem, e.g.,
[Kraevoy & Sheffer 04] ― min. distortion, mapped features
Applications: mesh parameterization, morphing, shape
registration, tracking, recognition, and retrieval, etc.
3
Background
A classical problem studied in computer vision mostly
We are interested in fully automatic and purely shape-
based approaches, i.e., without using prior knowledge
Goals: Invariance to common rigid and non-rigid transformation
Robustness against noise, different object size, etc.
Ultimately, return meaningful correspondences
Despite intense studies, all proposed methods can fail
on seemingly easy cases for humans
4
Two basic types of techniques
Extrinsic methods Point coordinates defined in
some global frame Optimization-based and
mostly iterative, e.g., iterative closest point (ICP)
Initial alignment is crucial
Intrinsic methods Point coordinates based on relative information A descriptor defined from the perspective of that point Descriptors can be absolute coordinates, e.g., spectral, or
statistical, e.g., shape contexts [Belongie et al. 02]
Non-rigid ICP [Chui et al. 2004]
rota
tion
5
Related works With the aid of initial manual feature correspondence
Cross parameterization [Praun et al. 01, Kraevoy & Sheffer 04]
Feature-guided ICP [Sumner & Popovic 04]
Barycentric interpolation between features [Zayer et al. 05]
Other deformation based approaches
Automatic extrinsic methods: ICP and its variants
Original ICP [Besl & McKay 92]
Many variants of rigid ICP [Rusinkiewicz & Levoy 01]
Robust ICP based on refinement [Zinber et al. 03] Non-rigid ICP based on thin-plate splines [Chui et al. 03]
6
Related works Local shape descriptors
Shape context [Belongie et al. 02, Körtgen et al. 03]
Spin images [Johnson & Hebert 99]
Other approaches that handle rigid transformations [Gelfand et al. 05, Li & Guskov 05]
Curvature map [Gatzke et al. SMI 05]
Spectral methods Original work on correspondence [Shapiro & Brady 92]
MDS for retrieval of isometric shapes [Elad & Kimmel 03]
Others: compression [Karni & Gotsman 00], spherical parameterization [Gotsman et al. 03], mesh sequencing [Isenberg & Lindstrom 05, Liu et al. 06], segmentation [Liu &
Zhang 04], surface reconstruction [Kolluri et al. 04]
7
Spectral correspondence
[Shapiro & Brady, 92]: Given two point sets P and Q Construct symmetric “affinity” matrices AP and AQ using
pair-wise L2 distances and a Gaussian kernel
Construct spectral embedding by k leading eigenvectors of AP and AQ, sorted by descending eigenvalues
Compute best matching using embedded coordinates via
L2 distance in the k-dimensional spectral domain
Why spectral correspondence? Affinities are intrinsic measure (but high-dimensional)
Eigenvectors have good approximation properties
Spectral embeddings normalize shapes with respect to all rigid body transformations and uniform scaling
8
Key observations
Flexibility of affinity measures
Whichever transformation one needs the correspondence
to be invariant of, build that invariance into affinities
Eigenvectors need to be scaled properly, e.g., at least
to handle data with difference sizes
Eigenvectors can “switch” (never reported before)
Signs of eigenvectors need to be consistent
Non-rigid shape transformation can cause non-rigid
transformation in the spectral domain
9
Summary of our approach
Use geodesic affinities for invariance to shape bending
Eigenvector scaling using squared root of eigenvalues
Proper handling of objects with difference sizes
Eigenvalue decay leaves approach less sensitive to k
Variance-normalization + interpretation from kernel PCA
Heuristics to resolve eigenvector switch and sign flip
Non-rigid ICP via thin-plate splines in spectral domain
Net result:
Proper correspondence of articulated shapes
Consistently more robust correspondence results
10
Evaluation paradigm
Visual examination via color plots Manually color one model based on parts Color second model using computed correspondence
Plot of percentage of correct matches Manually provided ground truth (small feature sets) Ground truth automatically identified via “in-place”
mesh decimation
Plot of correspondence error Sum of correspondence errors at the vertices Error at a vertex: geodesic distance between ground
truth and computed corresponding point
11
Basic steps of our algorithm
Eigenvector scaling
Non-rigid ICP via thin-plate splines
Geodesic-based
spectral embedding
Best
matching
12
Geodesic affinities
Given two meshes M1 and M2 of sizes n1 and n2
Affinity matrices A1 (n1n1) and A2 (n2n2) given by
where d1 and d2 are geodesic distances on M1 and M2
Gaussian: importance of far-away vertices attenuated
Gaussian width set as maximum geodesic distance
Other kernel, e.g., exponential, may be applied
22
22
21
21
2
),(
22
),(
1 and jid
ij
jid
ij eAeA
13
Spectral embeddings
Eigen-decompose each affinity matrix A = UUT
Obtain k-D spectral embedding of mesh vertices
using the k leading (scaled) eigenvectors of A
First eigenvector ignored as it is almost a constant
k
uuu
ikii
k
kk
uuu
E
21
2211
k-dimensional spectral embedding coordinates of ith the point of mesh
15
Eigenvector scaling
EkEkT gives the best rank-k approximation of the
affinity matrix A (namely, dot product affinity)
Scaling using the square root of the eigenvalues is
shown to normalize the
variance of data
The scaling is also a
natural one when seen
from the perspective of
kernel PCA [Jain 2006]
16
Eigenvector switching and sign flips
Signs of the eigenvectors are decided by eigensolver
and are difficult to correspond automatically
Discrepancy between shapes can also cause certain
eigenvectors to switch places
An eigenvector switch or a sign flip corresponds to a
reflection in the spectral domain
17
Exhaustive search and greedy heuristic
Reflection-invariant shape descriptors possible, e.g.,
high-D shape context or symmetric polynomials, but
more invariance less descriptivity [Frome et al. 04]
Choose among 2kk! possible eigenvector ordering and
sign flips to minimize a correspondence cost:
Besides exhaustive search (for very small k), can use
greedy scheme to order one eigenvector at a time
1
1
22
)(1 ˆˆ
n
iiCi EE
18
Non-rigid transformations
Perform non-rigid ICP using thin-plate splines in the spectral domain
Experimentally, very fast convergence (5-10 iterations)
19
Recap of algorithm
Eigenvector scaling
Non-rigid ICP via thin-plate splines
Geodesic-based
spectral embedding
Best
matching
20
Result: % of correct correspondences
Manual initial alignment used for first three!
% is out of 17-20 ground-truth matches
(200-300 vertices; k = 5 eigenvectors used throughout)
21
Visual results for correspondence
Models with articulation and moderate stretching
Many more results in color plate (page 300).
22
Limitations Intrinsic geodesic affinities
Symmetry issue
Topological issue: hybrid approach [Jain & Zhang 06]
Rather primitive heuristic for resolving eigenvector
switching and sign flips Effectiveness attributed to spectral normalization
Euclidean metric as correspondence cost No particular reason, except for a computational one
Challenge: what is right?
Computational complexity: O(n2logn)
23
Follow-up and future works Sampling via Nyström approximation [Liu et al. 06]
Spectral embedding: O(n2logn) O(pnlogn + p3)
Little loss of quality at low sampling (10 out of 4000)
Farthest point sampling used
More sophisticated sampling schemes [Liu & Zhang 06]
Retrieval of articulated shapes [Jain & Zhang 06]
Outperforms light-field descriptor [Chen et al. 03] and spherical Harmonics descriptor [Kazhdan et al. 03] (even when these are applied to spectral embeddings)
But not so on Princeton Benchmark database (yet) due to various artifacts in the models
How about eigenspaces?