functional maps: a flexible representation of maps between shapes
DESCRIPTION
Functional maps: A Flexible Representation of Maps Between Shapes. SIGGRAPH 2012. Maks Ovsjanikov 1 Mirela Ben-Chen 2 Justin Solomon 2 Adrian Butscher 2 Leonidas Guibas 2 1 LIX, Ecole Polytechnique 2 Stanford University. Presenter Yunhai @VCC. Introduction. Shape matching - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Functional maps: A Flexible Representation of Maps Between Shapes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/56816556550346895dd7d363/html5/thumbnails/1.jpg)
Functional maps: A Flexible Representation of Maps Between Shapes
SIGGRAPH 2012
Maks Ovsjanikov1 Mirela Ben-Chen2 Justin Solomon2
Adrian Butscher2 Leonidas Guibas2
1LIX, Ecole Polytechnique 2Stanford University
Presenter Yunhai@VCC
![Page 2: Functional maps: A Flexible Representation of Maps Between Shapes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/56816556550346895dd7d363/html5/thumbnails/2.jpg)
Introduction
• Shape matching• Rigid (rotation + translation)• Non-rigid (pairings of points or regions)
Correspondence
![Page 3: Functional maps: A Flexible Representation of Maps Between Shapes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/56816556550346895dd7d363/html5/thumbnails/3.jpg)
Correspondence
• The space of correspondences is exponential• Isometric matching: still a QAP (NP-hard)• No map continuity or global consistency
![Page 4: Functional maps: A Flexible Representation of Maps Between Shapes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/56816556550346895dd7d363/html5/thumbnails/4.jpg)
Correspondence
Dp
D(p)
Dq D(q)
dist(p,q) dist((p), (q))
![Page 5: Functional maps: A Flexible Representation of Maps Between Shapes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/56816556550346895dd7d363/html5/thumbnails/5.jpg)
Correspondence approaches
• Select small set of landmark points• Compute correspondences for subset• Extend sparse correspondence to dense
• Coarse similarities or symmetry ambiguities?
![Page 6: Functional maps: A Flexible Representation of Maps Between Shapes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/56816556550346895dd7d363/html5/thumbnails/6.jpg)
Do we need Point-to-point correspondence?
• It is neither possible nor necessary , because– inherent shape ambiguities or – the user may only be interested in
approximate alignment
![Page 7: Functional maps: A Flexible Representation of Maps Between Shapes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/56816556550346895dd7d363/html5/thumbnails/7.jpg)
This paper
• Mappings between functions on the shapes, generalizes the notion of correspondence
• Inference and manipulation of maps
![Page 8: Functional maps: A Flexible Representation of Maps Between Shapes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/56816556550346895dd7d363/html5/thumbnails/8.jpg)
Overview
• Functional map representation• Properties (basis, continuity, linearity, …)• Inference (obtain a point-to-point map)• Applications– Shape matching– Map improvement– Shape collections– Segmentation transfer
![Page 9: Functional maps: A Flexible Representation of Maps Between Shapes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/56816556550346895dd7d363/html5/thumbnails/9.jpg)
• Bijective mapping• Manifolds M and N• T induces a transformation for functions
• Given scalar function We obtain with
Functional map representation
![Page 10: Functional maps: A Flexible Representation of Maps Between Shapes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/56816556550346895dd7d363/html5/thumbnails/10.jpg)
• Bijective mapping• Manifolds M and N• T induces a transformation for functions
• Given scalar function We obtain with Induced transformation Generic space of real-value functions
Functional map representation
![Page 11: Functional maps: A Flexible Representation of Maps Between Shapes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/56816556550346895dd7d363/html5/thumbnails/11.jpg)
Functional map representation
• T can be recovered from TF
– Use an indicator function f(a) = 1
• TF is a linear map between function spaces
![Page 12: Functional maps: A Flexible Representation of Maps Between Shapes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/56816556550346895dd7d363/html5/thumbnails/12.jpg)
Functional map representation
• M and N have sets of basis functions Then, and • So, we can write TF in terms of the bases
C is then like a change of basis matrix
![Page 13: Functional maps: A Flexible Representation of Maps Between Shapes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/56816556550346895dd7d363/html5/thumbnails/13.jpg)
Functional map representation
• Generalized linear functional mapping
![Page 14: Functional maps: A Flexible Representation of Maps Between Shapes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/56816556550346895dd7d363/html5/thumbnails/14.jpg)
Examples of functional maps
Using Laplace-Beltrami eigenfunctionsLinear functional mapping is a 20x20 matrix
![Page 15: Functional maps: A Flexible Representation of Maps Between Shapes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/56816556550346895dd7d363/html5/thumbnails/15.jpg)
Examples of functional maps
![Page 16: Functional maps: A Flexible Representation of Maps Between Shapes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/56816556550346895dd7d363/html5/thumbnails/16.jpg)
Examples of functional maps
![Page 17: Functional maps: A Flexible Representation of Maps Between Shapes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/56816556550346895dd7d363/html5/thumbnails/17.jpg)
Properties: choice of basis
• Example of basis– Indicator functions Permutation matrix
• Motivation– Reduce representation complexity – Better suited for continuous mappings
• Choose based on compactness and stability
![Page 18: Functional maps: A Flexible Representation of Maps Between Shapes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/56816556550346895dd7d363/html5/thumbnails/18.jpg)
Properties: choice of basis
• Compactness: most natural functions should be well approximated by a few basis elements
• Stability: the space of functions spanned by all linear combinations of basis must be stable under small shape deformations
![Page 19: Functional maps: A Flexible Representation of Maps Between Shapes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/56816556550346895dd7d363/html5/thumbnails/19.jpg)
Properties: choice of basis
• Laplace-Beltrami eigenfunctions as the basis
• Although individual eigenfunctions are unstable, space of functions spanned is “stable”
![Page 20: Functional maps: A Flexible Representation of Maps Between Shapes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/56816556550346895dd7d363/html5/thumbnails/20.jpg)
Properties: choice of basis
• Compare two discretizations of the Laplace-Beltrami operator [Meyer et al. 2002]– With and without area normalization
![Page 21: Functional maps: A Flexible Representation of Maps Between Shapes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/56816556550346895dd7d363/html5/thumbnails/21.jpg)
Properties: choice of basis
Meshes: 27.8K points
Good qualityWith a 40x40 matrix, we have 17 times memory savings over a permutation of size 27.8K
![Page 22: Functional maps: A Flexible Representation of Maps Between Shapes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/56816556550346895dd7d363/html5/thumbnails/22.jpg)
Properties: choice of basis
Plus: near-isometric maps induce sparse matrices
![Page 23: Functional maps: A Flexible Representation of Maps Between Shapes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/56816556550346895dd7d363/html5/thumbnails/23.jpg)
Properties: continuity
• Naturally handles map continuity• Three types of continuity:– Changes of the input functionImage varies continuously under changes of a– Image functionLaplace-Beltrami is well-suited for smooth functions is smooth– Representation• Any matrix C is a functional mapping
![Page 24: Functional maps: A Flexible Representation of Maps Between Shapes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/56816556550346895dd7d363/html5/thumbnails/24.jpg)
Properties: continuity
Mapping obtained using an interpolation between two maps: C = C1 + (1- )C2
![Page 25: Functional maps: A Flexible Representation of Maps Between Shapes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/56816556550346895dd7d363/html5/thumbnails/25.jpg)
Properties: linearity of constraints
• Descriptor preservation, e.g., f(x) = (x)– Function pres. implies approximate desc. pres.
• Landmark point correspondences– Use f(x) that is distance function to the landmark
• Segment correspondences– Also distance functions or indicator functions
![Page 26: Functional maps: A Flexible Representation of Maps Between Shapes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/56816556550346895dd7d363/html5/thumbnails/26.jpg)
Properties: operator commutativity
• W.r.t. linear operators on the shapes– E.g., symmetry
• An example– C is a map between a man and a woman– RF ,SF is the map the left hand to right hand on the
woman and man, respectively– Man ->Woman, Woman left->Woman right Man left ->
Woman right– Man left -> Man Right Man -> Woman Man Left ->
Woman Right
![Page 27: Functional maps: A Flexible Representation of Maps Between Shapes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/56816556550346895dd7d363/html5/thumbnails/27.jpg)
Properties: regularization constraints
• Not every matrix C is a point-to-point map• It is most meaningful to consider orthonormal
or nearly-orthonormal functional matrices
![Page 28: Functional maps: A Flexible Representation of Maps Between Shapes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/56816556550346895dd7d363/html5/thumbnails/28.jpg)
Map inversion and composition
• Finding an inverse of a map that is not a bijection can be challenging
• In the functional case, an inverse is given simply by the inverse of the matrix C
• A good approximation is the transpose of C
• Composition becomes matrix multiplication
![Page 29: Functional maps: A Flexible Representation of Maps Between Shapes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/56816556550346895dd7d363/html5/thumbnails/29.jpg)
Functional map inference
• Construct a large system of equations• Each equation is one constraint• Constraint for function preservation• Find the matrix C that satisfies the constraints• In fact, we will need many constraints to
obtain C in a least-squares sense• Thus, we need candidate point-to-point or
segment-to-segment correspondences
![Page 30: Functional maps: A Flexible Representation of Maps Between Shapes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/56816556550346895dd7d363/html5/thumbnails/30.jpg)
Map refinement and conversion• Refine C and convert to a point-to-point map
• For each point x is the source embedding• Find closest point x’ in the target embedding• Find the optimal C by minimizing |Cx – x’|• Iterate this procedure
• Similar to spectral matching [Jain et al. 2007]• Differences
– Good initial estimate C– “Mixing” across eigenvectors
![Page 31: Functional maps: A Flexible Representation of Maps Between Shapes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/56816556550346895dd7d363/html5/thumbnails/31.jpg)
Applications: shape matching
• Compute Laplace-Beltrami eigenfunctions• Compute shape descriptors (WKS)• Compute segment correspondences• Add constraints into a linear system and solve• Refine the solution C• Obtain point-to-point correspondences
![Page 32: Functional maps: A Flexible Representation of Maps Between Shapes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/56816556550346895dd7d363/html5/thumbnails/32.jpg)
Applications: shape matching
• Applied on the benchmark of Kim et al. 2011
Examples of correspondences obtained
![Page 33: Functional maps: A Flexible Representation of Maps Between Shapes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/56816556550346895dd7d363/html5/thumbnails/33.jpg)
Applications: shape matching
Correspondence results on two datasets
![Page 34: Functional maps: A Flexible Representation of Maps Between Shapes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/56816556550346895dd7d363/html5/thumbnails/34.jpg)
Applications: map improvement
• The strength lies on the representation itself• Take point-to-point maps computed by the
other methods• Improve the maps with the functional maps• Adds regularization• Takes 15s for 50K point mesh in 3GHz CPU
![Page 35: Functional maps: A Flexible Representation of Maps Between Shapes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/56816556550346895dd7d363/html5/thumbnails/35.jpg)
Applications: map improvement
Color shows location of errors
![Page 36: Functional maps: A Flexible Representation of Maps Between Shapes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/56816556550346895dd7d363/html5/thumbnails/36.jpg)
Applications: map improvement
Color shows location of errors
![Page 37: Functional maps: A Flexible Representation of Maps Between Shapes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/56816556550346895dd7d363/html5/thumbnails/37.jpg)
Applications: map improvement
Evaluation on the benchmark
![Page 38: Functional maps: A Flexible Representation of Maps Between Shapes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/56816556550346895dd7d363/html5/thumbnails/38.jpg)
Applications: shape collections
• Iteratively Corrected Shape Maps (ICSM) [Nguyen et al. 2011] – compose maps on cycles L→M →N →L– Compare result to the identity map
• Map diffusion [Singer and Wu 2011]– construct a “SuperMap” for the whole collection,– replace a map with a weighted average of other
maps
![Page 39: Functional maps: A Flexible Representation of Maps Between Shapes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/56816556550346895dd7d363/html5/thumbnails/39.jpg)
Applications: shape collections
ICSM applied on the SCAPE dataset using the functional mapsEach entry is the average geodesic map between 11 shapes
![Page 40: Functional maps: A Flexible Representation of Maps Between Shapes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/56816556550346895dd7d363/html5/thumbnails/40.jpg)
Applications: shape collections
Geodesic errors of the mappings in this dataset
![Page 41: Functional maps: A Flexible Representation of Maps Between Shapes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/56816556550346895dd7d363/html5/thumbnails/41.jpg)
Applications: segmentation transfer
• Functional maps reduce the transfer of functions to matrix multiplication
• Without resorting to point-to-point maps
• Use indicator functions for segments• Perform matrix multiplication• Transform attributes into a “hard” clustering
![Page 42: Functional maps: A Flexible Representation of Maps Between Shapes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/56816556550346895dd7d363/html5/thumbnails/42.jpg)
Applications: segmentation transfer
Source segmentation, indicator function for one segment, transferred segmentation
![Page 43: Functional maps: A Flexible Representation of Maps Between Shapes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/56816556550346895dd7d363/html5/thumbnails/43.jpg)
Applications: segmentation transfer
Source segmentation, indicator function for one segment, transferred segmentation
![Page 44: Functional maps: A Flexible Representation of Maps Between Shapes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/56816556550346895dd7d363/html5/thumbnails/44.jpg)
Conclusion
• Novel representation of maps between shapes• Generalizes point-to-point maps• Constraints become linear
• More general classes of deformations?– Optimal choice of basis
![Page 45: Functional maps: A Flexible Representation of Maps Between Shapes](https://reader035.vdocuments.mx/reader035/viewer/2022062501/56816556550346895dd7d363/html5/thumbnails/45.jpg)
Discussion
• In practice, could pose the method as: “find a correspondence between eigenfunctions”? Actually, People has used spectral basis to compute the shape map– JAIN, V., ZHANG, H., AND VAN KAICK, O. 2007. Non-rigid spectral
correspondence of triangle meshes. International Journal on Shape Modeling 13, 1, 101–124
– MATEUS, D., HORAUD, R. P., KNOSSOW, D., CUZZOLIN, F., AND BOYER, E. 2008. Articulated shape matching using Laplacian eigenfunctions and unsupervised point registration. In Proc. CVPR, 1–8.
• What does the representation really bring? a new concept functional map?