Bayesian Inference of Bijective Non-Rigid Shape Correspondence

Matthias Vestner1, Roee Litman2, Alex Bronstein2,3,5, Emanuele Rodola1,4 and Daniel Cremers11Technische Universitat Munchen, Germany

2Tel-Aviv Univeristy, Israel3Technion, Israel Institute of Technology, Israel4Universita della Svizzera italiana, Switzerland

5Perceptual Computing Group, Intel, Israel

July 13, 2016

Abstract

Many algorithms for the computation of correspon-dences between deformable shapes rely on some vari-ant of nearest neighbor matching in a descriptorspace. Such are, for example, various point-wisecorrespondence recovery algorithms used as a post-processing stage in the functional correspondenceframework. In this paper, we show that such fre-quently used techniques in practice suffer from lackof accuracy and result in poor surjectivity. We pro-pose an alternative recovery technique guaranteeing abijective correspondence and producing significantlyhigher accuracy. We derive the proposed methodfrom a statistical framework of Bayesian inferenceand demonstrate its performance on several challeng-ing deformable 3D shape matching.

1 Introduction

In geometry processing, computer graphics, and vi-sion, estimating correspondence between 3D shapesaffected by different transformations is one of the fun-damental problems with a wide spectrum of appli-cations ranging from texture mapping to animation[17]. These problems are becoming increasingly im-portant due to the emergence of affordable 3D sens-ing technology. Of particular interest is the setting inwhich the objects are allowed to deform non-rigidly.

1.1 Related works

A traditional approach to correspondence problemsis finding a point-wise matching between (a subsetof) the points on two or more shapes. Minimum-distortion methods establish the matching by mini-mizing some structure distortion, which can includesimilarity of local features [30, 11, 6, 44], geodesic[27, 10, 12] or diffusion distances [14], or a combina-tion thereof [40]. Windheuser et al. [43] used thethin shell elastic energy of triangles, while Zeng etal. [45] used higher-order structures. Typically, thecomputational complexity of such methods is high,and there have been several attempts to alleviatethe computational complexity using hierarchical [36]or subsampling [39] methods. Several approachesformulate the correspondence problem as quadraticassignment and employ different relaxations thereof[41, 23, 33, 2, 12, 18]. Algorithms in this category typ-ically produce guaranteed bijective correspondencesbetween a sparse set of points, or a dense correspon-dence suffering from poor surjectivity.

Embedding methods try to exploit some assump-tion on the correspondence (e.g. approximate isome-try) in order to parametrize the correspondence prob-lem with a few degrees of freedom. Elad and Kim-mel [16] used multi-dimensional scaling to embed thegeodesic metric of the matched shapes into a low-dimensional Euclidean space, where alignment of theresulting “canonical forms” is then performed by sim-

1

arX

iv:1

607.

0342

5v1

[cs

.CV

] 1

2 Ju

l 201

6

1% 3% 5% 7%×diam

NN NN+Bayes CPD CPD+Bayes NN NN+Bayes CPD CPD+Bayes

Figure 1: Qualitative comparison of methods for pointwise correspondence recovery from a functional map.Current methods such as Nearest Neighbors (NN) and coherent point drift (CPD) suffer from bad accuracyand lack of surjectivity. Applying the proposed Bayesian estimation to either of them gives a guaranteedbijective matching with high accuracy. Left: We visualize the accuracy of the methods by transferring texturefrom the source shape X to the target shape Y. Neither Nearest Neighbors nor CPD produce bijectivemappings. The lack of surjectivity is visualized by assigning a fixed color (green) to y /∈ im(X ). Right: Thegeodesic error (distance between groundtruth and recovered match, relative to the shape diameter) inducedby the matching is visualized on the target shape Y.

ple rigid matching (ICP) [13, 8]. The works of [25, 37]used the eigenfunctions of the Laplace-Beltrami oper-ator as embedding coordinates and performed match-ing in the eigenspace. Lipman et al. [24, 19, 20]used conformal embeddings into disks and spheres toparametrize correspondences between homeomorphicsurfaces as Mobius transformations. By using locallyinjective flattenings, [4] achieve guaranteed bijectivematching. However, the majority of the matchingprocedures performed in the embedding space oftenproduces noisy correspondences at fine scales, andsuffers from poor surjectivity.

As opposed to point-wise correspondence methods,soft correspondence approaches assign a point on oneshape to more than one point on the other. Severalmethods formulated soft correspondence as a mass-transportation problem [26, 38]. Ovsjanikov et al.[29] introduced the functional correspondence frame-work, modeling the correspondence as a linear opera-tor between spaces of functions on two shapes, whichhas an efficient representation in the Laplacian eigen-bases. This approach was extended in several follow-up works [31, 22, 1] . A point-wise map is typicallyrecovered from a low-rank approximation of the func-tional correspondence by a matching procedure in the

representation basis, which also suffers from poor sur-jectivity.

1.2 Main contributions

As the main contribution of this paper we see theformulation of the intrinsic map denoising problem:Given a set of point-wise correspondences betweentwo shapes coming from any correspondence algo-rithm (for example, using one of the recovery algo-rithms outlined in Section 2), we consider them as anoisy realization of a latent bijective correspondence.We estimate this bijection using an intrinsic equiv-alent of the standard minimum mean squared error(MMSE) or minimum mean absolute error (MMAE)Bayesian estimators. To the best of our knowledge,despite their simplicity, these tools have not been pre-viously used for deformable shape analysis.

We show that the considered family of Bayesian es-timators leads to a linear assignment problem (LAP)guaranteeing bijective correspondence between theshapes. Despite the common wisdom, we demon-strate that the problem is efficiently solvable for rel-atively densely sampled shapes by means of the well-established auction algorithm [7] and a simple multi-

2

scale approach.Finally, we present a significant amount of empir-

ical evidence that the proposed denoising procedureconsistently improves the quality of the input corre-spondence coming from different algorithms.

2 Pointwise map recovery

We start by briefly overviewing several recent tech-niques used for the computation of pointwise cor-respondences between non-rigid shapes. We focuson approaches relying on the functional map formal-ism merely because these techniques produce state-of-the-art results, emphasizing that the proposed al-gorithm can accept any point-wise correspondence asthe input.

We model shapes as connected two-dimensionalRiemannian manifolds X (possibly with boundary)endowed with the standard measure da induced bythe volume form. Shape X is equipped with thesymmetric Laplace-Beltrami operator ∆X , general-izing the notion of Laplacian to manifolds. Themanifold Laplacian yields an eigen-decomposition∆Xφi = λiφi for i ≥ 1, with eigenvalues 0 = λ1 <λ2 ≤ . . . and eigenfunctions φii≥1 forming an or-thonormal basis of L2(X ). Due to the latter property,any function f ∈ L2(X ) can be represented via the(manifold) Fourier series expansion

f(x) =∑i≥1

〈f, φi〉Xφi(x) , (1)

where we use the standard manifold inner product〈f, g〉X =

∫X fgda.

Consider two manifolds X and Y, and let π : X →Y be a bijective mapping between them. In [29] itwas proposed to consider an operator T : L2(X ) →L2(Y), mapping functions on X to functions on Yvia the composition T (f) = f π−1. This sim-ple change in paradigm remarkably allows to identifymaps between manifolds as linear operators (namedfunctional maps) between Hilbert spaces. Because Tis a linear operator, it admits a matrix representa-tion with respect to a choice of bases φii≥1 andψii≥1 on L2(X ) and L2(Y), respectively. Assum-ing the bases to be orthogonal, the matrix CCC with

the elements (CCC)ij = 〈T (φi), ψj〉Y provides a repre-sentation of T . In particular, by choosing the deltafunctions supported on the shape vertices as basisfunctions, one obtains a permutation ΠΠΠ as a matrixrepresentation for the functional map.

A more compact way to represent T in matrix formis obtained by taking the Laplacian eigenfunctionsφii≥1, ψii≥1 of the respective manifolds as thechoice for a basis. In case π is a (near) isometry, theequality ψi = ±φi π−1 holds (approximately) forall i ≥ 1, leading to the matrix representation CCC be-ing diagonally dominant, i.e., (CCC)ij = 〈T (φi), ψj〉Y ≈±δij .

With this choice, Ovsjanikov et al. [29] proposedto truncate the matrix CCC after the first k × k coeffi-cients as a low-pass approximation of the functionalmap (typical values for k are in the range 20− 300).This is especially convenient for correspondence prob-lems, where one is required to solve for k2. At thesame time, in analogy to classical Fourier analysis,the truncation has a blurring effect on the correspon-dence. As a result, recovering the original bijection πfrom the spectral coefficients CCC leads to a non-trivialinverse problem.

Assume shapes X and Y have n points each, andlet the matrices ΦΦΦ,ΨΨΨ ∈ Rn×k contain the first k neigenvectors of the respective Laplacians. For thesake of simplicity we assume ΦΦΦ and ΨΨΨ to be area-weighted, allowing us to consider the standard dotproduct in all equations. The expression for CCC ∈Rk×k can now be compactly written as

CCC = ΨΨΨTΠΠΠΦΦΦ . (2)

Note that the matrix CCC is now a rank-k approxima-tion of T . The pointwise map recovery problem [32],which is highly underdetermined, consists in findinga n× n permutation ΠΠΠ satisfying (2). The followingtechniques have been proposed for this purpose.

Linear assignment problem (LAP). If we as-sume k = n, the terms on either side of (2) have thesame rank and the relation can be straightforwardlyinverted to yield ΠΠΠ = ΨΨΨCCCΦΦΦT. Since in the truncatedsetting we have k n, the best possible solution inthe `2 sense can be obtained by looking for a permu-

3

tation ΠΠΠ minimizing −〈ΠΠΠ,ΨΨΨCCCΦΦΦT〉F . This leads tothe equivalent linear assignment problem:

minΠΠΠ∈0,1n×n

‖CCCΦΦΦT −ΨΨΨTΠΠΠ‖2F (3)

s.t. ΠΠΠT111 = 111 , ΠΠΠ111 = 111 . (4)

The equality between the two expressions comes fromthe observation that ‖CCCΦΦΦT −ΨΨΨTΠΠΠ‖2F = ‖CCCΦΦΦT‖2F +‖ΨΨΨT‖2F − 2〈ΨΨΨCCCΦΦΦT,ΠΠΠ〉 for permutation matrices ΠΠΠ.Minimizing with respect to ΠΠΠ, and recalling thatΨΨΨTΨΨΨ = III, yields the equivalence.

The problem above admits an intuitive interpreta-tion. Denoting by eeei the indicator vector having thevalue 1 in the ith position and 0 otherwise, we seethat each column of ΦΦΦT contains the spectral coef-ficients ΦΦΦTeeei of delta functions δxi : X → 0, 1 forxi ∈ X and i = 1, . . . , n. Hence, the image via T ofall indicator functions on X is given by the columnsof CCCΦΦΦT. Problem (3) seeks for a permutation ΠΠΠ min-imizing the distance between columns of CCCΦΦΦT andcolumns of ΨΨΨTΠΠΠ in a `2 sense.

Nearest neighbors. In [29] the authors proposedto recover a pointwise correspondence between X andY by solving the nearest-neighbor problem

minPPP∈0,1n×n

‖CCCΦΦΦT −ΨΨΨTPPP‖2F (5)

s.t. PPPT111 = 111 . (6)

This can be seen as a simplified version of the LAPwhere the bi-stochasticity constraints (4) are relaxed,including all (binary) column-stochastic matrices PPPin the feasible set. A global solution to (5) can beobtained in an efficient manner by solving for eachcolumn of PPP separately: It is sufficient to seek for thenearest column of CCCΦΦΦT with respect to each columnof ΨΨΨT.

An immediate consequence of this separable ap-proach is that its minimizers are not guaranteed tobe bijections. A balanced version of (5), obtainedby exchanging the roles of CCCΦΦΦT and ΨΨΨT in an alter-nating fashion was proposed in [32], with moderateincrease in accuracy.

Iterative closest point (ICP). In [29] it was ad-ditionally proposed to solve for the (not necessarilybijective) PPP according to the nearest-neighbor ap-proach (5), followed by a refinement of CCC via theorthogonal Procrustes problem:

minCCC∈Rk×k

‖CCCΦΦΦT −ΨΨΨTPPP‖2F (7)

s.t. CCCTCCC = III . (8)

The PPP - andCCC- steps are alternated until convergence.In analogy to classical Iterative Closest Point (ICP)refinement [13, 8] operating in R3, this can be seenas a rigid alignment between point sets (columns of)ΦΦΦT and ΨΨΨTPPP in Rk.

Coherent point drift (CPD). The orthogonalrefinement of (5), (7) assumes the underlying mapto be area-preserving [29], and is therefore bound tofail in case the two shapes are non-isometric. Rodolaet al. [32] proposed to consider the non-rigid coun-terpart, for a given CCC:

minPPP∈[0,1]n×n

DKL(CCCΦΦΦT,ΨΨΨTPPP ) + λ‖ΩΩΩ(CCCΦΦΦT −ΨΨΨTPPP )‖2

(9)

s.t. PPPT111 = 111 , (10)

where DKL denotes the Kullback-Leibler divergencebetween probability distributions, ΩΩΩ is a low-pass op-erator promoting smooth velocity vectors, and λ > 0controls the regularity of the assignment. Problem(9) can be seen as a Tikhonov regularization of thedisplacement field relating the two sets of spectral co-efficients, with a measure of proximity given by theKL divergence between the two. The problem is thensolved via expectation-maximization by the coherentpoint drift algorithm [28].

3 Bayesian map estimation

We describe a Bayesian formulation of bijective mapestimation that views the given correspondence as arealization of a random process adding noise to a la-tent ideal correspondence. We denote by π : X →Y the latent bijective correspondence between the

4

π−1(y)

X Y

y

π0(x1)

π0(x2)

x1

xx2

dX (x, x1)

Figure 2: Conceptual illustration of the proposedBayesian estimator. For a fixed point y, a Gaussianprobability measure on Y is pulled back to a measureon X by the given correspondence π0. The estimatex of the latent preimage π−1(y) of y is computed byminimizing the expectation of dpX (x, ·) with respectto that measure.

shapes. Let X denote a random point on X drawnfrom a uniform distribution, in the sense that for ev-ery measurable set A ⊂ X , P(X ∈ A) ∝ area(A).Given X = x, we denote by the conditional randomvariable Y |X = x a point on Y with a Gaussian dis-tribution with some variance σ2 centered at π(x) thataccounts for the uncertainty in the map. The Gaus-sian distribution is interpreted in the sense that forevery measurable set B ⊂ Y,

P(Y ∈ B|X = x) ∝∫B

exp

(−d2Y(y, π(x))

2σ2

)da(y).

Using Bayes’ theorem, we can express the proba-bility density of the conditional random variable X|Yas

fX|Y (x|y) =fY |X(y|x)fX(x)

fY (y)∝ fY |X(y|x)

∝ exp

(−d2Y(y, π(x))

2σ2

).

Given some (possibly noisy and not necessarily bi-jective) correspondence π0 : X → Y, we considery = π0(x) as a realization of Y |X = x for everyx ∈ X . Our goal is to estimate the bijection π or itsinverse π−1 from these data.

Let us fix some y ∈ Y. A Bayesian estimator of x =π−1(y) given the observations π0 can be expressed as

x(y) = arg minx

EX|Y=y dpX (X, x)

= arg minx

∫XdpX (x, x) exp

(−d2Y(y, π0(x))

2σ2

)da(x).

In the Euclidean case, the above Bayesian estima-tor coincides with the minimum mean absolute error(MMAE) for p = 1 and the minimum mean squarederror (MMSE) for p = 2; in both cases, it has a closed-form solution as the geometric median and the cen-troid, respectively. The more general case discussedhere can be thought of as the intrinsic counterpart ofthe median and the centroid.

We estimate the whole inverse map π−1 by mini-mizing

π−1 = arg minπ−1

∫X×Y

dpX (x, π−1(y))e−d2Y (y,π0(x))

2σ2 da(x)da(y).

(11)

5

over all bijections π−1 : Y → X . Note that due tothe additional constraint that π−1 has to be a bijec-tion, the estimation cannot be done for each pointy independently. We also observe that iterating theprocess several times consistently improves the esti-mated map accuracy.

Finally, we note that when π is area-preserving or,more generally, scales the metric uniformly, the esti-mator (11) can be equivalently rewritten in terms ofπ as

π = arg minπ:X→Y

∫X×X

dpX (x, ξ)e−d2Y (π(ξ),π0(x))

2σ2 da(x)da(ξ).(12)

It is worthwhile mentioning that while being natu-ral, the assumption of uniform prior distribution ofX on X (embodied in the use of the standard areameasure in the above integral) can be replaced byother measures emphasizing regions where errors areless tolerable. Also, non-Gaussian noise models maybe more suitable for data coming from a specific cor-respondence algorithm. We defer these interestingquestions to future study.

Discretization. We consider the discretization of(12). We assume the shape X to be discretized at npoints with the corresponding discrete area elementsai and pairwise geodesic distance matrix DDDX . Simi-larly, the shape Y is discretized as the same numberof points, and its pairwise distance matrix is denotedby DDDY .

The bijective correspondence is represented by then× n permutation matrix ΠΠΠ sought by minimizing

ΠΠΠ = arg minΠΠΠ

tr(ΠΠΠTPPPΓΓΓ), (13)

where PPP is an n× n matrix with the elements

(PPP)ij = exp

(−

(DDDY)2π0(i),j

2σ2

),

and ΓΓΓ is an n × n matrix with (ΓΓΓ)ij = (DDDX )pijaiaj .Note that (13) is a linear assignment problem (LAP).For directly solving the LAP with the specific struc-ture of the score matrix given by PPPΓΓΓ, we found theauction algorithm [7] to perform the best in prac-tice. Its average runtime complexity is O(n2 log n),

with O(n2) storage complexity if a full score matrixis used. On regular hardware, this translates to sev-eral seconds for n ∼ 2.5×103, which quickly grows to20 seconds for n = 4× 103 and almost 10 minutes forn = 12×103, taking tens of gigabytes of memory. Wetherefore conclude that directly solving the full LAPis practical for n . 104, and in the following sectionpropose a multi-scale scheme that can scale to muchlarger numbers of points.

Another computational bottleneck stems from thecomputation of pairwise geodesic distances. For ex-ample, using fast marching [21] the computation re-quires O(n2 log n) computations and O(n2) storage.While the computations can be thoroughly paral-lelized and executed on a GPU, reducing the com-plexity by orders of magnitude [42], the storage of afull distance matrix is still prohibitive for n & 104.However, since geodesic distance maps are almost ev-erywhere smooth with constant gradient, their ap-proximation in a truncated harmonic basic is optimalin the `2 sense [3]. Instead of storing an n × n ma-trix DDDX , we store the k×n representation coefficientmatrix

AAAX = ΦΦΦTDDDX ,

where ΦΦΦ contains the first k n eigenfunctions of theLaplace-Beltrami operator on X . In order to ”decom-press” the i-th row of DDDX used in the computationof the LAP score, the corresponding column of AAAX ismultiplied from the left by ΦΦΦ.

It is also worthwhile mentioning that while thegeodesic metric is a natural candidate to computeintrinsic distances on a manifold, the proposed esti-mator can work with other choices. For example, dif-fusion distances [14] or other approximations of thegeodesic distances [15] are likely to work equally wellwhile being better amenable both for faster compu-tation and more compact storage.

Multiscale solution. In order to reduce the com-putation and storage complexity associated with thedirect solution of the LAP for large values of n, weadopt a multi-scale strategy. Both shapes are dis-cretized in a hierarchical fashion using farthest pointsampling, while the distances and the harmonics arecalculated at the finest scale and sub-sampled.

6

First, a full LAP (13) is solved at a coarse scale.The produced correspondence is interpolated to thenext scale and is used as the input correspondence π0

to the LAP. While numerous interpolation techniquesexist, we found that simple nearest neighbour inter-polation produces satisfactory results. At the finerscale, the space of possible bijections π(i) is restrictedto the points falling into a fixed radius r around eachπ0(i) (note that r has to be larger than the coarsesampling radius). This is equivalent to assigning infi-nite score to the prohibited permutations. For a suf-ficiently small r, this strategy results in sparse scorematrices, with density significantly lower than 1%.

4 Experiments

We start by evaluating the influence of the parame-ters p ∈ 1, 2 and σ on the quality of the Bayesianestimator (11). We initialize with noisy correspon-dences coming from a nearest neighbour (5) resultand evaluate on two datasets with different globalscales, see Figure 3. The optimal choice of σ2

amounts to approximately 6% of the target shapesarea for both choices of p but the quality is shown tobe stable in a vicinity.

We test our method recover bijections from on twotypes of initialization, namely functional maps of lowrank and sparse correspondences.

We conduct quantitative experiments on theFAUST dataset [9] (7K vertices) and on downsampledversions of the SCAPE [5] and KIDS [35] datasets(1K vertices). As quantitative quality criteria weevaluate the geodesic errors, the run times and thelack of surjectivity of the different methods. Wefurther show that our approach can directly tackleshapes having more then 10K vertices.

4.1 Recovery from a functional map

In this set of experiments the low rank approximationis given in terms of a functional map of different ranksin the harmonic basis. Comparisons are done againstnearest neighbors (NN) (5), bijective NN (3), ICP (7)and CPD (9).

0 2 4 6 8 10 12 14 16 18 200.005

0.01

0.015

0.02

0.025

0.03

0.035

0.04

σ2 in % of area

Ave

rage

geo

desi

c er

ror

Scape, p=1Scape, p=2Kids, p=1Kids, p=2

Figure 3: Dependency on parameters: We evaluatethe influence of the parameters p and σ on the qualityof the denoised matching. We initialize with a noisymatching coming from a nearest neighbor search indescriptor space. The evaluation is done on twodatasets with different global scales, namely KIDSand SCAPE.

Approximation of the groundtruth. Here weconstruct the low rank functional map using theknown groundtruth correspondences between theshapes. This is supposed to be the ideal input for allthe competing methods. As the input to our methodwe use the matchings found by nearest neighbors andits bijective version. We show quantitative compar-isons on 71 pairs from the SCAPE dataset (near iso-metric, 1K vertices) and 100 pairs from the FAUSTdataset (including inter-class pairs, 7K vertices). InFigures 4 and 5 we compare the accuracy, in Figure 6the lack of surjectivity is analyzed. We only show theperformance of a single application of the Bayesianestimator yet adumbrate experiments with multipleiterations in the following sections. Even after oneiteration, our method outperforms the state of theart method (9) as well in accuracy as in run time(Table 1). Even on shapes having more then 10Kvertices just one iteration of the Bayesian estimatorgives very good results, as can be seen in Figure 7.Memory consumption and run times however limitthe direct applicability of the single-scale Bayesianestimator.

7

20

30

40

50

60

70

80

90

100

% c

orre

spon

denc

es

20 harmonics

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 530

40

50

60

70

80

90

100

Geodesic error (in % of the diameter)

% c

orre

spon

denc

es

50 harmonics

NNBijective NNICPCPDNN+BayesianBij. NN+Bayesian

Figure 4: Recovering the groundtruth matching froma functional map with rank deficiency. We matched70 pairs from the near-isometric SCAPE dataset(1K). Plotted are the histograms of geodesic errors(solid line: mean; 90% of the matched pairs produceresults between the dotted lines). All methods boosttheir quality with increasing numbers of eigenfunc-tions. Denoising the results of nearest neighbors (yel-low) outperforms the state of the art method (green)while having only a fraction of its runtime (Table 1).Even better results are achieved when initializing theBayesian estimator with the result of bijective NN(orange).

% c

orre

spon

denc

es

20 harmonics

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5Geodesic error (in % of the diameter)

% c

orre

spon

denc

es

50 harmonics

NNBijective NNICPCPDNN+BayesianBij. NN+Bayesian

0

20

40

60

80

100

0

20

40

60

80

100

Figure 5: Evaluations from Figure 4 repeated on thehigher resolution FAUST shapes. A single iterationof our denoising algorithm boosts the performance ofthe simple nearest neighbor approach above the stateof the art (green). Notice that in this scenario bijec-tive NN is giving better results than CPD; denoisingthis matching leads to 80% exact matches averagedover all pairs.

8

0 0.02 0.04 0.06 0.080

20

40

60

Surjectivity radius

% c

orre

spon

denc

es

NN ICPCPD

0 0.02 0.04 0.06 0.08 0.1Surjectivity radius

SCAPE Faust

Figure 6: Histogram of distances on the targetshape to the image of the source shape calculated onSCAPE (left) and FAUST (right) for matching with20 (solid) and 50 (dotted) harmonics. In particular,the intersection with the y-axis tells the percentage ofpoints on the target that lack a preimage. For LAP-based approaches imposing bijectivity the image ofthe source covers the target.

Geodesic error (in % of the diameter)

% c

orre

spon

denc

es

NNBijective NNICPCPDNN+BayesianBij. NN+Bayesian

0 1 2 3 4 5 6 7 8 9 100

10

20

30

40

50

60

70

80

90

10020 harmonics

Figure 7: Our method can be applied to recover cor-respondences between high resolution shapes. Al-though the percentage of exact hits decreases withthe increase of the sampling density, the geodesic er-rors are compelling. Due to the significant increasein runtime particularly of CPD this experiment wasonly done on a subset of the pairs from the SCAPE12.5K benchmark.

% c

orre

spon

denc

es

20 harmonics

Geodesic error (in % of the diameter)

% c

orre

spon

denc

es

50 harmonics

0

20

40

60

80

100

0

20

40

60

80

100

NNBijective NNICPCPDNN+BayesianNN+2×BayesianBij. NN+BayesianBij. NN+2×Bayesian

0 1 2 3 4 5 6 7 8 9 10

Figure 8: Evaluation on realistic input data. We testdifferent recovery methods on a low rank functionalmap coming from an optimization process. Again ourmethod gives the best results. Iterating the Bayesianestimator improves the performance even more.

Using a functional map coming from an opti-mization process. We follow the approach from[31] to construct a realistic functional map match-ing, which typically requires region features as input.These features were detected using the consensus-segmentation method proposed in [34], and the re-sulting regions were matched by intersection w.r.t.the ground-truth. Both of these two methods wereexecuted with the same parameters as in their pub-licly available implementation. In Figure 8 this ini-tialization is evaluated on the SCAPE dataset.

4.2 Recovery from a sparse correspon-dence

In this set of experiments the low rank approxima-tion is given in terms of sparse correspondences be-

9

n 1000 1000 6890 6890k 20 50 20 50

Nearest neighbors 0.04 0.06 1.35 2.88Bijective NN 2.79 2.30 463.66 253.03

ICP 0.14 0.24 12.72 30.08CPD 4.79 4.67 1745.06 2085.65

NN + Bayesian 1.75 1.28 382.86 244.10Bij. NN + Bayesian 4.06 3.44 746.00 440.94

Table 1: Average runtimes in seconds. We comparethe runtimes of different recovery methods. Giventhe rank k of a functional map approximating thecorrespondence between shapes sampled at n pointseach, we report the time it takes to obtain a densematching. See Figures 4-8 for evaluations of accu-racy. Notice that while linear assignment problemsare known to be time demanding to solve for largernumbers of variables, the most dramatic increase ofrun time occurs when applying CPD.

tween high resolution shapes. This type of input canfor instance be obtained by minimizing energies un-der l1-constraints, such as [33], or appears in mul-tiresolution settings. We make use of groundtruthcorrespondences of a few points and interpolate thematching with the technique described in the captionof Figure 9. Figure 10 illustrates how iterations of theBayesian estimator improve the matching.

5 Conclusion

We considered the problem of bijective correspon-dence recovery by means of denoising a given set ofmatches coming from any of the existing algorithms(including those not guaranteeing bijection, or pro-ducing sparse correspondences). Viewing the denos-ing as a Bayesian estimation problem, we formulatedthe intrinsic equivalent of the mean and median filtersfrequently employed in signal processing, with the ad-ditional constraint of bijectivity embodied through anLAP.

We find surprising the fact that such a simple ideademonstrates a consistent improvment in the corre-spondence quality in all experiments we have con-ducted. We believe that tools from estimation theory

0

10

20

30

40

50

60

70

80

90

100

% c

orre

spon

denc

es

Kids 1K, 20 matches

Init 1 iter 2 iters 3 iters 4 iters 5 iters

0

10

20

30

40

50

60

70

80

90

100%

cor

resp

onde

nces

Faust 7K, 20 matches

0 1 2 3 4 5 6 7 8 9 100

10

20

30

40

50

60

70

80

90

100

Geodesic error (in % of the diameter)

% c

orre

spon

denc

es

Faust 7K, 50 matches

Figure 9: Correspondence accuracy on the KIDS 1K(top plot) and FAUST 7K (middle and bottom plots).Dense matches were produced by nearest-neighborinterpolation from 20 (top and middle) and 50 (bot-tom) sparse matches and used as the initialization ofthe Bayesian estimator iterated up to five times.

10

Figure 10: Obtaining a dense and bijective matching of high quality by iterative application of the Bayesianestimator on a sparse correspondence. The sparse correspondence is interpolated by assigning each pointthe on the source shape the same image as its nearest neighbor from the sparse set (second column). Thisinduces large geodesic errors for points being far away from the sparse set (bottom row, left) and is far frombeing surjective (second row from the bottom, left). Each iteration of the Bayesian estimator (five rightmostcolumns) yields better results, as illustrated by the magnified fragments.

11

that have been heavily used in other domains of sci-ence and engineering might be very useful in shapeanalysis, and invite the community to further explorethis direction.

Of special interest are the choice of the loss func-tion in the posterior expectation (which in this pa-per was restricted to the absolute and squared dis-tance), the prior distribution of X (which we assumeduniform), and the noise distribution (which was as-sumed Gaussian). Alternative estimators making useof Bayesian statistics, such as maximum a posteriori(MAP) estimators, should also be explored.

References

[1] Y. Aflalo, A. Dubrovina, and R. Kimmel. Spec-tral generalized multidimensional scaling. IJCV,2016.

[2] Yonathan Aflalo, Alexander Bronstein, and RonKimmel. On convex relaxation of graph isomor-phism. PNAS, 112(10):2942–2947, 2015.

[3] Yonathan Aflalo and Ron Kimmel. Spectral mul-tidimensional scaling. PNAS, 110(45):18052–18057, 2013.

[4] Noam Aigerman, Roi Poranne, and Yaron Lip-man. Lifted bijections for low distortion sur-face mappings. ACM Transactions on Graphics(TOG), 33(4):69, 2014.

[5] Dragomir Anguelov et al. SCAPE: Shapecompletion and animation of people. TOG,24(3):408–416, 2005.

[6] M. Aubry, U. Schlickewei, and D. Cremers. Thewave kernel signature: A quantum mechanicalapproach to shape analysis. In Proc. ICCV,2011.

[7] Dimitri P Bertsekas. Network optimization: con-tinuous and discrete models. Citeseer, 1998.

[8] P. J. Besl and N. D. McKay. A method for reg-istration of 3D shapes. PAMI, 14(2):239–256,1992.

[9] Federica Bogo, Javier Romero, Matthew Loper,and Michael J. Black. FAUST: Dataset and eval-uation for 3D mesh registration. In Proc. CVPR,2014.

[10] A. M. Bronstein, M. M. Bronstein, and R. Kim-mel. Generalized multidimensional scaling: aframework for isometry-invariant partial surfacematching. PNAS, 103(5):1168–1172, 2006.

[11] Michael M. Bronstein and Iasonas Kokkinos.Scale-invariant heat kernel signatures for non-rigid shape recognition. In Proc. CVPR, 2010.

[12] Q. Chen and V. Koltun. Robust nonrigid regis-tration by convex optimization. In Proc. ICCV,2015.

[13] Y. Chen and G. Medioni. Object modeling byregistration of multiple range images. In Proc.Conf. Robotics and Automation, 1991.

[14] R. R. Coifman, S. Lafon, A. B. Lee, M. Mag-gioni, B. Nadler, F. Warner, and S. W. Zucker.Geometric diffusions as a tool for harmonic anal-ysis and structure definition of data: Diffusionmaps. PNAS, 102(21):7426–7431, 2005.

[15] Keenan Crane, Clarisse Weischedel, and MaxWardetzky. Geodesics in heat: A new approachto computing distance based on heat flow. ACMTrans. on Graphics, 32(5):152, 2013.

[16] A. Elad and R. Kimmel. Bending invariant rep-resentations for surfaces. In Proc. CVPR, 2001.

[17] O.V. Kaick, H. Zhang, G. Hamarneh, andD. Cohen-Or. A survey on shape correspon-dence. Computer Graphics Forum, 20:1–23,2010.

[18] I. Kezurer, S. Kovalsky, R. Basri, and Y. Lip-man. Tight relaxations of quadratic matching.Computer Graphics Forum, 34(5), 2015.

[19] Vladimir G. Kim, Yaron Lipman, Xiaobai Chen,and Thomas A. Funkhouser. Mobius trans-formations for global intrinsic symmetry anal-ysis. Computer Graphics Forum, 29(5):1689–1700, 2010.

12

[20] Vladimir G Kim, Yaron Lipman, and ThomasFunkhouser. Blended intrinsic maps. TOG,30(4):79, 2011.

[21] Ron Kimmel and James A Sethian. Com-puting geodesic paths on manifolds. PNAS,95(15):8431–8435, 1998.

[22] A. Kovnatsky, M. M. Bronstein, X. Bresson, andP. Vandergheynst. Functional correspondence bymatrix completion. In Proc. CVPR, 2015.

[23] Marius Leordeanu and Martial Hebert. A spec-tral technique for correspondence problems us-ing pairwise constraints. In Proc. ICCV, 2005.

[24] Y. Lipman and I. Daubechies. ConformalWasserstein distances: Comparing surfaces inpolynomial time. Advances in Mathematics,227(3):1047 – 1077, 2011.

[25] Diana Mateus, Radu P. Horaud, David Knos-sow, Fabio Cuzzolin, and Edmond Boyer. Ar-ticulated shape matching using laplacian eigen-functions and unsupervised point registration.In Proc. CVPR, 2008.

[26] Facundo Memoli. Gromov-Wasserstein Dis-tances and the Metric Approach to ObjectMatching. Foundations of Computational Math-ematics, pages 1–71, 2011.

[27] Facundo Memoli and Guillermo Sapiro. Atheoretical and computational framework forisometry invariant recognition of point clouddata. Foundations of Computational Mathemat-ics, 5(3):313–347, 2005.

[28] A. Myronenko and X. Song. Point set reg-istration: Coherent point drift. TPAMI,32(12):2262–2275, 2010.

[29] Maks Ovsjanikov, Mirela Ben-Chen, JustinSolomon, Adrian Butscher, and LeonidasGuibas. Functional maps: a flexible represen-tation of maps between shapes. ACM Trans. onGraphics, 31(4):30, 2012.

[30] Maks Ovsjanikov, Quentin Merigot, FacundoMemoli, and Leonidas Guibas. One point iso-metric matching with the heat kernel. ComputerGraphics Forum, 29(5):1555–1564, 2010.

[31] Jonathan Pokrass, Alexander M Bronstein,Michael M Bronstein, Pablo Sprechmann, andGuillermo Sapiro. Sparse modeling of intrinsiccorrespondences. In Computer Graphics Forum,volume 32, pages 459–468. Wiley Online Library,2013.

[32] E. Rodola, M. Moeller, and D. Cremers. Point-wise map recovery and refinement from func-tional correspondence. In Proceedings Vision,Modeling and Visualization (VMV), Aachen,Germany, 2015.

[33] Emanuele Rodola, Alex M Bronstein, An-drea Albarelli, Filippo Bergamasco, and An-drea Torsello. A game-theoretic approach to de-formable shape matching. In Proc. CVPR, 2012.

[34] Emanuele Rodola, Samuel Rota Bulo, andDaniel Cremers. Robust region detection viaconsensus segmentation of deformable shapes.In Computer Graphics Forum, volume 33, pages97–106. Wiley Online Library, 2014.

[35] Emanuele Rodola, Samuel Rota Bulo, ThomasWindheuser, Matthias Vestner, and Daniel Cre-mers. Dense non-rigid shape correspondence us-ing random forests. In Proc. CVPR, 2014.

[36] Y. Sahillioglu and Y. Yemez. Coarse-to-fine com-binatorial matching for dense isometric shapecorrespondence. Computer Graphics Forum,30(5):1461–1470, 2011.

[37] Alon Shtern and Ron Kimmel. Matching lboeigenspace of non-rigid shapes via high orderstatistics. arXiv:1310.4459, 2013.

[38] Justin Solomon, Andy Nguyen, AdrianButscher, Mirela Ben-Chen, and LeonidasGuibas. Soft maps between surfaces. InComputer Graphics Forum, volume 31, pages1617–1626, 2012.

13

[39] A. Tevs et al. Intrinsic shape matching byplanned landmark sampling. Computer Graph-ics Forum, 30(2):543–552, 2011.

[40] Lorenzo Torresani, Vladimir Kolmogorov, andCarsten Rother. Feature correspondence viagraph matching: Models and global optimiza-tion. In Proc. ECCV, 2008.

[41] Shinji Umeyama. An eigendecomposition ap-proach to weighted graph matching problems.PAMI, 10(5):695–703, 1988.

[42] Ofir Weber, Yohai S Devir, Alexander M Bron-stein, Michael M Bronstein, and Ron Kimmel.Parallel algorithms for approximation of dis-tance maps on parametric surfaces. ACM Trans.on Graphics, 27(4):104, 2008.

[43] T. Windheuser, U. Schlickewei, F. R. Schmidt,and D. Cremers. Geometrically consistent elasticmatching of 3d shapes: A linear programmingsolution. In IEEE International Conference onComputer Vision (ICCV), 2011.

[44] A. Zaharescu, E. Boyer, K. Varanasi, and R. Ho-raud. Surface feature detection and descriptionwith applications to mesh matching. In Proc.CVPR, 2009.

[45] Y. Zeng et al. Dense non-rigid surface registra-tion using high-order graph matching. In Proc.CVPR, 2010.

14