1 michael bronstein shape google: geometric words and expressions for invariant shape retrieval...
Post on 19-Dec-2015
223 views
TRANSCRIPT
![Page 1: 1 Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval Shape Optimal invariant metrics for shape retrieval Michael](https://reader030.vdocuments.mx/reader030/viewer/2022033104/56649d2e5503460f94a051d2/html5/thumbnails/1.jpg)
1Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval
Shape
Optimal invariant metrics for shape retrieval
Michael Bronstein
Department of Computer ScienceTechnion – Israel Institute of Technology
![Page 2: 1 Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval Shape Optimal invariant metrics for shape retrieval Michael](https://reader030.vdocuments.mx/reader030/viewer/2022033104/56649d2e5503460f94a051d2/html5/thumbnails/2.jpg)
2Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval
![Page 3: 1 Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval Shape Optimal invariant metrics for shape retrieval Michael](https://reader030.vdocuments.mx/reader030/viewer/2022033104/56649d2e5503460f94a051d2/html5/thumbnails/3.jpg)
3Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval
Tagged shapes
Shapes withoutmetadata
Man, person, humanPersonText search
Content-based search
3D warehouse
![Page 4: 1 Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval Shape Optimal invariant metrics for shape retrieval Michael](https://reader030.vdocuments.mx/reader030/viewer/2022033104/56649d2e5503460f94a051d2/html5/thumbnails/4.jpg)
4Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval
Outline
Feature descriptor
Geometric words
Bag of words?
![Page 5: 1 Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval Shape Optimal invariant metrics for shape retrieval Michael](https://reader030.vdocuments.mx/reader030/viewer/2022033104/56649d2e5503460f94a051d2/html5/thumbnails/5.jpg)
5Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval
Invariance
Local geodesic distance histogram
Rigid Inelastic Topology
Gaussian curvature
Scale
Wang, B 2010
Heat kernel signature (HKS)
Scale-invariant HKS (SI-HKS)
![Page 6: 1 Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval Shape Optimal invariant metrics for shape retrieval Michael](https://reader030.vdocuments.mx/reader030/viewer/2022033104/56649d2e5503460f94a051d2/html5/thumbnails/6.jpg)
6Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval
Heat kernels
Heat kernel is a fundamental solution of the heat equation with
point heat source at (heat value at point after time )
Initial conditions: heat distribution at time
Solution : heat distribution at time
Heat equation governs heat propagation
on a surface
![Page 7: 1 Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval Shape Optimal invariant metrics for shape retrieval Michael](https://reader030.vdocuments.mx/reader030/viewer/2022033104/56649d2e5503460f94a051d2/html5/thumbnails/7.jpg)
7Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval
Heat kernel signature
Sun, Ovsjanikov & Guibas SGP 2009
Multiscale local shape descriptor
can be interpreted as probability of Brownian motion to return to
the same point after time (represents “stability” of the point)
Time (scale)
![Page 8: 1 Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval Shape Optimal invariant metrics for shape retrieval Michael](https://reader030.vdocuments.mx/reader030/viewer/2022033104/56649d2e5503460f94a051d2/html5/thumbnails/8.jpg)
8Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval
Heat kernel signature
Sun, Ovsjanikov, Guibas SGP 2009Ovsjanikov, BB & Guibas NORDIA 2009
Heat kernel signatures represented in RGB space
![Page 9: 1 Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval Shape Optimal invariant metrics for shape retrieval Michael](https://reader030.vdocuments.mx/reader030/viewer/2022033104/56649d2e5503460f94a051d2/html5/thumbnails/9.jpg)
9Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval
Scale invariance
B, Kokkinos CVPR 2010
Original shape Scaled by
HKS= HKS=
Not scale invariant!
![Page 10: 1 Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval Shape Optimal invariant metrics for shape retrieval Michael](https://reader030.vdocuments.mx/reader030/viewer/2022033104/56649d2e5503460f94a051d2/html5/thumbnails/10.jpg)
10Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval
Scale-invariant heat kernel signature
B, Kokkinos CVPR 2010
Log scale-space
Scaling = shift and multiplicative
constant in HKS
log + d/d
Undo scaling
Fourier transformmagnitude
Undo shift
0 100 200 300-15
-10
-5
0
t0 100 200 300
-0.04
-0.03
-0.02
-0.01
0
t0 2 4 6 8 10 12 14 16 18 20
0
1
2
3
4
=2k/T
![Page 11: 1 Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval Shape Optimal invariant metrics for shape retrieval Michael](https://reader030.vdocuments.mx/reader030/viewer/2022033104/56649d2e5503460f94a051d2/html5/thumbnails/11.jpg)
11Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval
Scale invariance
B, Kokkinos CVPR 2010
Heat Kernel Signature Scale-invariantHeat Kernel Signature
![Page 12: 1 Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval Shape Optimal invariant metrics for shape retrieval Michael](https://reader030.vdocuments.mx/reader030/viewer/2022033104/56649d2e5503460f94a051d2/html5/thumbnails/12.jpg)
12Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval
Scale invariance
B, Kokkinos CVPR 2010
Heat Kernel Signature Scale-invariantHeat Kernel Signature
![Page 13: 1 Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval Shape Optimal invariant metrics for shape retrieval Michael](https://reader030.vdocuments.mx/reader030/viewer/2022033104/56649d2e5503460f94a051d2/html5/thumbnails/13.jpg)
13Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval
Modeling vs learning
Wang, B 2010
![Page 14: 1 Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval Shape Optimal invariant metrics for shape retrieval Michael](https://reader030.vdocuments.mx/reader030/viewer/2022033104/56649d2e5503460f94a051d2/html5/thumbnails/14.jpg)
14Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval
Learning invariance
Positives P Negatives N
T
![Page 15: 1 Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval Shape Optimal invariant metrics for shape retrieval Michael](https://reader030.vdocuments.mx/reader030/viewer/2022033104/56649d2e5503460f94a051d2/html5/thumbnails/15.jpg)
15Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval
Similarity learning
with high probability
false positive
false negative
positive
negative
![Page 16: 1 Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval Shape Optimal invariant metrics for shape retrieval Michael](https://reader030.vdocuments.mx/reader030/viewer/2022033104/56649d2e5503460f94a051d2/html5/thumbnails/16.jpg)
16Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval
Similarity-preserving hashing
-1 -1 +1 -1-1 -1 -1 +1-1 -1 +1 +1
+1 +1 -1 +1+1 +1 +1 -1+1 +1 +1 +1
with high probability
with low probability
Collision:
= # of distinct bits
Gionis, Indik, Motwani 1999Shakhnarovich 2005
![Page 17: 1 Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval Shape Optimal invariant metrics for shape retrieval Michael](https://reader030.vdocuments.mx/reader030/viewer/2022033104/56649d2e5503460f94a051d2/html5/thumbnails/17.jpg)
17Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval
Boosting
Construct 1D embedding
Similarity is approximated by
Downweight pairs with
Upweight pairs with
BBK 2010; BB Ovsjanikov, Guibas 2010Shakhnarovich 2005
-1
+1
![Page 18: 1 Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval Shape Optimal invariant metrics for shape retrieval Michael](https://reader030.vdocuments.mx/reader030/viewer/2022033104/56649d2e5503460f94a051d2/html5/thumbnails/18.jpg)
18Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval
Boosting
BBK 2010; BB Ovsjanikov, Guibas 2010Shakhnarovich 2005
Construct 1D embedding
Similarity is approximated by
Downweight pairs with
Upweight pairs with
-1
+1
-1 -1
+1 -1
+1 +1
-1 +1
![Page 19: 1 Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval Shape Optimal invariant metrics for shape retrieval Michael](https://reader030.vdocuments.mx/reader030/viewer/2022033104/56649d2e5503460f94a051d2/html5/thumbnails/19.jpg)
19Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval
SHREC 2010 dataset
![Page 20: 1 Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval Shape Optimal invariant metrics for shape retrieval Michael](https://reader030.vdocuments.mx/reader030/viewer/2022033104/56649d2e5503460f94a051d2/html5/thumbnails/20.jpg)
20Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval
SHREC 2010 datasetBB et al, 3DOR 2010
Total dataset size: 1K shapes (715 queries)
Positives: 10K
Negatives: 100K
![Page 21: 1 Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval Shape Optimal invariant metrics for shape retrieval Michael](https://reader030.vdocuments.mx/reader030/viewer/2022033104/56649d2e5503460f94a051d2/html5/thumbnails/21.jpg)
21Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval
ShapeGoogle with HKS descriptorBB et al, 3DOR 2010
![Page 22: 1 Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval Shape Optimal invariant metrics for shape retrieval Michael](https://reader030.vdocuments.mx/reader030/viewer/2022033104/56649d2e5503460f94a051d2/html5/thumbnails/22.jpg)
22Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval
ShapeGoogle with SI-HKS descriptorBB et al, 3DOR 2010
![Page 23: 1 Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval Shape Optimal invariant metrics for shape retrieval Michael](https://reader030.vdocuments.mx/reader030/viewer/2022033104/56649d2e5503460f94a051d2/html5/thumbnails/23.jpg)
23Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval
Similarity sensitive hashing (96 bit)BB et al, 3DOR 2010
![Page 24: 1 Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval Shape Optimal invariant metrics for shape retrieval Michael](https://reader030.vdocuments.mx/reader030/viewer/2022033104/56649d2e5503460f94a051d2/html5/thumbnails/24.jpg)
24Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval
WaldHash
B2, Ovsjanikov, Guibas 2010
Construct embedding by maximizing
Remove pairs with
Downweight pairs with
Upweight pairs with
Early decision
and sample in new pairs into the training set
positive
negative
![Page 25: 1 Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval Shape Optimal invariant metrics for shape retrieval Michael](https://reader030.vdocuments.mx/reader030/viewer/2022033104/56649d2e5503460f94a051d2/html5/thumbnails/25.jpg)
25Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval
30%
B2, Ovsjanikov, Guibas 2010
![Page 26: 1 Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval Shape Optimal invariant metrics for shape retrieval Michael](https://reader030.vdocuments.mx/reader030/viewer/2022033104/56649d2e5503460f94a051d2/html5/thumbnails/26.jpg)
26Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval
Incommensurablespaces!
BB, Michel, Paragios CVPR 2010
Cross-modal similarity
Modality 1 Modality 2
How to compare apples to oranges?
Triangular meshes Point clouds
![Page 27: 1 Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval Shape Optimal invariant metrics for shape retrieval Michael](https://reader030.vdocuments.mx/reader030/viewer/2022033104/56649d2e5503460f94a051d2/html5/thumbnails/27.jpg)
27Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval
Modality 1 Modality 2
with high probability
Cross-modality embedding
BB, Michel, Paragios CVPR 2010
![Page 28: 1 Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval Shape Optimal invariant metrics for shape retrieval Michael](https://reader030.vdocuments.mx/reader030/viewer/2022033104/56649d2e5503460f94a051d2/html5/thumbnails/28.jpg)
28Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval
Modality 1 Modality 2
Cross-modality hashing
-1 -1 +1 -1-1 -1 -1 +1-1 -1 +1 +1
+1 +1 -1 +1+1 +1 +1 -1+1 -1 -1 +1
with high probability
with low probability
Collision:
BB, Michel, Paragios CVPR 2010
![Page 29: 1 Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval Shape Optimal invariant metrics for shape retrieval Michael](https://reader030.vdocuments.mx/reader030/viewer/2022033104/56649d2e5503460f94a051d2/html5/thumbnails/29.jpg)
29Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval
Query
8x8 dimensionalbag of expressions
Database
32-dimensionalbag of words
Cross-representation 3D shape retrieval
1052 shapes
BB, Michel, Paragios CVPR 2010
![Page 30: 1 Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval Shape Optimal invariant metrics for shape retrieval Michael](https://reader030.vdocuments.mx/reader030/viewer/2022033104/56649d2e5503460f94a051d2/html5/thumbnails/30.jpg)
30Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval
Me
an
av
erag
e p
rec
isio
n
Number of bits
Retrieval performance
BB, Michel, Paragios CVPR 2010