deep learning for robust normal estimation in unstructured ... · 20 cnn for normal estimation...
TRANSCRIPT
![Page 1: Deep Learning for Robust Normal Estimation in Unstructured ... · 20 CNN for normal estimation Training Requires annotated data for training Using big mesh: - not satisfactory - not](https://reader033.vdocuments.mx/reader033/viewer/2022050323/5f7d3f01ab8e2e3a483fd3b8/html5/thumbnails/1.jpg)
Deep Learning for Robust Normal Estimationin Unstructured Point Clouds
Renaud MarletAlexandre Boulch
![Page 2: Deep Learning for Robust Normal Estimation in Unstructured ... · 20 CNN for normal estimation Training Requires annotated data for training Using big mesh: - not satisfactory - not](https://reader033.vdocuments.mx/reader033/viewer/2022050323/5f7d3f01ab8e2e3a483fd3b8/html5/thumbnails/2.jpg)
2
Normal estimation in point clouds
Normal: 3D normalized vector
At each point: local orientation of the surface
![Page 3: Deep Learning for Robust Normal Estimation in Unstructured ... · 20 CNN for normal estimation Training Requires annotated data for training Using big mesh: - not satisfactory - not](https://reader033.vdocuments.mx/reader033/viewer/2022050323/5f7d3f01ab8e2e3a483fd3b8/html5/thumbnails/3.jpg)
3
Normal estimation in point clouds
Surface reconstruction[BDLGM14]
Primitive extraction[SWK07]
Rendering[ABCO*03]
Full references in the paper
![Page 4: Deep Learning for Robust Normal Estimation in Unstructured ... · 20 CNN for normal estimation Training Requires annotated data for training Using big mesh: - not satisfactory - not](https://reader033.vdocuments.mx/reader033/viewer/2022050323/5f7d3f01ab8e2e3a483fd3b8/html5/thumbnails/4.jpg)
4
Normal estimation in point clouds
Noise and outliersSharp features
Density variations Computation time
Main issues
![Page 5: Deep Learning for Robust Normal Estimation in Unstructured ... · 20 CNN for normal estimation Training Requires annotated data for training Using big mesh: - not satisfactory - not](https://reader033.vdocuments.mx/reader033/viewer/2022050323/5f7d3f01ab8e2e3a483fd3b8/html5/thumbnails/5.jpg)
5
Normal estimation in point clouds
Normal estimation need neighborhood information
![Page 6: Deep Learning for Robust Normal Estimation in Unstructured ... · 20 CNN for normal estimation Training Requires annotated data for training Using big mesh: - not satisfactory - not](https://reader033.vdocuments.mx/reader033/viewer/2022050323/5f7d3f01ab8e2e3a483fd3b8/html5/thumbnails/6.jpg)
7
Existing methods
Sample consensus
RegressionVoronoï
[HDD*92] HOPPE et al[CP05] CAZALS and POUGET [DG04] DEY and OSWAMI
[LSK*10] LI et al.
Other references in the paper
Hough Space
Hough Transform
[BM12] BOULCH and MARLET
![Page 7: Deep Learning for Robust Normal Estimation in Unstructured ... · 20 CNN for normal estimation Training Requires annotated data for training Using big mesh: - not satisfactory - not](https://reader033.vdocuments.mx/reader033/viewer/2022050323/5f7d3f01ab8e2e3a483fd3b8/html5/thumbnails/7.jpg)
8
Existing methods
Hough SpaceSample consensus
RegressionVoronoï
Hough Transform
[HDD*92] HOPPE et al[CP05] CAZALS and POUGET [DG04] DEY and OSWAMI
[LSK*10] LI et al. [BM12] BOULCH and MARLET
Other references in the paper
![Page 8: Deep Learning for Robust Normal Estimation in Unstructured ... · 20 CNN for normal estimation Training Requires annotated data for training Using big mesh: - not satisfactory - not](https://reader033.vdocuments.mx/reader033/viewer/2022050323/5f7d3f01ab8e2e3a483fd3b8/html5/thumbnails/8.jpg)
9
Principles of our method
CNNFilled accumulator
Selected normal 2 coordinates
Point
Point with normal
Hough transform
![Page 9: Deep Learning for Robust Normal Estimation in Unstructured ... · 20 CNN for normal estimation Training Requires annotated data for training Using big mesh: - not satisfactory - not](https://reader033.vdocuments.mx/reader033/viewer/2022050323/5f7d3f01ab8e2e3a483fd3b8/html5/thumbnails/9.jpg)
10
Robust Randomized Hough Transform
Point and neighborhood
Pick three points
Estimate plane normal
Vote in the accumulator
Iterate
Statistical bounds on number of hypotheses to pick in the paper
Principle
![Page 10: Deep Learning for Robust Normal Estimation in Unstructured ... · 20 CNN for normal estimation Training Requires annotated data for training Using big mesh: - not satisfactory - not](https://reader033.vdocuments.mx/reader033/viewer/2022050323/5f7d3f01ab8e2e3a483fd3b8/html5/thumbnails/10.jpg)
11
Robust Randomized Hough Transform
Accumulator design
Discretized half sphere[BM12]
Need for multiple evaluations
Grid accumulator
Better resolutionSuited for CNN
![Page 11: Deep Learning for Robust Normal Estimation in Unstructured ... · 20 CNN for normal estimation Training Requires annotated data for training Using big mesh: - not satisfactory - not](https://reader033.vdocuments.mx/reader033/viewer/2022050323/5f7d3f01ab8e2e3a483fd3b8/html5/thumbnails/11.jpg)
12
Robust Randomized Hough Transform
Accumulator design
PCANormal hypotheses 2D projection in Hough space
PCA
3D rotation (local 3D coordinates)
2D rotation
Accumulation
![Page 12: Deep Learning for Robust Normal Estimation in Unstructured ... · 20 CNN for normal estimation Training Requires annotated data for training Using big mesh: - not satisfactory - not](https://reader033.vdocuments.mx/reader033/viewer/2022050323/5f7d3f01ab8e2e3a483fd3b8/html5/thumbnails/12.jpg)
13
Estimation from accumulator
Estimator
Filled accumulator
Normal coordinates in Hough Space
(x, y)
![Page 13: Deep Learning for Robust Normal Estimation in Unstructured ... · 20 CNN for normal estimation Training Requires annotated data for training Using big mesh: - not satisfactory - not](https://reader033.vdocuments.mx/reader033/viewer/2022050323/5f7d3f01ab8e2e3a483fd3b8/html5/thumbnails/13.jpg)
14
Estimation from accumulator
Estimator
Filled accumulator
Normal coordinates in Hough Space
[BM12] Maximum of the
accumulator
(x, y)
![Page 14: Deep Learning for Robust Normal Estimation in Unstructured ... · 20 CNN for normal estimation Training Requires annotated data for training Using big mesh: - not satisfactory - not](https://reader033.vdocuments.mx/reader033/viewer/2022050323/5f7d3f01ab8e2e3a483fd3b8/html5/thumbnails/14.jpg)
15
CNN for normal estimation
Estimator
Filled accumulator
Normal coordinates in Hough Space
CNN
(x, y)
![Page 15: Deep Learning for Robust Normal Estimation in Unstructured ... · 20 CNN for normal estimation Training Requires annotated data for training Using big mesh: - not satisfactory - not](https://reader033.vdocuments.mx/reader033/viewer/2022050323/5f7d3f01ab8e2e3a483fd3b8/html5/thumbnails/15.jpg)
16
Deep learning
80's and 90'sTheory,
optimization... Since 2000's
GPU
DATA
![Page 16: Deep Learning for Robust Normal Estimation in Unstructured ... · 20 CNN for normal estimation Training Requires annotated data for training Using big mesh: - not satisfactory - not](https://reader033.vdocuments.mx/reader033/viewer/2022050323/5f7d3f01ab8e2e3a483fd3b8/html5/thumbnails/16.jpg)
17
Convolutional Neural NetworksMain layer types
ConvolutionsInput: rectangle of pixelsRegular grid of neurons Share weights
PoolingDimension reduction
Fully connectedInput: all neurons of previous layer
Activation layerReLU, TanhIncrease non linearity
n1
nk
f (x)=max (o , x)
![Page 17: Deep Learning for Robust Normal Estimation in Unstructured ... · 20 CNN for normal estimation Training Requires annotated data for training Using big mesh: - not satisfactory - not](https://reader033.vdocuments.mx/reader033/viewer/2022050323/5f7d3f01ab8e2e3a483fd3b8/html5/thumbnails/17.jpg)
18
CNN for normal estimation
Stack of Convolutions and Pooling
Descriptor Classifier
Stack of fully connected layers
LeNet like architecture
![Page 18: Deep Learning for Robust Normal Estimation in Unstructured ... · 20 CNN for normal estimation Training Requires annotated data for training Using big mesh: - not satisfactory - not](https://reader033.vdocuments.mx/reader033/viewer/2022050323/5f7d3f01ab8e2e3a483fd3b8/html5/thumbnails/18.jpg)
19
CNN for normal estimation
Estimator
Filled accumulator
Normal coordinates in Hough Space
CNN
Conv 3
x3 +
ReLU
Conv 3
x3 +
ReLU
Max P
oolin
g 2
x2
Conv 3
x3 +
ReLU
Conv 3
x3 +
ReLU
FC +
ReLU
FC +
ReLU
FC
FC +
ReLU
Max P
oolin
g 2
x2
(x, y)
![Page 19: Deep Learning for Robust Normal Estimation in Unstructured ... · 20 CNN for normal estimation Training Requires annotated data for training Using big mesh: - not satisfactory - not](https://reader033.vdocuments.mx/reader033/viewer/2022050323/5f7d3f01ab8e2e3a483fd3b8/html5/thumbnails/19.jpg)
20
CNN for normal estimationTraining
Requires annotated data for training
Using big mesh:- not satisfactory - not enough difficult points- real point clouds (no ground truth)
Conv 3x3 + ReLU
Conv 3x3 + ReLU
Max Pooling 2x2
Conv 3x3 + ReLU
Conv 3x3 + ReLU
FC + ReLU
FC + ReLU
FC
FC + ReLU
Max Pooling 2x2
Training accumulators
Corresponding normals
n1 = (x
1, y
1)
n2 = (x
2, y
2)
n3 = (x
3, y
3)
...
![Page 20: Deep Learning for Robust Normal Estimation in Unstructured ... · 20 CNN for normal estimation Training Requires annotated data for training Using big mesh: - not satisfactory - not](https://reader033.vdocuments.mx/reader033/viewer/2022050323/5f7d3f01ab8e2e3a483fd3b8/html5/thumbnails/20.jpg)
21
CNN for normal estimationTraining
Conv 3x3 + ReLU
Conv 3x3 + ReLU
Max Pooling 2x2
Conv 3x3 + ReLU
Conv 3x3 + ReLU
FC + ReLU
FC + ReLU
FC
FC + ReLU
Max Pooling 2x2
Synthetic angles
Training accumulators
Corresponding normals
n1 = (x
1, y
1)
n2 = (x
2, y
2)
n3 = (x
3, y
3)
...
L=‖n− n̂‖22
![Page 21: Deep Learning for Robust Normal Estimation in Unstructured ... · 20 CNN for normal estimation Training Requires annotated data for training Using big mesh: - not satisfactory - not](https://reader033.vdocuments.mx/reader033/viewer/2022050323/5f7d3f01ab8e2e3a483fd3b8/html5/thumbnails/21.jpg)
22
CNN for normal estimation
Scale 1
Scale 2
Scale 3
Multi-channeledaccumulator
Multiple neighborhood observations
![Page 22: Deep Learning for Robust Normal Estimation in Unstructured ... · 20 CNN for normal estimation Training Requires annotated data for training Using big mesh: - not satisfactory - not](https://reader033.vdocuments.mx/reader033/viewer/2022050323/5f7d3f01ab8e2e3a483fd3b8/html5/thumbnails/22.jpg)
23
Robust Randomized Hough Transform
Robustness to density variation
Point cloud with density
variations
Compute local scale
Assign picking probability to each point
![Page 23: Deep Learning for Robust Normal Estimation in Unstructured ... · 20 CNN for normal estimation Training Requires annotated data for training Using big mesh: - not satisfactory - not](https://reader033.vdocuments.mx/reader033/viewer/2022050323/5f7d3f01ab8e2e3a483fd3b8/html5/thumbnails/23.jpg)
24
Experiments
Depthmap + CNN vs Hough + CNN
Comparison with existing methods
![Page 24: Deep Learning for Robust Normal Estimation in Unstructured ... · 20 CNN for normal estimation Training Requires annotated data for training Using big mesh: - not satisfactory - not](https://reader033.vdocuments.mx/reader033/viewer/2022050323/5f7d3f01ab8e2e3a483fd3b8/html5/thumbnails/24.jpg)
25
Experiments
Ground truth + outliers
Hough + CNNEstimation without outliers
Hough + CNNEstimation with outliers
![Page 25: Deep Learning for Robust Normal Estimation in Unstructured ... · 20 CNN for normal estimation Training Requires annotated data for training Using big mesh: - not satisfactory - not](https://reader033.vdocuments.mx/reader033/viewer/2022050323/5f7d3f01ab8e2e3a483fd3b8/html5/thumbnails/25.jpg)
26
Experiments
Ground laser office scene
Aerial laser scene (DFC 2015)
Plane fitting [HDD*92] Our method plain Our method density-adaptive
Our method plain Our method density-adaptive
![Page 26: Deep Learning for Robust Normal Estimation in Unstructured ... · 20 CNN for normal estimation Training Requires annotated data for training Using big mesh: - not satisfactory - not](https://reader033.vdocuments.mx/reader033/viewer/2022050323/5f7d3f01ab8e2e3a483fd3b8/html5/thumbnails/26.jpg)
27
Experiments
Château de Sceaux, SfM point cloud
Reconstructed using OpenMVG by Pierre Moulonhttps://github.com/openMVG/openMVG
![Page 27: Deep Learning for Robust Normal Estimation in Unstructured ... · 20 CNN for normal estimation Training Requires annotated data for training Using big mesh: - not satisfactory - not](https://reader033.vdocuments.mx/reader033/viewer/2022050323/5f7d3f01ab8e2e3a483fd3b8/html5/thumbnails/27.jpg)
28
When to use it ?
Smooth surfaceHomogeneous density
Sharp edgesDensity variationsOutliersNoise
PerspectivesAdaptation to structured point clouds
Work on training sets and architectures
Geometric transformation in Hough space
✔
✘
![Page 28: Deep Learning for Robust Normal Estimation in Unstructured ... · 20 CNN for normal estimation Training Requires annotated data for training Using big mesh: - not satisfactory - not](https://reader033.vdocuments.mx/reader033/viewer/2022050323/5f7d3f01ab8e2e3a483fd3b8/html5/thumbnails/28.jpg)
29
Conclusion
Normal estimation in unstructured point cloud
New trend in geometry processing
Deep data driven approach
Hough transform helps the network
![Page 29: Deep Learning for Robust Normal Estimation in Unstructured ... · 20 CNN for normal estimation Training Requires annotated data for training Using big mesh: - not satisfactory - not](https://reader033.vdocuments.mx/reader033/viewer/2022050323/5f7d3f01ab8e2e3a483fd3b8/html5/thumbnails/29.jpg)
30
Hough + CNN :C++ and Lua/Torch 7Trained models available
[BM12] Header onlyOriginal: CGAL, PCLUpdated version (density sensitive): Eigen / NanoFlann
Thank you
</code>
sites.google.com/site/boulchalexandre
![Page 30: Deep Learning for Robust Normal Estimation in Unstructured ... · 20 CNN for normal estimation Training Requires annotated data for training Using big mesh: - not satisfactory - not](https://reader033.vdocuments.mx/reader033/viewer/2022050323/5f7d3f01ab8e2e3a483fd3b8/html5/thumbnails/30.jpg)
31
Computation times
Model Cube Armadillo DFC Detail
Omotondo DFC tile
Size 20k 173k 185k 997k 2.3M
[HDD*92] 0.3 2.1 1.9 12 25
[DG04] 3.2 55 41 441 1243
[CP05] 5.8 50 54 304 711
[BM12] 1.9 13 11 44 147
[LSK*10] 8.8 64 75 392 902
CNN 1s 4.5 33 34 183 423
CNN 3s 5.9 48 52 273 639
CNN 5s 7.9 69 73 382 897