3D Computer VisionIntroductory meeting

Nassir Navab, Federico Tombari, Vasileios Belagiannis, Wadim Kehl, Benjamin Busam

Seminar contents● this seminar includes a selection of recent papers in the field of 3D

computer vision ● papers have been selected to cover different aspects of the topic:

○ 3D keypoint detection, 3D descriptors○ 3D/RGBD object recognition/classification○ 3D semantic segmentation○ 3D modeling and reconstruction○ 3D retrieval○ stereo vision○ point cloud registration○ human pose estimation

Goals● You are going to learn:

○ about relevant recent trends in the field of 3D Computer Vision○ how to read and understand a scientific article○ how to write a scientific report○ how to give a talk to an audience, and deal with related questions

afterwards

Seminar Schedule● 4 sessions, 1 every Tuesday, 2pm-4pm

○ November 25○ December 2○ December 9○ December 16

● Each session will include up to three presentations (first come, first served)

● Seminarraum 03.13.010

Presentation● Each presentation is 25 minutes + 15 minutes for Q&A● Based on slides (Powerpoints, Latex, ..), see website for templates● The presentation should cover all relevant aspects of the paper

○ Introduction and state of the art○ Main contribution(s)○ Experimental results○ Discussion, summary and future work

● The presentation should be self-contained● All students are expected to attend all presentations and interact during

Q&A (this will influence your final mark)

Report● The report should summarize the paper in the way it has been presented

during the talk● Language: English● Max 8 pages● Template on course website● Once ready, send the report to supervisor, within one week from the day

of the presentation

Evaluation criteria● Quality of presentation (both regarding slides and speech)● Quality of the report● Comprehension of the scientific contents of the presented work● Interaction and participation during the other talks

M. Blum et al., “A Learned Feature Descriptor for Object Recognition in RGB-D Data”, ICRA 2012

● Automatic extraction of local feature descriptors for RGB-D data

● Learning of “convolutional k-means” feature responses around salient 2D points

R. Socher et al., “Convolutional-Recursive Deep Learning for 3D Object Classification”, NIPS 2012

● Mixed convolutional neural networks (CNN) and recursive networks (RNN) for object detection in RGB-D data

● CNNs learn low-level features ● RNNs learn high-level features on

CNN output

D. Munoz et al., “Contextual Classification with Functional Max-Margin Markov Networks”, CVPR 2009

● Multi-label classification of 2D and 3D data via Markov Networks

● Contextual neighborhood information boosts accuracy

● Functional-gradient formulation for learning higher-order model parameters

R. Newcombe et al., “KinectFusion: Real-Time Dense Surface Tracking and Mapping”, ISMAR 2011

● Volumetric online reconstruction of scenes via a RGB-D sensor

● Implicit representation of scene geometry by a level-set function

● Simultaneous model reconstruction and camera tracking

A.V. Segal, D. Haehnel, S. Thrun, “Generalized-ICP”, RSS 2005

● Iterative Closest Point (ICP)○ Algorithm for Point Cloud

Registration

● Generalized-ICP○ Combines point-to-point ICP

and point-to-plane ICP into robust probabilistic plane-to-plane framework

● Application Areas○ 2D / 3D multi-scan surface

reconstruction○ Point cloud tracking

C. Rhemann et al., “Fast cost-volume filtering for visual correspondence and beyond”, CVPR 2011.

● Multi-labeling of 3D-content○ stereo image pair○ real-time, accurate

● Idea○ Construct cost-volume○ Filter it○ Select labels (winner-take-all)

● Application areas○ Disparity maps○ Optical flow fields

F. Tombari et al., “Unique signatures of Histograms for local surface description”, ECCV 2010

● Descriptor for point clouds and 3D meshes for 3D object recognition, point cloud registration, 3D retrieval, …

● Employs a 3D grid oriented by means of a stable Local Reference Frame computed on a local neighborhood

● Proves to be robust against noise and point density variations

J. Sun, M. Ovsjanikov, L. Guibas, “A Concise and Provably Informative Multi-Scale Signature Based on Heat Diffusion”, Comput. Graph. Forum, Vol. 28:5, pages 1383-1392, 2009.

● Description of 3D local parts of a 3D Mesh via diffusion of the heat equation

● Intrinsic representation invariant to isometry transformations

● Can be used also to detect repeatable keypoints on the 3D surface

J. Shotton et al., “Real-time human pose recognition in parts from single depth images”,Communications of the ACM, 2013

● Human 3D skeleton estimation● Based on depth data obtained

from a 3D sensor● Use of Random Forest to learn

body parts

S. Li and A.B. Chan, 3D Human Pose Estimation from Monocular Images with Deep Convolutional Neural Network, ACCV 2014.

● Human 3D skeleton estimation

● Image data

● Convolutional neural network