5 stereo reconstruction - new york universityfergus/teaching/vision_2012/5_stereo.pdf · this line...
TRANSCRIPT
![Page 1: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/1.jpg)
5 – Stereo Reconstruction
Slides from A. Zisserman & S. Lazebnik
![Page 2: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/2.jpg)
Overview
• Single camera geometry
• Recap of Homogenous coordinates
• Perspective projection model
• Camera calibration
• Stereo Reconstruction
• Epipolar geometry
• Stereo correspondence
• Triangulation
![Page 3: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/3.jpg)
Single camera geometry
![Page 4: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/4.jpg)
Projection
![Page 5: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/5.jpg)
Projection
![Page 6: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/6.jpg)
Projective Geometry
• Recovery of structure from one image is inherently
ambiguous
• Today focus on geometry that maps world to camera
image
x
X?X?
X?
![Page 7: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/7.jpg)
Recall: Pinhole camera model
• Principal axis: line from the camera center
perpendicular to the image plane
• Normalized (camera) coordinate system: camera
center is at the origin and the principal axis is the z-axis
![Page 8: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/8.jpg)
)/,/(),,( ZYfZXfZYX
101
0
0
1
Z
Y
X
f
f
Z
Yf
Xf
Z
Y
X
Recall: Pinhole camera model
PXx
![Page 9: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/9.jpg)
Recap: Homogeneous coordinates
Slide by Steve Seitz
),(),,(z
yf
z
xfzyx
![Page 10: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/10.jpg)
Principal point
• Principal point (p): point where principal axis intersects the
image plane (origin of normalized coordinate system)
• Normalized coordinate system: origin is at the principal point
• Image coordinate system: origin is in the corner
• How to go from normalized coordinate system to image
coordinate system?
![Page 11: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/11.jpg)
)/,/(),,( yx pZYfpZXfZYX
101
0
0
1
Z
Y
X
pf
pf
Z
pZYf
pZXf
Z
Y
X
y
x
y
x
Principal point offset
principal point: ),( yx pp
![Page 12: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/12.jpg)
101
01
01
1Z
Y
X
pf
pf
Z
ZpYf
ZpXf
y
x
y
x
Principal point offset
1
y
x
pf
pf
K calibration matrix 0|IKP
principal point: ),( yx pp
![Page 13: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/13.jpg)
111
yy
xx
y
x
y
x
pf
pf
m
m
K
Pixel coordinates
• mx pixels per meter in horizontal direction,
my pixels per meter in vertical direction
Pixel size: yx mm
11
pixels/m m pixels
![Page 14: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/14.jpg)
C~
-X~
RX~
cam
Camera rotation and translation
• In general, the camera
coordinate frame will
be related to the world
coordinate frame by a
rotation and a
translation
coords. of point
in camera framecoords. of camera center
in world framecoords. of a point
in world frame (nonhomogeneous)
![Page 15: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/15.jpg)
C~
-X~
RX~
cam
X10
C~
RR
1
X~
10
C~
RRXcam
XC~
R|RKX0|IKx cam ,t|RKP C~
Rt
Camera rotation and translation
In non-homogeneous
coordinates:
Note: C is the null space of the camera projection matrix (PC=0)
![Page 16: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/16.jpg)
Camera parameters
• Intrinsic parameters
• Principal point coordinates
• Focal length
• Pixel magnification factors
• Skew (non-rectangular pixels)
• Radial distortion
111
yy
xx
y
x
y
x
pf
pf
m
m
K
![Page 17: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/17.jpg)
Camera parameters
• Intrinsic parameters
• Principal point coordinates
• Focal length
• Pixel magnification factors
• Skew (non-rectangular pixels)
• Radial distortion
• Extrinsic parameters
• Rotation and translation relative to world coordinate
system
![Page 18: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/18.jpg)
Camera calibration
• Given n points with known 3D coordinates Xi and known
image projections xi, estimate the camera parameters
? P
Xi
xi
![Page 19: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/19.jpg)
ii PXx
Camera calibration
0PXx ii0
XP
XP
XP
1 3
2
1
i
T
i
T
i
T
i
i
y
x
0
P
P
P
0XX
X0X
XX0
3
2
1
T
ii
T
ii
T
ii
T
i
T
ii
T
i
xy
x
y
Two linearly independent equations
![Page 20: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/20.jpg)
Camera calibration
• P has 11 degrees of freedom (12 parameters, but
scale is arbitrary)
• One 2D/3D correspondence gives us two linearly
independent equations
• Homogeneous least squares
• 6 correspondences needed for a minimal solution
0pA0
P
P
P
X0X
XX0
X0X
XX0
3
2
1111
111
T
nn
TT
n
T
nn
T
n
T
TTT
TTT
x
y
x
y
![Page 21: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/21.jpg)
Camera calibration
• Note: for coplanar points that satisfy ΠTX=0,
we will get degenerate solutions (Π,0,0), (0,Π,0), or
(0,0,Π)
0Ap0
P
P
P
X0X
XX0
X0X
XX0
3
2
1111
111
T
nn
TT
n
T
nn
T
n
T
TTT
TTT
x
y
x
y
![Page 22: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/22.jpg)
Camera calibration
• Once we‟ve recovered the numerical form of the camera
matrix, we still have to figure out the intrinsic and
extrinsic parameters
• This is a matrix decomposition problem, not an
estimation problem (see F&P sec. 3.2, 3.3)
![Page 23: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/23.jpg)
CSE 576, Spring 2008 Projective Geometry 23
Alternative: multi-plane calibration
Images courtesy Jean-Yves Bouguet, Intel Corp.
Advantage• Only requires a plane
• Don‟t have to know positions/orientations
• Good code available online!
– Intel‟s OpenCV library: http://www.intel.com/research/mrl/research/opencv/
– Matlab version by Jean-Yves Bouget:
http://www.vision.caltech.edu/bouguetj/calib_doc/index.html
– Zhengyou Zhang‟s web site: http://research.microsoft.com/~zhang/Calib/
![Page 24: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/24.jpg)
Stereo Reconstruction
known
camera
viewpoints
Shape (3D) from two (or more) images
![Page 26: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/26.jpg)
Scenarios
The two images can arise from
• A stereo rig consisting of two cameras
• the two images are acquired simultaneously
or
• A single moving camera (static scene)
• the two images are acquired sequentially
The two scenarios are geometrically equivalent
![Page 27: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/27.jpg)
Stereo head
Camera on a mobile vehicle
![Page 28: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/28.jpg)
The objective
Given two images of a scene acquired by known cameras compute the
3D position of the scene (structure recovery)
Basic principle: triangulate from corresponding image points
• Determine 3D point at intersection of two back-projected rays
![Page 29: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/29.jpg)
Corresponding points are images of the same scene point
Triangulation
C C /
The back-projected points generate rays which intersect at the
3D scene point
![Page 30: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/30.jpg)
An algorithm for stereo reconstruction
1. For each point in the first image determine the
corresponding point in the second image
(this is a search problem)
2. For each pair of matched points determine the 3D
point by triangulation
(this is an estimation problem)
![Page 31: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/31.jpg)
The correspondence problem
Given a point x in one image find the corresponding point in the other
image
This appears to be a 2D search problem, but it is reduced to a 1D search
by the epipolar constraint
![Page 32: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/32.jpg)
1. Epipolar geometry
• the geometry of two cameras
• reduces the correspondence problem to a line search
2. Stereo correspondence algorithms
3. Triangulation
Outline
![Page 33: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/33.jpg)
Notation
x x /
X
C C /
The two cameras are P and P/, and a 3D point X is imaged as
for equations involving homogeneous quantities „=‟ means „equal up to scale‟
P P/
Warning
![Page 34: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/34.jpg)
Epipolar geometry
![Page 35: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/35.jpg)
Epipolar geometry
Given an image point in one view, where is the corresponding point
in the other view?
epipolar line
?
baseline
• A point in one view “generates” an epipolar line in the other view
• The corresponding point lies on this line
epipole C /C
![Page 36: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/36.jpg)
Epipolar line
Epipolar constraint
• Reduces correspondence problem to 1D search along an
epipolar line
![Page 37: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/37.jpg)
Epipolar geometry continued
Epipolar geometry is a consequence of the coplanarity of the camera
centres and scene point
x x /
X
C C /
The camera centres, corresponding points and scene point lie
in a single plane, known as the epipolar plane
![Page 38: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/38.jpg)
Nomenclature
• The epipolar line l/
is the image of the ray through x
• The epipole e is the point of intersection of the line joining the camera centres
with the image plane
this line is the baseline for a stereo rig, and
the translation vector for a moving camera
• The epipole is the image of the centre of the other camera: e = PC/ , e/ = P/C
xx /
X
C C /
e
left epipolar line
right epipolar line
e/
l/
![Page 39: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/39.jpg)
The epipolar pencil
e e /
baseline
X
As the position of the 3D point X varies, the epipolar planes “rotate” about
the baseline. This family of planes is known as an epipolar pencil. All
epipolar lines intersect at the epipole.
(a pencil is a one parameter family)
![Page 40: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/40.jpg)
The epipolar pencil
e e /
baseline
X
As the position of the 3D point X varies, the epipolar planes “rotate” about
the baseline. This family of planes is known as an epipolar pencil. All
epipolar lines intersect at the epipole.
(a pencil is a one parameter family)
![Page 41: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/41.jpg)
Epipolar geometry example I: parallel cameras
Epipolar geometry depends only on the relative pose (position and
orientation) and internal parameters of the two cameras, i.e. the position of
the camera centres and image planes. It does not depend on the scene
structure (3D points external to the camera).
![Page 42: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/42.jpg)
Epipolar geometry example II: converging cameras
Note, epipolar lines are in general not parallel
e e /
![Page 43: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/43.jpg)
Homogeneous notation for lines
![Page 44: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/44.jpg)
• The line l through the two points p and q is l = p x q
Example: compute the point of intersection of the two lines l and m
in the figure below
Proof
y
x
1
2
• The intersection of two lines l and m is the point x = l x m
l
m
which is the point (2,1)
![Page 45: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/45.jpg)
Matrix representation of the vector cross product
![Page 46: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/46.jpg)
Example: compute the cross product of l and m
![Page 47: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/47.jpg)
Algebraic representation of epipolar geometry
We know that the epipolar geometry defines a mapping
x l/
point in first
imageepipolar line in
second image
![Page 48: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/48.jpg)
P
Derivation of the algebraic expression
Outline
Step 1: for a point x in the first image
back project a ray with camera P
Step 2: choose two points on the ray and
project into the second image with camera P/
Step 3: compute the line through the two
image points using the relation l/= p x q
P/
![Page 49: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/49.jpg)
• choose camera matrices
internal
calibrationrotation translation
from world to camera
coordinate frame
• first camera
world coordinate frame aligned with first camera
• second camera
![Page 50: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/50.jpg)
Step 1: for a point x in the first image
back project a ray with cameraP
A point x back projects to a ray
where Z is the point‟s depth, since
satisfies
![Page 51: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/51.jpg)
Step 2: choose two points on the ray and
project into the second image with camera P/
P/
Consider two points on the ray
• Z = 0 is the camera centre
• Z = is the point at infinity
Project these two points into the second view
![Page 52: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/52.jpg)
Using the identity
Compute the line through the points
F
F is the fundamental matrix
Step 3: compute the line through the two
image points using the relation l/= p x q
![Page 53: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/53.jpg)
Example I: compute the fundamental matrix for a parallel camera stereo rig
• reduces to y = y/ , i.e. raster correspondence (horizontal scan-lines)
f
f
XY
Z
![Page 54: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/54.jpg)
f
f
XY
Z
Geometric interpretation ?
![Page 55: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/55.jpg)
Example II: compute F for a forward translating camera
f
f
XY
Z
![Page 56: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/56.jpg)
f
f
X Y
Z
first image second image
![Page 57: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/57.jpg)
![Page 58: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/58.jpg)
![Page 59: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/59.jpg)
Summary: Properties of the Fundamental matrix
![Page 60: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/60.jpg)
Admin Interlude
• Assignment 1 due next week at start of
class
• Office hours right after class!
• Meet Hao Li ([email protected]) whos is
the class TA
• Who did not get my class emails?
• Please come and (legibly!) give me your
email again.
![Page 61: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/61.jpg)
Stereo correspondence
algorithms
![Page 62: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/62.jpg)
Problem statement
Given: two images and their associated cameras compute
corresponding image points.
Algorithms may be classified into two types:
1. Dense: compute a correspondence at every pixel
2. Sparse: compute correspondences only for features
The methods may be top down or bottom up
![Page 63: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/63.jpg)
Top down matching
1. Group model (house, windows, etc) independently in
each image
2. Match points (vertices) between images
![Page 64: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/64.jpg)
Bottom up matching
• epipolar geometry reduces the correspondence search from 2D
to a 1D search on corresponding epipolar lines
• 1D correspondence problem
b/
a/
bca
CBA
c/
![Page 65: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/65.jpg)
![Page 66: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/66.jpg)
Stereograms
• Invented by Sir Charles Wheatstone, 1838
![Page 67: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/67.jpg)
Red/green stereograms
![Page 68: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/68.jpg)
Random dot stereograms
![Page 69: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/69.jpg)
![Page 70: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/70.jpg)
Autostereograms
Autostereograms: www.magiceye.com
![Page 71: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/71.jpg)
Autostereograms
Autostereograms: www.magiceye.com
![Page 72: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/72.jpg)
Correspondence algorithms
Algorithms may be top down or bottom up – random dot stereograms
are an existence proof that bottom up algorithms are possible
From here on only consider bottom up algorithms
Algorithms may be classified into two types:
1. Dense: compute a correspondence at every pixel
2. Sparse: compute correspondences only for features
![Page 73: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/73.jpg)
Example image pair – parallel cameras
![Page 74: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/74.jpg)
First image
![Page 75: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/75.jpg)
Second image
![Page 76: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/76.jpg)
Dense correspondence algorithm
Search problem (geometric constraint): for each point in the left image, the
corresponding point in the right image lies on the epipolar line (1D ambiguity)
Disambiguating assumption (photometric constraint): the intensity
neighbourhood of corresponding points are similar across images
Measure similarity of neighbourhood intensity by cross-correlation
Parallel camera example – epipolar lines are corresponding rasters
epipolar
line
![Page 77: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/77.jpg)
Intensity profiles
• Clear correspondence between intensities, but also noise and ambiguity
![Page 78: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/78.jpg)
region A
Normalized Cross Correlation
region B
vector a vector b
write regions as vectors
a
b
![Page 79: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/79.jpg)
Cross-correlation of neighbourhood regions
epipolar
line
translate so that mean is zero
(exercise)
![Page 80: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/80.jpg)
left image band
right image band
cross
correlation
1
0
0.5
x
![Page 81: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/81.jpg)
left image band
right image band
cross
correlation
1
0
x
0.5
target region
![Page 82: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/82.jpg)
Why is cross-correlation such a poor measure in the second case?
1. The neighbourhood region does not have a “distinctive” spatial intensity
distribution
2. Foreshortening effects
fronto-parallel surface
imaged length the same
slanting surface
imaged lengths differ
![Page 83: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/83.jpg)
Limitations of similarity constraint
Textureless surfaces Occlusions, repetition
Non-Lambertian surfaces, specularities
![Page 84: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/84.jpg)
Results with window search
Window-based matching Ground truth
Data
![Page 85: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/85.jpg)
Sketch of a dense correspondence algorithm
For each pixel in the left image
• compute the neighbourhood cross correlation along the corresponding epipolar line in the right image
• the corresponding pixel is the one with the highest cross correlation
Parameters
• size (scale) of neighbourhood
• search disparity
Other constraints
• uniqueness
• ordering
• smoothness of disparity field
Applicability
• textured scene, largely fronto-parallel
![Page 86: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/86.jpg)
Stereo matching as energy minimization
MAP estimate of disparity image D: )()|,(),|( 2121 DPDIIPIIDP
I1I2 D
W1(i) W2(i+D(i)) D(i)
)(),,( smooth21data DEDIIEE
)(log)|,(log),|(log 2121 DPDIIPIIDP
ji
jDiDE,neighbors
smooth )()(2
21data ))(()(i
iDiWiWE
![Page 87: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/87.jpg)
Stereo matching as energy minimization
I1I2 D
• Energy functions of this form can be minimized using
graph cuts
Y. Boykov, O. Veksler, and R. Zabih, Fast Approximate Energy Minimization via Graph Cuts, PAMI 2001
W1(i) W2(i+D(i)) D(i)
)(),,( smooth21data DEDIIEE
ji
jDiDE,neighbors
smooth )()(2
21data ))(()(i
iDiWiWE
![Page 88: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/88.jpg)
Graph cuts solution
Graph cuts Ground truth
For the latest and greatest: http://www.middlebury.edu/stereo/
Y. Boykov, O. Veksler, and R. Zabih, Fast Approximate Energy
Minimization via Graph Cuts, PAMI 2001
![Page 89: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/89.jpg)
Example dense correspondence algorithm
left image right image
![Page 91: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/91.jpg)
Texture mapped 3D triangulation
![Page 92: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/92.jpg)
range map
Pentagon example
left image right image
![Page 93: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/93.jpg)
Rectification
e e /
For converging cameras
• epipolar lines are not parallel
![Page 94: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/94.jpg)
Project images onto plane parallel to baseline
epipolar plane
![Page 95: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/95.jpg)
Rectification continued
Convert converging cameras to parallel camera
geometry by an image mapping
Image mapping is a 2D homography (projective transformation)
(exercise)
![Page 96: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/96.jpg)
Rectification continued
Convert converging cameras to parallel camera
geometry by an image mapping
Image mapping is a 2D homography (projective transformation)
(exercise)
![Page 97: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/97.jpg)
Exampleoriginal stereo pair
rectified stereo pair
![Page 98: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/98.jpg)
Note
• image movement (disparity) is inversely proportional to depth Z
• depth is inversely proportional to disparity
Example: depth and disparity for a parallel camera stereo rig
Then, y/ = y, and the disparity
Derivation
x
x/
d
![Page 99: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/99.jpg)
Triangulation
![Page 100: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/100.jpg)
1. Vector solution
C C /
Compute the mid-point of the shortest line between the
two rays
![Page 101: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/101.jpg)
2. Linear triangulation (algebraic solution)
![Page 102: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/102.jpg)
Problem: does not minimize anything meaningful
Advantage: extends to more than two views
![Page 103: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/103.jpg)
3. Minimizing a geometric/statistical error
![Page 104: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/104.jpg)
• It can be shown that if the measurement noise is
Gaussian mean zero, , then minimizing
geometric error is the Maximum Likelihood Estimate of X
• The minimization appears to be over three parameters
(the position X), but the problem can be reduced to a
minimization over one parameter
![Page 105: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/105.jpg)
Different formulation of the problem
![Page 106: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/106.jpg)
Minimization method
• Parametrize the pencil of epipolar lines in the first image by t,
such that the epipolar line is l(t)
• Using F compute the corresponding epipolar line in the second
image l/ (t)
• Express the distance function explicitly as a
function of t
• Find the value of t that minimizes the distance function
• Solution is a 6th degree polynomial in t
![Page 107: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/107.jpg)
Other approaches
to obtaining 3D
structure
![Page 108: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/108.jpg)
Active stereo with structured light
• Project “structured” light patterns onto the object
• simplifies the correspondence problem
• Allows us to use only one camera
camera
projector
L. Zhang, B. Curless, and S. M. Seitz. Rapid Shape Acquisition Using Color Structured
Light and Multi-pass Dynamic Programming. 3DPVT 2002
![Page 109: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/109.jpg)
Active stereo with structured light
L. Zhang, B. Curless, and S. M. Seitz. Rapid Shape Acquisition Using Color
Structured Light and Multi-pass Dynamic Programming. 3DPVT 2002
![Page 110: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/110.jpg)
Microsoft Kinect
![Page 111: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/111.jpg)
Laser scanning
• Optical triangulation
• Project a single stripe of laser light
• Scan it across the surface of the object
• This is a very precise version of structured light scanning
Digital Michelangelo Projecthttp://graphics.stanford.edu/projects/mich/
Source: S. Seitz
![Page 112: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/112.jpg)
Laser scanned models
The Digital Michelangelo Project, Levoy et al.
Source: S. Seitz
![Page 113: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/113.jpg)
Laser scanned models
The Digital Michelangelo Project, Levoy et al.
Source: S. Seitz
![Page 114: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/114.jpg)
Laser scanned models
The Digital Michelangelo Project, Levoy et al.
Source: S. Seitz
![Page 115: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/115.jpg)
Laser scanned models
The Digital Michelangelo Project, Levoy et al.
Source: S. Seitz
![Page 116: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/116.jpg)
Laser scanned models
The Digital Michelangelo Project, Levoy et al.
Source: S. Seitz
![Page 117: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/117.jpg)
Aligning range images
• A single range scan is not sufficient to describe a
complex surface
• Need techniques to register multiple range images
B. Curless and M. Levoy, A Volumetric Method for Building Complex Models from
Range Images, SIGGRAPH 1996
![Page 118: 5 Stereo Reconstruction - New York Universityfergus/teaching/vision_2012/5_Stereo.pdf · this line is the baseline for a stereo rig, and the translation vector for a moving camera](https://reader035.vdocuments.mx/reader035/viewer/2022081222/5f78d542e354377eca316df8/html5/thumbnails/118.jpg)
Aligning range images
• A single range scan is not sufficient to describe a
complex surface
• Need techniques to register multiple range images
• … which brings us to multi-view stereo