triangulation - %% triangulation % go to each correspondence and compute the 3d point x (3xn) matrix...
TRANSCRIPT
Triangulation
How to Disambiguate?
v
Bob Alice
X
P K I 0bob 3 3 1
P K R talice
u
v
Bob Alice
X
P K I 0bob 3 3 1
P K R talice
u
v
Bob Alice
X
P K I 0bob 3 3 1
P K R talice
u
-11K u
-11 K u
v
Bob Alice
X
P K I 0bob 3 3 1
P K R talice
u
-11K u
Baseline
T-R t
-11 K u
v
Bob Alice
X
P K I 0bob 3 3 1
P K R talice
u
-11K u
Baseline
T-R t
T -1 T2 - R K v R t
Direction
-11 K u
v
Bob Alice
X
P K I 0bob 3 3 1
P K R talice
u
-11K u
Baseline
T-R t
T -1 T2 - R K v R t
Direction
-1 T -1 T1 2 - X K u R K v R t
-11 K u
3D point
# of unknowns: 2 # of equations: 3
v
Bob Alice
X
P K I 0bob 3 3 1
P K R talice
u
-11K u
Baseline
T-R t
T -1 T2 - R K v R t
Direction
-1 T -1 T1 2 - X K u R K v R t
-11 K u
3D point
-1 T -1 T1 2 - K u R K v R t-
v
Bob Alice
X
P K I 0bob 3 3 1
P K R talice
u
-11K u
Baseline
T-R t
T -1 T2 - R K v R t
Direction
-1 T -1 T1 2 - X K u R K v R t
-11 K u
3D point
-1 T -1 T1 2 - K u R K v R t-
1-1 T -1 T
2
-
K u R K v R t-
v
Bob Alice
X
P K I 0bob 3 3 1
P K R talice
u
-11K u
Baseline
T-R t
T -1 T2 - R K v R t
Direction
-1 T -1 T1 2 - X K u R K v R t
-11 K u
3D point
-1 T -1 T1 2 - K u R K v R t-
1-1 T -1 T
2
-
K u R K v R t-
3x2
A x b
v
Bob Alice
X
P K I 0bob 3 3 1
P K R talice
u
-11K u
Baseline
1-1 T -1 T
2
-
K u R K v R t-
3x2
A x b
What if two does not meet at a point?
v
Bob Alice
X
P K I 0bob 3 3 1
P K R talice
u
-11K u
Baseline
1-1 T -1 T
2
-
K u R K v R t-
3x2
A x b
What if two does not meet at a point?
Least square solution finds somewhere in the middle.
v
Bob Alice
X
bobP aliceP
u
Baseline
General Case General camera pose
1 bob=1 1
u XP
Two 3D vectors are parallel.
bob1 1
u XP 0
bob1 1
u XP 0
Skew-symmetric matrix
v
Bob Alice
X
bobP aliceP
u
Baseline
General Case General camera pose
1 bob=1 1
u XP
Two 3D vectors are parallel.
bob1 1
u XP 0
bob1 1
u XP 0
Skew-symmetric matrix
: Knowns
: Unknowns
v
Bob Alice
X
bobP aliceP
u
Baseline
General Case General camera pose
1 bob=1 1
u XP
Two 3D vectors are parallel.
bob1 1
u XP 0
bob1 1
u XP 0 : Knowns
: Unknowns
3x4
Can we solve for X? (single view reconstruction) Why not?
v
Bob Alice
X
bobP aliceP
u
Baseline
General Case General camera pose
1 bob=1 1
u XP
Two 3D vectors are parallel.
bob1 1
u XP 0
bob1 1
u XP 0 : Knowns
: Unknowns
2x4
v
Bob Alice
X
bobP aliceP
u
Baseline
General Case General camera pose
1 bob=1 1
u XP
Two 3D vectors are parallel.
bob1 1
u XP 0
bob1 1
u XP 0 : Knowns
: Unknowns
alice1
vP
4x4
v
Bob Alice
X
bobP aliceP
u
Baseline
General Case General camera pose
1 bob=1 1
u XP
Two 3D vectors are parallel.
bob1 1
u XP 0
bob1 1
u XP 0 : Knowns
: Unknowns
alice1
vP
mikeP
mike1
wP
function Triangulation %% Data loading load('triangulation.mat'); % (C1, R1) and (C2, R2) are camera center and orientation of camera 1 and 2, respectively. % u and v are Nx2 correspondences %% Camera matrix build K = [700/2 0 960/2; 0 700/2 540/2; 0 0 1]; % Build camera matrix 1 and 2 % P1 % P2 %% Triangulation % Go to each correspondence and compute the 3D point X (3xN) matrix for i = 1 : size(u,1) % Construct A matrix % Solve linear least squares to get 3D point % X(:,i) = point_3d; end
Download Triangulation.m and triangulation.mat files
Fill out
Fill out
bob
alice
1
1
1
uP
X0
vP
Bob
X
u
Special Case: Stereo
v
Mike
• Same orientation
Bob
X
u
Special Case: Stereo
v
Mike
• Same orientation • Alignment between X axis and baseline
Bob
X
u
Special Case: Stereo
v
Mike
• Same orientation • Alignment between X axis and baseline
Bob
X
u
Special Case: Stereo
v
Mike
• Same orientation • Alignment between X axis and baseline
Top view
Bob
X
u
Special Case: Stereo
v
Mike
• Same orientation • Alignment between X axis and baseline
Top view Epipole? Point at infinity
Epipolar line
Special Case: Stereo
Special Case: Stereo
Special Case: Stereo
Bob
u
Special Case: Stereo
v
Mike
Epipolar line
Bob
u
Special Case: Stereo
v
Mike
Epipolar line Lx Rx
Z
f
b
Z=?
: Baseline Focal length
Dep
th
RxLx
Bob
u
Special Case: Stereo
v
Mike
Epipolar line Lx Rx
Z
f
b
Z=?
RxLx
d
: Baseline Focal length
Dep
th
: Disparity
dLx Rx
Bob
u
Special Case: Stereo
v
Mike
Epipolar line Lx Rx
Z
f
b
Z=?
RxLx
d
: Baseline Focal length
Dep
th
: Disparity
dLx Rx
Bob
u
Special Case: Stereo
v
Mike
Epipolar line Lx Rx
Z
f
b
Z=?
RxLx
d
: Baseline Focal length
Dep
th
: Disparity
dLx Rx
Two triangles are similar.
Z f
b d
fZ b
d
Stereo Rectification
Stereo Rectification
u v
• Same orientation
• Alignment between X axis and baseline
Epipolar line
bob = P K I 0 mike = - P KR I C
Tx
Trect y
Tz
=
r
R r
r
x =C
rC
z z x x
z
z z x x
=
r r r rr
r r r r: Orthogonal projection
z
0
0
1
rwhere
y z x= r r r
Stereo Rectification
u vEpipolar line
bob = P K I 0 mike = - P KR I C
Homography by pure rotation: rectR
-1bob rect=H KR K
T -1mike rect=H KR R K
Dense Feature Matching using SIFT Flow
descriptor1 descriptor2
Search direction
Find a minimum distance over the epipolar line
Disparity Map = Inverse Depth
Disparity Map: Stereo Matching using SIFT Flow
Disparity Map: Stereo Matching using SIFT Flow Disparity Map: Stereo Matching using SIFT Flow Disparity Map = Inverse Depth
Disparity Map: Stereo Matching using SIFT Flow Disparity Map: Stereo Matching using SIFT Flow Disparity Map = Inverse Depth
Dense Reconstruction using a Monocular Camera