recovering structure from a single view pinhole...
TRANSCRIPT
![Page 1: Recovering structure from a single view Pinhole ...inst.eecs.berkeley.edu/~ee290t/fa19/lectures/lecture9-1-introduction-to-epipolar... · Pinhole perspective projection Recovering](https://reader033.vdocuments.mx/reader033/viewer/2022041822/5e5ef3cb3dcfb35a5e441af2/html5/thumbnails/1.jpg)
Pinhole perspective projectionRecovering structure from a single view
C
Ow
Pp
Calibration rigScene
Camera K
From calibration rig
From points and lines at infinity + orthogonal lines and planes ® structure of the scene, K
® location/pose of the rig, K
Knowledge about scene (point correspondences, geometry of lines & planes, etc…
![Page 2: Recovering structure from a single view Pinhole ...inst.eecs.berkeley.edu/~ee290t/fa19/lectures/lecture9-1-introduction-to-epipolar... · Pinhole perspective projection Recovering](https://reader033.vdocuments.mx/reader033/viewer/2022041822/5e5ef3cb3dcfb35a5e441af2/html5/thumbnails/2.jpg)
Pinhole perspective projectionRecovering structure from a single view
C
Ow
Pp
Calibration rigScene
Camera K
Why is it so difficult?
Intrinsic ambiguity of the mapping from 3D to image (2D)
![Page 3: Recovering structure from a single view Pinhole ...inst.eecs.berkeley.edu/~ee290t/fa19/lectures/lecture9-1-introduction-to-epipolar... · Pinhole perspective projection Recovering](https://reader033.vdocuments.mx/reader033/viewer/2022041822/5e5ef3cb3dcfb35a5e441af2/html5/thumbnails/3.jpg)
Recovering structure from a single view
Intrinsic ambiguity of the mapping from 3D to image (2D)
Courtesy slide S. Lazebnik
![Page 4: Recovering structure from a single view Pinhole ...inst.eecs.berkeley.edu/~ee290t/fa19/lectures/lecture9-1-introduction-to-epipolar... · Pinhole perspective projection Recovering](https://reader033.vdocuments.mx/reader033/viewer/2022041822/5e5ef3cb3dcfb35a5e441af2/html5/thumbnails/4.jpg)
Two eyes help!
![Page 5: Recovering structure from a single view Pinhole ...inst.eecs.berkeley.edu/~ee290t/fa19/lectures/lecture9-1-introduction-to-epipolar... · Pinhole perspective projection Recovering](https://reader033.vdocuments.mx/reader033/viewer/2022041822/5e5ef3cb3dcfb35a5e441af2/html5/thumbnails/5.jpg)
O1 O2
pp’
Twoeyeshelp!
Thisiscalledtriangulation
K’ =knownK =known
R, T
P = l × "lP
l 'l[Eq. 1]
![Page 6: Recovering structure from a single view Pinhole ...inst.eecs.berkeley.edu/~ee290t/fa19/lectures/lecture9-1-introduction-to-epipolar... · Pinhole perspective projection Recovering](https://reader033.vdocuments.mx/reader033/viewer/2022041822/5e5ef3cb3dcfb35a5e441af2/html5/thumbnails/6.jpg)
• FindP*thatminimizesd(p,M P*)+ d(p ',M 'P*)
O1 O2
pp’
P
Triangulation
[Eq. 2]
MP*M’P*
P*
![Page 7: Recovering structure from a single view Pinhole ...inst.eecs.berkeley.edu/~ee290t/fa19/lectures/lecture9-1-introduction-to-epipolar... · Pinhole perspective projection Recovering](https://reader033.vdocuments.mx/reader033/viewer/2022041822/5e5ef3cb3dcfb35a5e441af2/html5/thumbnails/7.jpg)
Multi(stereo)-viewgeometry
• Camerageometry:Givencorrespondingpointsintwoimages,findcameramatrices,positionandpose.
• Scenegeometry:Findcoordinatesof3Dpointfromitsprojectioninto2ormultipleimages.
• Correspondence:Givenapointp inoneimage,howcanIfindthecorrespondingpointp’ inanotherone?
![Page 8: Recovering structure from a single view Pinhole ...inst.eecs.berkeley.edu/~ee290t/fa19/lectures/lecture9-1-introduction-to-epipolar... · Pinhole perspective projection Recovering](https://reader033.vdocuments.mx/reader033/viewer/2022041822/5e5ef3cb3dcfb35a5e441af2/html5/thumbnails/8.jpg)
The epipolar geometry
C,C’,x,x’ and X are coplanar
(a)
![Page 9: Recovering structure from a single view Pinhole ...inst.eecs.berkeley.edu/~ee290t/fa19/lectures/lecture9-1-introduction-to-epipolar... · Pinhole perspective projection Recovering](https://reader033.vdocuments.mx/reader033/viewer/2022041822/5e5ef3cb3dcfb35a5e441af2/html5/thumbnails/9.jpg)
The epipolar geometry
What if only C,C’,x are known?
b
• If we know x, how is the corresponding
point x’ constrained?
• l’ is the Epipolar line
corresponding to point x
• Upshot: if we know C and C’
for a stereo correspondence
algorithm, no need to search
all over the second image,
but just only over the epipolar
line.
C C’
![Page 10: Recovering structure from a single view Pinhole ...inst.eecs.berkeley.edu/~ee290t/fa19/lectures/lecture9-1-introduction-to-epipolar... · Pinhole perspective projection Recovering](https://reader033.vdocuments.mx/reader033/viewer/2022041822/5e5ef3cb3dcfb35a5e441af2/html5/thumbnails/10.jpg)
The epipolar geometry
All points on p project on l and l’
a
C C’
• Baseline: connects two camera centers
• Epipole: point of intersection of baseline
with image plane
• Epipole: image in one view of the camera
center of the other view.
![Page 11: Recovering structure from a single view Pinhole ...inst.eecs.berkeley.edu/~ee290t/fa19/lectures/lecture9-1-introduction-to-epipolar... · Pinhole perspective projection Recovering](https://reader033.vdocuments.mx/reader033/viewer/2022041822/5e5ef3cb3dcfb35a5e441af2/html5/thumbnails/11.jpg)
The epipolar geometry
Family of planes p and lines l and l’
Intersection in e and e’
b
• Epipolar plane: A plane containing the
baseline.
• There is a one parameter family , or a
pencil, of epipolar planes
• Epipolar line is the intersection of an
epipolar plane with the image plane
• All epipolar lines intersect at the epipole
• An epipolar plane intersects the left and
right image planes in epipolar lines,
and defines the correspondence
between the lines.
![Page 12: Recovering structure from a single view Pinhole ...inst.eecs.berkeley.edu/~ee290t/fa19/lectures/lecture9-1-introduction-to-epipolar... · Pinhole perspective projection Recovering](https://reader033.vdocuments.mx/reader033/viewer/2022041822/5e5ef3cb3dcfb35a5e441af2/html5/thumbnails/12.jpg)
The epipolar geometry
epipoles e,e’
= intersection of baseline with image plane
= projection of projection center in other image
= vanishing point of camera motion direction
an epipolar plane = plane containing baseline (1-D family)
an epipolar line = intersection of epipolar plane with image
(always come in corresponding pairs)
![Page 13: Recovering structure from a single view Pinhole ...inst.eecs.berkeley.edu/~ee290t/fa19/lectures/lecture9-1-introduction-to-epipolar... · Pinhole perspective projection Recovering](https://reader033.vdocuments.mx/reader033/viewer/2022041822/5e5ef3cb3dcfb35a5e441af2/html5/thumbnails/13.jpg)
An Introduction toComputer Vision
10p
Two views of a collection of objectswo v ews o co ec o o objec s
Assume that the full camera calibration is known. That is the extrinsic parameters of both cameras are known.
![Page 14: Recovering structure from a single view Pinhole ...inst.eecs.berkeley.edu/~ee290t/fa19/lectures/lecture9-1-introduction-to-epipolar... · Pinhole perspective projection Recovering](https://reader033.vdocuments.mx/reader033/viewer/2022041822/5e5ef3cb3dcfb35a5e441af2/html5/thumbnails/14.jpg)
An Introduction toComputer Vision
11p
Some points have been identified in the right hand viewSo e po s ve bee de ed e g d v ew
![Page 15: Recovering structure from a single view Pinhole ...inst.eecs.berkeley.edu/~ee290t/fa19/lectures/lecture9-1-introduction-to-epipolar... · Pinhole perspective projection Recovering](https://reader033.vdocuments.mx/reader033/viewer/2022041822/5e5ef3cb3dcfb35a5e441af2/html5/thumbnails/15.jpg)
An Introduction toComputer Vision
12p
These points define epipolar lines in the left hand viewese po s de e ep po es e e d v ew
![Page 16: Recovering structure from a single view Pinhole ...inst.eecs.berkeley.edu/~ee290t/fa19/lectures/lecture9-1-introduction-to-epipolar... · Pinhole perspective projection Recovering](https://reader033.vdocuments.mx/reader033/viewer/2022041822/5e5ef3cb3dcfb35a5e441af2/html5/thumbnails/16.jpg)
An Introduction toComputer Vision
13p
The corresponding points are located on the epipolar lines in the left hand view
![Page 17: Recovering structure from a single view Pinhole ...inst.eecs.berkeley.edu/~ee290t/fa19/lectures/lecture9-1-introduction-to-epipolar... · Pinhole perspective projection Recovering](https://reader033.vdocuments.mx/reader033/viewer/2022041822/5e5ef3cb3dcfb35a5e441af2/html5/thumbnails/17.jpg)
An Introduction toComputer Vision
14p
The points in the left hand view in turn define epipolar lines in the right hand view, and these lines pass through the points in the right hand view.
![Page 18: Recovering structure from a single view Pinhole ...inst.eecs.berkeley.edu/~ee290t/fa19/lectures/lecture9-1-introduction-to-epipolar... · Pinhole perspective projection Recovering](https://reader033.vdocuments.mx/reader033/viewer/2022041822/5e5ef3cb3dcfb35a5e441af2/html5/thumbnails/18.jpg)
An Introduction toComputer Vision
15Epipolar Geometryp
Epipolar geometry is – dependant only on the (internal and external) camera parameters.dependant only on the (internal and external) camera parameters.– independent of the 3D structure of a scene.
Figure courtesy of Richard Hartley, from Multiple View Geometry in Computer Vision, Hartley and Zisserman, Cambridge, 2000.
![Page 19: Recovering structure from a single view Pinhole ...inst.eecs.berkeley.edu/~ee290t/fa19/lectures/lecture9-1-introduction-to-epipolar... · Pinhole perspective projection Recovering](https://reader033.vdocuments.mx/reader033/viewer/2022041822/5e5ef3cb3dcfb35a5e441af2/html5/thumbnails/19.jpg)
Example:
converging
cameras
![Page 20: Recovering structure from a single view Pinhole ...inst.eecs.berkeley.edu/~ee290t/fa19/lectures/lecture9-1-introduction-to-epipolar... · Pinhole perspective projection Recovering](https://reader033.vdocuments.mx/reader033/viewer/2022041822/5e5ef3cb3dcfb35a5e441af2/html5/thumbnails/20.jpg)
• Epipolar Plane • Epipoles e, e’
• Epipolar Lines• Baseline
Epipolar geometry
O1 O2
p’
P
p
e e’
= intersections of baseline with image planes = projections of the other camera center
![Page 21: Recovering structure from a single view Pinhole ...inst.eecs.berkeley.edu/~ee290t/fa19/lectures/lecture9-1-introduction-to-epipolar... · Pinhole perspective projection Recovering](https://reader033.vdocuments.mx/reader033/viewer/2022041822/5e5ef3cb3dcfb35a5e441af2/html5/thumbnails/21.jpg)
Example of epipolar lines
![Page 22: Recovering structure from a single view Pinhole ...inst.eecs.berkeley.edu/~ee290t/fa19/lectures/lecture9-1-introduction-to-epipolar... · Pinhole perspective projection Recovering](https://reader033.vdocuments.mx/reader033/viewer/2022041822/5e5ef3cb3dcfb35a5e441af2/html5/thumbnails/22.jpg)
O1 O2
P
e’p p’e
Example: Parallel image planes
• Baseline intersects the image plane at infinity• Epipoles are at infinity• Epipolar lines are parallel to u axis
u u
v v
![Page 23: Recovering structure from a single view Pinhole ...inst.eecs.berkeley.edu/~ee290t/fa19/lectures/lecture9-1-introduction-to-epipolar... · Pinhole perspective projection Recovering](https://reader033.vdocuments.mx/reader033/viewer/2022041822/5e5ef3cb3dcfb35a5e441af2/html5/thumbnails/23.jpg)
Example:ParallelImagePlanes
![Page 24: Recovering structure from a single view Pinhole ...inst.eecs.berkeley.edu/~ee290t/fa19/lectures/lecture9-1-introduction-to-epipolar... · Pinhole perspective projection Recovering](https://reader033.vdocuments.mx/reader033/viewer/2022041822/5e5ef3cb3dcfb35a5e441af2/html5/thumbnails/24.jpg)
Example: motion parallel with image plane
![Page 25: Recovering structure from a single view Pinhole ...inst.eecs.berkeley.edu/~ee290t/fa19/lectures/lecture9-1-introduction-to-epipolar... · Pinhole perspective projection Recovering](https://reader033.vdocuments.mx/reader033/viewer/2022041822/5e5ef3cb3dcfb35a5e441af2/html5/thumbnails/25.jpg)
An Introduction toComputer Vision
22p
![Page 26: Recovering structure from a single view Pinhole ...inst.eecs.berkeley.edu/~ee290t/fa19/lectures/lecture9-1-introduction-to-epipolar... · Pinhole perspective projection Recovering](https://reader033.vdocuments.mx/reader033/viewer/2022041822/5e5ef3cb3dcfb35a5e441af2/html5/thumbnails/26.jpg)
Example: Forward translation
• The epipoles have same position in both images• Epipole called FOE (focus of expansion)
O2
e
e’
O1
P
![Page 27: Recovering structure from a single view Pinhole ...inst.eecs.berkeley.edu/~ee290t/fa19/lectures/lecture9-1-introduction-to-epipolar... · Pinhole perspective projection Recovering](https://reader033.vdocuments.mx/reader033/viewer/2022041822/5e5ef3cb3dcfb35a5e441af2/html5/thumbnails/27.jpg)
- Two views of the same object - Given a point on left image, how can I find the corresponding point on right image?
Epipolar Constraint
pWhere is p’?
![Page 28: Recovering structure from a single view Pinhole ...inst.eecs.berkeley.edu/~ee290t/fa19/lectures/lecture9-1-introduction-to-epipolar... · Pinhole perspective projection Recovering](https://reader033.vdocuments.mx/reader033/viewer/2022041822/5e5ef3cb3dcfb35a5e441af2/html5/thumbnails/28.jpg)
Epipolar geometry
O1 O2
P
p p’
Epipolar line 2
![Page 29: Recovering structure from a single view Pinhole ...inst.eecs.berkeley.edu/~ee290t/fa19/lectures/lecture9-1-introduction-to-epipolar... · Pinhole perspective projection Recovering](https://reader033.vdocuments.mx/reader033/viewer/2022041822/5e5ef3cb3dcfb35a5e441af2/html5/thumbnails/29.jpg)
Epipolar Constraint
p p’Epipolar line 2
![Page 30: Recovering structure from a single view Pinhole ...inst.eecs.berkeley.edu/~ee290t/fa19/lectures/lecture9-1-introduction-to-epipolar... · Pinhole perspective projection Recovering](https://reader033.vdocuments.mx/reader033/viewer/2022041822/5e5ef3cb3dcfb35a5e441af2/html5/thumbnails/30.jpg)
• l = E p’ is the epipolar line associated with p’ • l’ = ET p is the epipolar line associated with p • E e’ = 0 and ET e = 0 • E is 3x3 matrix; 5 DOF• E is singular (rank two)
Epipolar Constraint
O1 O2
p’
P
p
e e’l l’
pT ⋅ E p ' = 0[Eq. 10]
![Page 31: Recovering structure from a single view Pinhole ...inst.eecs.berkeley.edu/~ee290t/fa19/lectures/lecture9-1-introduction-to-epipolar... · Pinhole perspective projection Recovering](https://reader033.vdocuments.mx/reader033/viewer/2022041822/5e5ef3cb3dcfb35a5e441af2/html5/thumbnails/31.jpg)
Fundamental Matrix
F is a projective mapping x l’ from a point x in one image to its
Corresponding epipolar line in the other image
l’ = Fx
![Page 32: Recovering structure from a single view Pinhole ...inst.eecs.berkeley.edu/~ee290t/fa19/lectures/lecture9-1-introduction-to-epipolar... · Pinhole perspective projection Recovering](https://reader033.vdocuments.mx/reader033/viewer/2022041822/5e5ef3cb3dcfb35a5e441af2/html5/thumbnails/32.jpg)
The fundamental matrix F
algebraic representation of epipolar geometry
l'x
we will see that mapping is (singular) correlation
(i.e. projective mapping from points to lines)
represented by the fundamental matrix F
![Page 33: Recovering structure from a single view Pinhole ...inst.eecs.berkeley.edu/~ee290t/fa19/lectures/lecture9-1-introduction-to-epipolar... · Pinhole perspective projection Recovering](https://reader033.vdocuments.mx/reader033/viewer/2022041822/5e5ef3cb3dcfb35a5e441af2/html5/thumbnails/33.jpg)
Skew Symmetric Matrix for a vector a
• [a]x is skew symmetric matrix for vector a
• If a = (a1 , a2 , a3 )T then,
[a]x = [ 0 -a3 a2
a3 0 -a1
-a2 a1 0 ]
• Cross product between two vectors a and be can be written in terms
of skew symmetric matrix for a:
axb = [a]x b
![Page 34: Recovering structure from a single view Pinhole ...inst.eecs.berkeley.edu/~ee290t/fa19/lectures/lecture9-1-introduction-to-epipolar... · Pinhole perspective projection Recovering](https://reader033.vdocuments.mx/reader033/viewer/2022041822/5e5ef3cb3dcfb35a5e441af2/html5/thumbnails/34.jpg)
The fundamental matrix Fgeometric derivation
xHx' π
x'e'l' FxxHe' π
mapping from 2-D to 1-D family (rank 2)
• Plane π, not passing through either of the
camera centers
• Ray through C corresponding to image point
x, meets plane π in a point in 3D called X.
• Project X to a point x’ in the second image
• “Transfer via the plane π”.
• l’ is the epipolar line for x x’ must like on l’
• x and x’ are projectively equivalent to the
planar point set Xi
• There is a 2D homography mapping
each xi to x’i
C C’
πH
x'e'l'
![Page 35: Recovering structure from a single view Pinhole ...inst.eecs.berkeley.edu/~ee290t/fa19/lectures/lecture9-1-introduction-to-epipolar... · Pinhole perspective projection Recovering](https://reader033.vdocuments.mx/reader033/viewer/2022041822/5e5ef3cb3dcfb35a5e441af2/html5/thumbnails/35.jpg)
The fundamental matrix F
algebraic derivation
IPP
PP'e'F
xPP'CP'l'
(note: doesn’t work for C=C’ F=0)
xP
λX• Line l’ joints two points: can be written as
cross product of those two points:
• First point is P’C which is e’
• Second point is projection P’ of P+x onto
second image plane
l’ = e’ cross product with ( P’ P+ x )
P+ is pseudo
inverse of P
CC’
![Page 36: Recovering structure from a single view Pinhole ...inst.eecs.berkeley.edu/~ee290t/fa19/lectures/lecture9-1-introduction-to-epipolar... · Pinhole perspective projection Recovering](https://reader033.vdocuments.mx/reader033/viewer/2022041822/5e5ef3cb3dcfb35a5e441af2/html5/thumbnails/36.jpg)
The fundamental matrix F
correspondence condition
0Fxx'T
The fundamental matrix satisfies the condition
that for any pair of corresponding points x↔x’ in
the two images
0l'x'T l’ = FxCombine these two:
• Upshot: A way of characterizing fundamental matrix without reference to
camera matrices, i.e. only in terms of corresponding image points
• How many correspondences are needed find F? at least 7.
![Page 37: Recovering structure from a single view Pinhole ...inst.eecs.berkeley.edu/~ee290t/fa19/lectures/lecture9-1-introduction-to-epipolar... · Pinhole perspective projection Recovering](https://reader033.vdocuments.mx/reader033/viewer/2022041822/5e5ef3cb3dcfb35a5e441af2/html5/thumbnails/37.jpg)
The fundamental matrix F
F is the unique 3x3 rank 2 matrix that
satisfies x’TFx=0 for all x↔x’
(i) Transpose: if F is fundamental matrix for (P,P’), then FT is fundamental
matrix for (P’,P)
(ii) Epipolar lines: for any point x in the first image, the corresponding
epipolar line is l’ = Fx ; same with converse: l = FT x’ represents the
epipolar line corresponding to x’ in the second image
(i) Epipoles: for any point x, the epipolar line l’ = Fx contains the epipole
e’. Thus e’TFx=0, x e’TF=0; similarly Fe=0
e’ is the left null vector of F; e is the right null vector of F
(i) F has 7 d.o.f. , i.e. 3x3-1(homogeneous)-1(rank2)
(ii) F is a correlation, projective mapping from a point x to a line l’=Fx (not a
proper correlation, i.e. not invertible)
If l and l’ are corresponding epipolar lines, then any point x on l is
mapped to the same line l’ no inverse mapping F not proper correlation
![Page 38: Recovering structure from a single view Pinhole ...inst.eecs.berkeley.edu/~ee290t/fa19/lectures/lecture9-1-introduction-to-epipolar... · Pinhole perspective projection Recovering](https://reader033.vdocuments.mx/reader033/viewer/2022041822/5e5ef3cb3dcfb35a5e441af2/html5/thumbnails/38.jpg)
O1 O2
pp’
P
Epipolar Constraint
0pFpT =¢
F = Fundamental Matrix(Faugeras and Luong, 1992)
[ ] 1-´
- ¢××= KRTKF T[Eq. 13]
[Eq. 14]
![Page 39: Recovering structure from a single view Pinhole ...inst.eecs.berkeley.edu/~ee290t/fa19/lectures/lecture9-1-introduction-to-epipolar... · Pinhole perspective projection Recovering](https://reader033.vdocuments.mx/reader033/viewer/2022041822/5e5ef3cb3dcfb35a5e441af2/html5/thumbnails/39.jpg)
Epipolar Constraint
O1 O2
p’
P
p
e e’
• l = F p’ is the epipolar line associated with p’ • l’= FT p is the epipolar line associated with p • F e’ = 0 and FT e = 0• F is 3x3 matrix; 7 DOF • F is singular (rank two)
pT ⋅ F p ' = 0
l’l
![Page 40: Recovering structure from a single view Pinhole ...inst.eecs.berkeley.edu/~ee290t/fa19/lectures/lecture9-1-introduction-to-epipolar... · Pinhole perspective projection Recovering](https://reader033.vdocuments.mx/reader033/viewer/2022041822/5e5ef3cb3dcfb35a5e441af2/html5/thumbnails/40.jpg)
Why F is useful?
- Suppose F is known- No additional information about the scene and camera is given- Given a point on left image, we can compute the corresponding epipolar line in the second image
l’ = FT pp p’
![Page 41: Recovering structure from a single view Pinhole ...inst.eecs.berkeley.edu/~ee290t/fa19/lectures/lecture9-1-introduction-to-epipolar... · Pinhole perspective projection Recovering](https://reader033.vdocuments.mx/reader033/viewer/2022041822/5e5ef3cb3dcfb35a5e441af2/html5/thumbnails/41.jpg)
WhyFisuseful?
• Fcapturesinformationabouttheepipolargeometryof2views+cameraparameters
• MOREIMPORTANTLY: Fgivesconstraintsonhowthescenechangesunderviewpointtransformation(withoutreconstructingthescene!)
• Powerfultoolin:• 3Dreconstruction• Multi-viewobject/scenematching
![Page 42: Recovering structure from a single view Pinhole ...inst.eecs.berkeley.edu/~ee290t/fa19/lectures/lecture9-1-introduction-to-epipolar... · Pinhole perspective projection Recovering](https://reader033.vdocuments.mx/reader033/viewer/2022041822/5e5ef3cb3dcfb35a5e441af2/html5/thumbnails/42.jpg)
O1 O2
P
e’p p’
e
Example: Parallel image planes
E=?K1=K2 = knownHint :
x
y
z
x parallel to O1O2
R = I T = (T, 0, 0)
u
v
![Page 43: Recovering structure from a single view Pinhole ...inst.eecs.berkeley.edu/~ee290t/fa19/lectures/lecture9-1-introduction-to-epipolar... · Pinhole perspective projection Recovering](https://reader033.vdocuments.mx/reader033/viewer/2022041822/5e5ef3cb3dcfb35a5e441af2/html5/thumbnails/43.jpg)
E =
0 −Tz TyTz 0 −Tx−Ty Tx 0
"
#
$$$$
%
&
''''
R =0 0 00 0 −T0 T 0
"
#
$$$
%
&
'''
Essentialmatrixforparallelimages
T = [ T 0 0 ]R = I
[ ] RTE ×= ´
[Eq. 20]
![Page 44: Recovering structure from a single view Pinhole ...inst.eecs.berkeley.edu/~ee290t/fa19/lectures/lecture9-1-introduction-to-epipolar... · Pinhole perspective projection Recovering](https://reader033.vdocuments.mx/reader033/viewer/2022041822/5e5ef3cb3dcfb35a5e441af2/html5/thumbnails/44.jpg)
O1
P
p p’
Example: Parallel image planes
x
y
z
horizontal!What are the directions of epipolar lines?
l = E p ' =0 0 00 0 −T0 T 0
"
#
$$$
%
&
'''
u 'v '1
"
#
$$$
%
&
'''=
0−TT v '
"
#
$$$
%
&
'''
l l’
u
v
e’e
u’
v’
![Page 45: Recovering structure from a single view Pinhole ...inst.eecs.berkeley.edu/~ee290t/fa19/lectures/lecture9-1-introduction-to-epipolar... · Pinhole perspective projection Recovering](https://reader033.vdocuments.mx/reader033/viewer/2022041822/5e5ef3cb3dcfb35a5e441af2/html5/thumbnails/45.jpg)
Example: Parallel image planes
How are pand p’ related?
pT ⋅ E p ' = 0
P
e’p p’
e
y
z
xO1
u
v
u’
v’
![Page 46: Recovering structure from a single view Pinhole ...inst.eecs.berkeley.edu/~ee290t/fa19/lectures/lecture9-1-introduction-to-epipolar... · Pinhole perspective projection Recovering](https://reader033.vdocuments.mx/reader033/viewer/2022041822/5e5ef3cb3dcfb35a5e441af2/html5/thumbnails/46.jpg)
Example: Parallel image planes
( ) ( )T
0 0 0 0p E p 0 1 0 0 0 1 0
0 0 1
uu v T v u v T Tv Tv
T Tv
¢é ù æ ö æ öç ÷ ç ÷ê ú¢ ¢ ¢= Þ - = Þ - = Þ =ç ÷ ç ÷ê úç ÷ ç ÷¢ê úë û è ø è ø
How are p and p’ related?
P
e’p p’
e
y
z
O1x
⇒ v = "v
u
v
u’
v’
![Page 47: Recovering structure from a single view Pinhole ...inst.eecs.berkeley.edu/~ee290t/fa19/lectures/lecture9-1-introduction-to-epipolar... · Pinhole perspective projection Recovering](https://reader033.vdocuments.mx/reader033/viewer/2022041822/5e5ef3cb3dcfb35a5e441af2/html5/thumbnails/47.jpg)
Example: Parallel image planes
Rectification: making two images “parallel” Why it is useful? • Epipolar constraint ® v = v’
• New views can be synthesized by linear interpolation
P
e’p p’
e
y
z
O1x
u
v
u’
v’
![Page 48: Recovering structure from a single view Pinhole ...inst.eecs.berkeley.edu/~ee290t/fa19/lectures/lecture9-1-introduction-to-epipolar... · Pinhole perspective projection Recovering](https://reader033.vdocuments.mx/reader033/viewer/2022041822/5e5ef3cb3dcfb35a5e441af2/html5/thumbnails/48.jpg)
Application: view morphingS. M. Seitz and C. R. Dyer, Proc. SIGGRAPH 96, 1996, 21-30