computer vision the 2d projective plane and it’s applications hz ch 2. in particular: ch 2.1-4,...
TRANSCRIPT
![Page 1: Computer Vision The 2D projective plane and it’s applications HZ Ch 2. In particular: Ch 2.1-4, 2.7, Szelisky: Ch 2.1.1, 2.1.2 Estimation:HZ: Ch 4.1-4.2.5,](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e8a5503460f94b8f917/html5/thumbnails/1.jpg)
Computer Vision
The 2D projective plane and The 2D projective plane and it’s applications it’s applications
HZ Ch 2. In particular: Ch 2.1-4, 2.7,
Szelisky: Ch 2.1.1, 2.1.2Estimation:HZ: Ch 4.1-4.2.5, 4.4.4-4.8 cursorly
Richard Hartley and Andrew Zisserman, Multiple View Geometry, Cambridge University Publishers, 2nd ed. 2004
![Page 2: Computer Vision The 2D projective plane and it’s applications HZ Ch 2. In particular: Ch 2.1-4, 2.7, Szelisky: Ch 2.1.1, 2.1.2 Estimation:HZ: Ch 4.1-4.2.5,](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e8a5503460f94b8f917/html5/thumbnails/2.jpg)
Homogeneous coordinates
0=++ cbyax ( ) ( ) 0=1x,y,a,b,c T
( ) ( ) 0≠∀,~ ka,b,cka,b,c TT
Homogeneous representation of 2D points and lines
equivalence class of vectors, any vector is representative
Set of all equivalence classes in R3(0,0,0)T forms P2
( ) ( ) 0≠∀,1,,~1,, kyxkyx TT
The point x lies on the line l if and only if
Homogeneous coordinates
Inhomogeneous coordinates ( )Tyx,
( )T321 ,, xxx but only 2DOF
Note that scale is unimportant for incidence relation
0=xlT
![Page 3: Computer Vision The 2D projective plane and it’s applications HZ Ch 2. In particular: Ch 2.1-4, 2.7, Szelisky: Ch 2.1.1, 2.1.2 Estimation:HZ: Ch 4.1-4.2.5,](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e8a5503460f94b8f917/html5/thumbnails/3.jpg)
X
Y
Z
Homogeneous coordinates
X
Y
Z
s
s 0
X∞
X
Y
0
X∞ =
• Perspective imaging models 2d projective space
• Each 3D ray is a point in P2 : homogeneous coords.
• Ideal points
• P2 is R2 plus a “line at infinity” l∞
The 2D projective plane
Projective point
l∞
x
yX
Y1Z= Inhomogeneou
s equivalent
![Page 4: Computer Vision The 2D projective plane and it’s applications HZ Ch 2. In particular: Ch 2.1-4, 2.7, Szelisky: Ch 2.1.1, 2.1.2 Estimation:HZ: Ch 4.1-4.2.5,](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e8a5503460f94b8f917/html5/thumbnails/4.jpg)
Lines
HZ • Ideal line ~ the plane parallel to the image
A
B
C
l =X=lTX = XTl = AX + BY + CZ = 0
l∞ =
0
0
1
• Projective line ~ a plane through the origin
For any 2d projective property, a dual property holds when the role of points and lines are interchanged.
Duality:
X
Y
0
X∞ =
l1 l2X =X1 X2=l
The line joining two points The point joining two lines
X
Y
Z
X
l“line at infinity
”
![Page 5: Computer Vision The 2D projective plane and it’s applications HZ Ch 2. In particular: Ch 2.1-4, 2.7, Szelisky: Ch 2.1.1, 2.1.2 Estimation:HZ: Ch 4.1-4.2.5,](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e8a5503460f94b8f917/html5/thumbnails/5.jpg)
Points from lines and vice-versa
l'×l=x
Intersections of lines
The intersection of two lines and is l l'
Line joining two points
The line through two points and is x'×x=lx x'
Example
1=x
1=y
1
)1-,1,0( y
x
1
)1-,0,1( y
x
Note:[ ] 'xx='x×x ×
with
0-
0-
-0
x
xy
xz
yz
![Page 6: Computer Vision The 2D projective plane and it’s applications HZ Ch 2. In particular: Ch 2.1-4, 2.7, Szelisky: Ch 2.1.1, 2.1.2 Estimation:HZ: Ch 4.1-4.2.5,](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e8a5503460f94b8f917/html5/thumbnails/6.jpg)
Ideal points and the line at infinity
T0,,l'l ab
Intersections of parallel lines
( ) ( )TTand ',,=l' ,,=l cbacba
Example
1=x 2=x
Ideal points ( )T0,, 21 xx
Line at infinity ( )T1,0,0=l∞
∞22 l∪= RP
tangent vector
normal direction
-ab,( )ba,
Note that in P2 there is no distinction
between ideal points and others
![Page 7: Computer Vision The 2D projective plane and it’s applications HZ Ch 2. In particular: Ch 2.1-4, 2.7, Szelisky: Ch 2.1.1, 2.1.2 Estimation:HZ: Ch 4.1-4.2.5,](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e8a5503460f94b8f917/html5/thumbnails/7.jpg)
Conics
Curve described by 2nd-degree equation in the plane
0=+++++ 22 feydxcybxyax
0=+++++ 233231
2221
21 fxxexxdxcxxbxax
3
2
3
1 , xxyx
xx or homogenized
0=xx CT
or in matrix form
fed
ecb
dba
2/2/
2/2/
2/2/
Cwith
{ }fedcba :::::5DOF:
![Page 8: Computer Vision The 2D projective plane and it’s applications HZ Ch 2. In particular: Ch 2.1-4, 2.7, Szelisky: Ch 2.1.1, 2.1.2 Estimation:HZ: Ch 4.1-4.2.5,](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e8a5503460f94b8f917/html5/thumbnails/8.jpg)
Five points define a conic
For each point the conic passes through
0=+++++ 22 feydxcyybxax iiiiii
or
( ) 0=.1,,,,, 22 ciiiiii yxyyxx ( )Tfedcba ,,,,,=c
0
1
1
1
1
1
552555
25
442444
24
332333
23
222222
22
112111
21
c
yxyyxx
yxyyxx
yxyyxx
yxyyxx
yxyyxx
stacking constraints yields
![Page 9: Computer Vision The 2D projective plane and it’s applications HZ Ch 2. In particular: Ch 2.1-4, 2.7, Szelisky: Ch 2.1.1, 2.1.2 Estimation:HZ: Ch 4.1-4.2.5,](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e8a5503460f94b8f917/html5/thumbnails/9.jpg)
Tangent lines to conics
The line l tangent to C at point x on C is given by l=Cx
lx
C
![Page 10: Computer Vision The 2D projective plane and it’s applications HZ Ch 2. In particular: Ch 2.1-4, 2.7, Szelisky: Ch 2.1.1, 2.1.2 Estimation:HZ: Ch 4.1-4.2.5,](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e8a5503460f94b8f917/html5/thumbnails/10.jpg)
Dual conics
0=ll *CTA line tangent to the conic C satisfies
Dual conics = line conics = conic envelopes
1-* CC In general (C full rank):
![Page 11: Computer Vision The 2D projective plane and it’s applications HZ Ch 2. In particular: Ch 2.1-4, 2.7, Szelisky: Ch 2.1.1, 2.1.2 Estimation:HZ: Ch 4.1-4.2.5,](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e8a5503460f94b8f917/html5/thumbnails/11.jpg)
Degenerate conics
A conic is degenerate if matrix C is not of full rank
TT ml+lm=C
e.g. two lines (rank 2)
e.g. repeated line (rank 1)
Tll=C
l
l
m
Degenerate line conics: 2 points (rank 2), double point (rank1)
( ) CC ≠**Note that for degenerate conics
![Page 12: Computer Vision The 2D projective plane and it’s applications HZ Ch 2. In particular: Ch 2.1-4, 2.7, Szelisky: Ch 2.1.1, 2.1.2 Estimation:HZ: Ch 4.1-4.2.5,](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e8a5503460f94b8f917/html5/thumbnails/12.jpg)
Conics
•Conic: Conic: – Euclidean geometry: hyperbola, ellipse, parabola & degenerateEuclidean geometry: hyperbola, ellipse, parabola & degenerate– Projective geometry: equivalent under projective transformProjective geometry: equivalent under projective transform– Defined by 5 pointsDefined by 5 points
•Tangent lineTangent line
•Dual conic C*Dual conic C*
0
022
xx C
feydxcybxyaxT
fed
ecb
dba
C
2/2/
2/2/
2/2/
xl C
0* ll CT
inhomogeneous
homogeneous
![Page 13: Computer Vision The 2D projective plane and it’s applications HZ Ch 2. In particular: Ch 2.1-4, 2.7, Szelisky: Ch 2.1.1, 2.1.2 Estimation:HZ: Ch 4.1-4.2.5,](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e8a5503460f94b8f917/html5/thumbnails/13.jpg)
Projective transformations
• Homographies, collineations, projectivitiesHomographies, collineations, projectivities
• 3x3 nonsingular H3x3 nonsingular H
3
2
1
333231
232221
131211
3
2
1
'
'
'
x
x
x
hhh
hhh
hhh
x
x
x
l0= Hà Tl
xTl = 0 x0Tl0= 0
maps P2 to P2
8 degrees of freedomdetermined by 4 corresponding points
x0= Hx• Transforming Lines?subspaces preserved
xTHTl0= 0substitution
dual transformation
![Page 14: Computer Vision The 2D projective plane and it’s applications HZ Ch 2. In particular: Ch 2.1-4, 2.7, Szelisky: Ch 2.1.1, 2.1.2 Estimation:HZ: Ch 4.1-4.2.5,](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e8a5503460f94b8f917/html5/thumbnails/14.jpg)
Planar Projective Warping
HZA novel view rendered via
four points with known structurexi
0= Hxii = 1. . .4
xi xi0
![Page 15: Computer Vision The 2D projective plane and it’s applications HZ Ch 2. In particular: Ch 2.1-4, 2.7, Szelisky: Ch 2.1.1, 2.1.2 Estimation:HZ: Ch 4.1-4.2.5,](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e8a5503460f94b8f917/html5/thumbnails/15.jpg)
Planar Projective Warping
HZOriginal Top-down Facing right
Artifacts are apparent where planarity is violated...
![Page 16: Computer Vision The 2D projective plane and it’s applications HZ Ch 2. In particular: Ch 2.1-4, 2.7, Szelisky: Ch 2.1.1, 2.1.2 Estimation:HZ: Ch 4.1-4.2.5,](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e8a5503460f94b8f917/html5/thumbnails/16.jpg)
2d Homographies
2 images of a plane
2 images from the same viewpoint (Perspectivity)
![Page 17: Computer Vision The 2D projective plane and it’s applications HZ Ch 2. In particular: Ch 2.1-4, 2.7, Szelisky: Ch 2.1.1, 2.1.2 Estimation:HZ: Ch 4.1-4.2.5,](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e8a5503460f94b8f917/html5/thumbnails/17.jpg)
Panoramic imagingAppl: Quicktime VR, robot navigation etc.
Homographies of the world, unite!
![Page 18: Computer Vision The 2D projective plane and it’s applications HZ Ch 2. In particular: Ch 2.1-4, 2.7, Szelisky: Ch 2.1.1, 2.1.2 Estimation:HZ: Ch 4.1-4.2.5,](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e8a5503460f94b8f917/html5/thumbnails/18.jpg)
Image mosaics are stitched by homographies
HZ
![Page 19: Computer Vision The 2D projective plane and it’s applications HZ Ch 2. In particular: Ch 2.1-4, 2.7, Szelisky: Ch 2.1.1, 2.1.2 Estimation:HZ: Ch 4.1-4.2.5,](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e8a5503460f94b8f917/html5/thumbnails/19.jpg)
The line at infinity
l
1
0
0
1t
0ll
TT
TT
A
AH A
The line at infinity l is a fixed line under a projective transformation H if and only if H is an affinity
Note: But points on l can be rearranged to new points on l
![Page 20: Computer Vision The 2D projective plane and it’s applications HZ Ch 2. In particular: Ch 2.1-4, 2.7, Szelisky: Ch 2.1.1, 2.1.2 Estimation:HZ: Ch 4.1-4.2.5,](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e8a5503460f94b8f917/html5/thumbnails/20.jpg)
Affine properties from images
Projection(Imaging)
RectificationPost-processing
![Page 21: Computer Vision The 2D projective plane and it’s applications HZ Ch 2. In particular: Ch 2.1-4, 2.7, Szelisky: Ch 2.1.1, 2.1.2 Estimation:HZ: Ch 4.1-4.2.5,](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e8a5503460f94b8f917/html5/thumbnails/21.jpg)
Affine rectification
v1 v2
l1
l2 l4
l3
l∞21 vvl
211 llv 432 llv
APA
lll
HH
321
010
001
0,l 3321 llll T
Point transformation for Aff Rect:
Exercise: Verify
T321 lll
TT ]1,0,0[321 lllPAH
![Page 22: Computer Vision The 2D projective plane and it’s applications HZ Ch 2. In particular: Ch 2.1-4, 2.7, Szelisky: Ch 2.1.1, 2.1.2 Estimation:HZ: Ch 4.1-4.2.5,](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e8a5503460f94b8f917/html5/thumbnails/22.jpg)
GroupGroup TransformationTransformation InvariantsInvariants DistortionDistortion
ProjectiveProjective
8 DOF8 DOF
• Cross ratioCross ratio
• IntersectionIntersection
• TangencyTangency
AffineAffine
6 DOF6 DOF
• ParallelismParallelism
• Relative dist in 1dRelative dist in 1d
• Line at infinityLine at infinity
MetricMetric
4 DOF4 DOF
• Relative distancesRelative distances
• AnglesAngles
• Dual conicDual conic
EuclideanEuclidean
3 DOF3 DOF
• LengthsLengths
• AreasAreas
Geometric strata: 2d overview
1TS
sRH
O
t
1TA
AH
O
t
1TE
RH
O
t
v
AH TP v
t
C*
2 dofl
2 dof
l
C*
![Page 23: Computer Vision The 2D projective plane and it’s applications HZ Ch 2. In particular: Ch 2.1-4, 2.7, Szelisky: Ch 2.1.1, 2.1.2 Estimation:HZ: Ch 4.1-4.2.5,](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e8a5503460f94b8f917/html5/thumbnails/23.jpg)
Parameter estimation in geometric transforms
•2D homography2D homographyGiven a set of (xGiven a set of (xii,x,xii’), compute H (x’), compute H (xii’=Hx’=Hxii))
•3D to 2D camera projection3D to 2D camera projectionGiven a set of (XGiven a set of (Xii,x,xii), compute P (x), compute P (xii=PX=PXii))
•Fundamental matrixFundamental matrixGiven a set of (xGiven a set of (xii,x,xii’), compute F ’), compute F (x(xii’’TTFxFxii=0)=0)
•Trifocal tensor Trifocal tensor Given a set of (xGiven a set of (xii,x,xii’,x’,xii”), compute T”), compute T
cs499
Useful in
Grad research
![Page 24: Computer Vision The 2D projective plane and it’s applications HZ Ch 2. In particular: Ch 2.1-4, 2.7, Szelisky: Ch 2.1.1, 2.1.2 Estimation:HZ: Ch 4.1-4.2.5,](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e8a5503460f94b8f917/html5/thumbnails/24.jpg)
Math tools 1:Solving Linear Systems
• If If mm == nn ( (AA is a square matrix), then we can obtain is a square matrix), then we can obtain the solution by simple inversion:the solution by simple inversion:
• If If mm >> nn, then the system is , then the system is over-constrainedover-constrained and and AA is not invertible is not invertible – Use Matlab “\” to obtain Use Matlab “\” to obtain least-squares solutionleast-squares solution xx == A\b A\b to to Ax Ax
=b =b internally Matlab uses QR-factorization (cmput418/340) to solve this.internally Matlab uses QR-factorization (cmput418/340) to solve this. – Can also write this using pseudoinverse Can also write this using pseudoinverse AA+ + == ((AATTAA))-1-1AATT to obtain to obtain
least-squares solutionleast-squares solution xx == A A++bb
![Page 25: Computer Vision The 2D projective plane and it’s applications HZ Ch 2. In particular: Ch 2.1-4, 2.7, Szelisky: Ch 2.1.1, 2.1.2 Estimation:HZ: Ch 4.1-4.2.5,](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e8a5503460f94b8f917/html5/thumbnails/25.jpg)
Fitting Lines
• A 2-D point A 2-D point xx = ( = (xx, , yy)) is on a line with slope is on a line with slope
mm and intercept and intercept bb if and only if if and only if yy = = mxmx + + bb • Equivalently,Equivalently,
• So the line defined by two points So the line defined by two points xx11, , xx22 is the is the
solution to the following system of equations:solution to the following system of equations:
![Page 26: Computer Vision The 2D projective plane and it’s applications HZ Ch 2. In particular: Ch 2.1-4, 2.7, Szelisky: Ch 2.1.1, 2.1.2 Estimation:HZ: Ch 4.1-4.2.5,](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e8a5503460f94b8f917/html5/thumbnails/26.jpg)
Fitting Lines
• With more than two points, there is no guarantee With more than two points, there is no guarantee that they will all be on the same linethat they will all be on the same line
• Least-squares solution obtained from Least-squares solution obtained from pseudoinverse is line that is “closest” to all of the pseudoinverse is line that is “closest” to all of the pointspoints
courtesy ofVanderbilt U.
![Page 27: Computer Vision The 2D projective plane and it’s applications HZ Ch 2. In particular: Ch 2.1-4, 2.7, Szelisky: Ch 2.1.1, 2.1.2 Estimation:HZ: Ch 4.1-4.2.5,](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e8a5503460f94b8f917/html5/thumbnails/27.jpg)
Example: Fitting a Line
• Suppose we have points Suppose we have points (2, 1)(2, 1), , (5, 2)(5, 2), , (7, (7, 3)3), and , and (8, 3)(8, 3)
• Then Then
and and x x == A A++b b == (0.3571, 0.2857)(0.3571, 0.2857)TT
Matlab: Matlab: x x == A\b A\b
![Page 28: Computer Vision The 2D projective plane and it’s applications HZ Ch 2. In particular: Ch 2.1-4, 2.7, Szelisky: Ch 2.1.1, 2.1.2 Estimation:HZ: Ch 4.1-4.2.5,](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e8a5503460f94b8f917/html5/thumbnails/28.jpg)
Example: Fitting a Line
![Page 29: Computer Vision The 2D projective plane and it’s applications HZ Ch 2. In particular: Ch 2.1-4, 2.7, Szelisky: Ch 2.1.1, 2.1.2 Estimation:HZ: Ch 4.1-4.2.5,](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e8a5503460f94b8f917/html5/thumbnails/29.jpg)
Homogeneous Systems of Equations
• Suppose we want to solve Suppose we want to solve AA x = 0x = 0• There is a trivial solution There is a trivial solution x = 0x = 0, but we don’t want , but we don’t want
this. For what other values of this. For what other values of xx is is AA xx close to close to 00??• This is satisfied by computing the This is satisfied by computing the singular value singular value
decompositiondecomposition (SVD) (SVD) A A == UDV UDVTT (a non- (a non-negative diagonal matrix between two orthogonal negative diagonal matrix between two orthogonal matrices) and taking matrices) and taking xx as the last column of as the last column of VV– Note that Matlab returns Note that Matlab returns [U, D, V] = svd(A)[U, D, V] = svd(A)
![Page 30: Computer Vision The 2D projective plane and it’s applications HZ Ch 2. In particular: Ch 2.1-4, 2.7, Szelisky: Ch 2.1.1, 2.1.2 Estimation:HZ: Ch 4.1-4.2.5,](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e8a5503460f94b8f917/html5/thumbnails/30.jpg)
Line-Fitting as a Homogeneous System
• A 2-D homogeneous point A 2-D homogeneous point xx = ( = (xx, , yy, 1), 1)TT is on is on
the line the line ll = ( = (aa, , bb, , cc))TT only when only when
axax + + byby + + cc = 0 = 0• We can write this equation with a dot product: We can write this equation with a dot product:
x•x• ll = 0 = 0,, and hence the following system is and hence the following system is
implied for multiple points implied for multiple points xx11,, x x22, ..., , ..., xxnn::
![Page 31: Computer Vision The 2D projective plane and it’s applications HZ Ch 2. In particular: Ch 2.1-4, 2.7, Szelisky: Ch 2.1.1, 2.1.2 Estimation:HZ: Ch 4.1-4.2.5,](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e8a5503460f94b8f917/html5/thumbnails/31.jpg)
Example: Homogeneous Line-Fitting
• Again we have 4 points, but now in homogeneous form: Again we have 4 points, but now in homogeneous form:
(2, 1, 1)(2, 1, 1), , (5, 2, 1)(5, 2, 1), , (7, 3, 1)(7, 3, 1), and , and (8, 3, 1)(8, 3, 1)• Our system is:Our system is:
• Taking the SVD of Taking the SVD of AA, we get:, we get:compare to x = (0.3571, 0.2857)T
![Page 32: Computer Vision The 2D projective plane and it’s applications HZ Ch 2. In particular: Ch 2.1-4, 2.7, Szelisky: Ch 2.1.1, 2.1.2 Estimation:HZ: Ch 4.1-4.2.5,](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e8a5503460f94b8f917/html5/thumbnails/32.jpg)
Parameter estimation in geometric transforms
•2D homography2D homographyGiven a set of (xGiven a set of (xii,x,xii’), compute H (x’), compute H (xii’=Hx’=Hxii))
•3D to 2D camera projection3D to 2D camera projectionGiven a set of (XGiven a set of (Xii,x,xii), compute P (x), compute P (xii=PX=PXii))
•Fundamental matrixFundamental matrixGiven a set of (xGiven a set of (xii,x,xii’), compute F ’), compute F (x(xii’’TTFxFxii=0)=0)
•Trifocal tensor Trifocal tensor Given a set of (xGiven a set of (xii,x,xii’,x’,xii”), compute T”), compute T
cs499
Useful in
Grad research
![Page 33: Computer Vision The 2D projective plane and it’s applications HZ Ch 2. In particular: Ch 2.1-4, 2.7, Szelisky: Ch 2.1.1, 2.1.2 Estimation:HZ: Ch 4.1-4.2.5,](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e8a5503460f94b8f917/html5/thumbnails/33.jpg)
Estimating Homography Hgiven image points x
HZA novel view rendered via
four points with known structurexi
0= Hxii = 1. . .4
xi xi0
![Page 34: Computer Vision The 2D projective plane and it’s applications HZ Ch 2. In particular: Ch 2.1-4, 2.7, Szelisky: Ch 2.1.1, 2.1.2 Estimation:HZ: Ch 4.1-4.2.5,](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e8a5503460f94b8f917/html5/thumbnails/34.jpg)
Number of measurements required
• At least as many independent equations as degrees of At least as many independent equations as degrees of freedom requiredfreedom required
• Example: Example:
Hxx'
1'
λ
333231
232221
131211
y
x
hhh
hhh
hhh
w
y
x
2 independent equations / point8 degrees of freedom
4x2≥8
![Page 35: Computer Vision The 2D projective plane and it’s applications HZ Ch 2. In particular: Ch 2.1-4, 2.7, Szelisky: Ch 2.1.1, 2.1.2 Estimation:HZ: Ch 4.1-4.2.5,](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e8a5503460f94b8f917/html5/thumbnails/35.jpg)
Approximate solutions
•Minimal solutionMinimal solution4 points yield an exact solution for H4 points yield an exact solution for H
•More pointsMore points– No exact solution, because measurements are inexact No exact solution, because measurements are inexact
(“noise”)(“noise”)– Search for “best” according to some cost functionSearch for “best” according to some cost function– Algebraic or geometric/statistical costAlgebraic or geometric/statistical cost
![Page 36: Computer Vision The 2D projective plane and it’s applications HZ Ch 2. In particular: Ch 2.1-4, 2.7, Szelisky: Ch 2.1.1, 2.1.2 Estimation:HZ: Ch 4.1-4.2.5,](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e8a5503460f94b8f917/html5/thumbnails/36.jpg)
Many ways to solve:
Different Cost functions => differencesDifferent Cost functions => differencesin solution in solution
•Algebraic distanceAlgebraic distance
•Geometric distanceGeometric distance
•Reprojection errorReprojection error
•ComparisonComparison
•Geometric interpretationGeometric interpretation
![Page 37: Computer Vision The 2D projective plane and it’s applications HZ Ch 2. In particular: Ch 2.1-4, 2.7, Szelisky: Ch 2.1.1, 2.1.2 Estimation:HZ: Ch 4.1-4.2.5,](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e8a5503460f94b8f917/html5/thumbnails/37.jpg)
Gold Standard algorithm
•Cost function that is optimal for some Cost function that is optimal for some assumptionsassumptions
•Computational algorithm that minimizes it is Computational algorithm that minimizes it is called “Gold Standard” algorithm called “Gold Standard” algorithm
•Other algorithms can then be compared to itOther algorithms can then be compared to it
![Page 38: Computer Vision The 2D projective plane and it’s applications HZ Ch 2. In particular: Ch 2.1-4, 2.7, Szelisky: Ch 2.1.1, 2.1.2 Estimation:HZ: Ch 4.1-4.2.5,](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e8a5503460f94b8f917/html5/thumbnails/38.jpg)
Estimating H: The Direct Linear Transformation (DLT) Algorithm
• xxi i =H=HXXii is an equation involving homogeneous is an equation involving homogeneous
vectors, so vectors, so HHXXii and and xxi i need only be in the same need only be in the same
direction, not strictly equaldirection, not strictly equal
• We can specify “same directionality” by using a We can specify “same directionality” by using a cross product formulation:cross product formulation:
0Hxx ii
![Page 39: Computer Vision The 2D projective plane and it’s applications HZ Ch 2. In particular: Ch 2.1-4, 2.7, Szelisky: Ch 2.1.1, 2.1.2 Estimation:HZ: Ch 4.1-4.2.5,](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e8a5503460f94b8f917/html5/thumbnails/39.jpg)
Direct Linear Transformation(DLT)
ii Hxx 0Hxx ii
i
i
i
i
xh
xh
xh
Hx3
2
1
T
T
T
iiii
iiii
iiii
ii
yx
xw
wy
xhxh
xhxh
xhxh
Hxx12
31
23
TT
TT
TT
0
h
h
h
0xx
x0x
xx0
3
2
1
TTT
TTT
TTT
iiii
iiii
iiii
xy
xw
yw
Tiiii wyx ,,x
0hA i
![Page 40: Computer Vision The 2D projective plane and it’s applications HZ Ch 2. In particular: Ch 2.1-4, 2.7, Szelisky: Ch 2.1.1, 2.1.2 Estimation:HZ: Ch 4.1-4.2.5,](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e8a5503460f94b8f917/html5/thumbnails/40.jpg)
Direct Linear Transformation(DLT)
• Equations are linear in Equations are linear in hh
0
h
h
h
0xx
x0x
xx0
3
2
1
TTT
TTT
TTT
iiii
iiii
iiii
xy
xw
yw
0AAA 321 iiiiii wyx
0hA i
• Only 2 out of 3 are linearly independent Only 2 out of 3 are linearly independent
(indeed, 2 eq/pt)(indeed, 2 eq/pt)
0
h
h
h
x0x
xx0
3
2
1
TTT
TTT
iiii
iiii
xw
yw
(only drop third row if wi’≠0)• Holds for any homogeneous representation, Holds for any homogeneous representation,
e.g. (e.g. (xxii’,’,yyii’,1)’,1)
![Page 41: Computer Vision The 2D projective plane and it’s applications HZ Ch 2. In particular: Ch 2.1-4, 2.7, Szelisky: Ch 2.1.1, 2.1.2 Estimation:HZ: Ch 4.1-4.2.5,](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e8a5503460f94b8f917/html5/thumbnails/41.jpg)
Direct Linear Transformation(DLT)
•Solving for Solving for HH
0Ah 0h
A
A
A
A
4
3
2
1
size A is 8x9 or 12x9, but rank 8
Trivial solution is h=09T is not interesting
1-D null-space yields solution of interestpick for example the one with 1h
![Page 42: Computer Vision The 2D projective plane and it’s applications HZ Ch 2. In particular: Ch 2.1-4, 2.7, Szelisky: Ch 2.1.1, 2.1.2 Estimation:HZ: Ch 4.1-4.2.5,](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e8a5503460f94b8f917/html5/thumbnails/42.jpg)
Direct Linear Transformation(DLT)
•Over-determined solutionOver-determined solution
No exact solution because of inexact measurementi.e. “noise”
0Ah 0h
A
A
A
n
2
1
Find approximate solution- Additional constraint needed to avoid 0, e.g.
- not possible, so minimize
1h Ah0Ah
![Page 43: Computer Vision The 2D projective plane and it’s applications HZ Ch 2. In particular: Ch 2.1-4, 2.7, Szelisky: Ch 2.1.1, 2.1.2 Estimation:HZ: Ch 4.1-4.2.5,](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e8a5503460f94b8f917/html5/thumbnails/43.jpg)
DLT algorithm
ObjectiveGiven n≥4 2D to 2D point correspondences {xi↔xi’}, determine the 2D homography matrix H such that xi’=Hxi
Algorithm
(i) For each correspondence xi ↔xi’ compute Ai. Usually only two first rows needed.
(ii) Assemble n 2x9 matrices Ai into a single 2nx9 matrix A
(iii) Obtain SVD of A. Solution for h is last column of V
(iv) Determine H from h (reshape)
![Page 44: Computer Vision The 2D projective plane and it’s applications HZ Ch 2. In particular: Ch 2.1-4, 2.7, Szelisky: Ch 2.1.1, 2.1.2 Estimation:HZ: Ch 4.1-4.2.5,](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e8a5503460f94b8f917/html5/thumbnails/44.jpg)
Inhomogeneous solution
'
'h~
''000'''
'''''000
ii
ii
iiiiiiiiii
iiiiiiiiii
xw
yw
xyxxwwwywx
yyyxwwwywx
Since h can only be computed up to scale, pick hj=1, e.g. h9=1, and solve for 8-vector
h~
Solve using Gaussian elimination (4 points) or using linear least-squares (more than 4 points)
However, if h9=0 this approach fails also poor results if h9 close to zeroTherefore, not recommended for general homographiesNote h9=H33=0 if origin is mapped to infinity
0
1
0
0
H100Hxl 0
T
![Page 45: Computer Vision The 2D projective plane and it’s applications HZ Ch 2. In particular: Ch 2.1-4, 2.7, Szelisky: Ch 2.1.1, 2.1.2 Estimation:HZ: Ch 4.1-4.2.5,](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e8a5503460f94b8f917/html5/thumbnails/45.jpg)
Normalizing transformations
• Since DLT is not invariant to coordinate Since DLT is not invariant to coordinate transforms, what is a good choice of transforms, what is a good choice of coordinates?coordinates?e.g.e.g.– Translate centroid to originTranslate centroid to origin– Scale to a average distance to the originScale to a average distance to the origin– Independently on both imagesIndependently on both images
2
1
norm
100
2/0
2/0
T
hhw
whwOr
![Page 46: Computer Vision The 2D projective plane and it’s applications HZ Ch 2. In particular: Ch 2.1-4, 2.7, Szelisky: Ch 2.1.1, 2.1.2 Estimation:HZ: Ch 4.1-4.2.5,](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e8a5503460f94b8f917/html5/thumbnails/46.jpg)
Normalized DLT algorithm
ObjectiveGiven n≥4 2D to 2D point correspondences {xi↔xi’}, determine the 2D homography matrix H such that xi’=Hxi
Algorithm
(i) Normalize points
(ii) Apply DLT algorithm to
(iii) Denormalize solution
,x~x~ ii inormiinormi xTx~,xTx~
norm-1
norm TH~
TH
![Page 47: Computer Vision The 2D projective plane and it’s applications HZ Ch 2. In particular: Ch 2.1-4, 2.7, Szelisky: Ch 2.1.1, 2.1.2 Estimation:HZ: Ch 4.1-4.2.5,](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e8a5503460f94b8f917/html5/thumbnails/47.jpg)
Importance of normalization
0
h
h
h
0001
1000
3
2
1
iiiiiii
iiiiiii
xyxxxyx
yyyxyyx
~102 ~102 ~102 ~102 ~104 ~104 ~10211
orders of magnitude difference!
![Page 48: Computer Vision The 2D projective plane and it’s applications HZ Ch 2. In particular: Ch 2.1-4, 2.7, Szelisky: Ch 2.1.1, 2.1.2 Estimation:HZ: Ch 4.1-4.2.5,](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e8a5503460f94b8f917/html5/thumbnails/48.jpg)
Degenerate configurations
x4
x1
x3
x2
x4
x1
x3
x2H? H’?
x1
x3
x2
x4
0Hxx iiConstraints: i=1,2,3,4
TlxH 4* Define:
4444* xxlxxH kT
3,2,1 ,0xlxxH 4* iii
TThen,
H* is rank-1 matrix and thus not a homography
(case A) (case B)
If H* is unique solution, then no homography mapping xi→xi’(case B)
If further solution H exist, then also αH*+βH (case A) (2-D null-space in stead of 1-D null-space)
![Page 49: Computer Vision The 2D projective plane and it’s applications HZ Ch 2. In particular: Ch 2.1-4, 2.7, Szelisky: Ch 2.1.1, 2.1.2 Estimation:HZ: Ch 4.1-4.2.5,](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e8a5503460f94b8f917/html5/thumbnails/49.jpg)
•First 3D proj geomFirst 3D proj geom
•Then review and more on camera modelsThen review and more on camera models
•Then following P estimationThen following P estimation
![Page 50: Computer Vision The 2D projective plane and it’s applications HZ Ch 2. In particular: Ch 2.1-4, 2.7, Szelisky: Ch 2.1.1, 2.1.2 Estimation:HZ: Ch 4.1-4.2.5,](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e8a5503460f94b8f917/html5/thumbnails/50.jpg)
• Projection equationProjection equation
xxii=P=PiiXX
• Resection:Resection:– xxii,X P,X Pii
A 3D Vision Problem: Multi-view geometry - resection
Given image points and 3D points calculate camera projection matrix.
![Page 51: Computer Vision The 2D projective plane and it’s applications HZ Ch 2. In particular: Ch 2.1-4, 2.7, Szelisky: Ch 2.1.1, 2.1.2 Estimation:HZ: Ch 4.1-4.2.5,](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e8a5503460f94b8f917/html5/thumbnails/51.jpg)
Estimating camera matrix P
•Given a number of correspondences between 3-Given a number of correspondences between 3-
D points and their 2-D image projections D points and their 2-D image projections XXii x xii, we would like to determine the , we would like to determine the camera camera
projection matrixprojection matrix PP such that such that xxi i == PXPXii for for
all all ii
![Page 52: Computer Vision The 2D projective plane and it’s applications HZ Ch 2. In particular: Ch 2.1-4, 2.7, Szelisky: Ch 2.1.1, 2.1.2 Estimation:HZ: Ch 4.1-4.2.5,](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e8a5503460f94b8f917/html5/thumbnails/52.jpg)
A Calibration Target
courtesy of B. Wilburn
XZ
Y
Xi
xi
![Page 53: Computer Vision The 2D projective plane and it’s applications HZ Ch 2. In particular: Ch 2.1-4, 2.7, Szelisky: Ch 2.1.1, 2.1.2 Estimation:HZ: Ch 4.1-4.2.5,](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e8a5503460f94b8f917/html5/thumbnails/53.jpg)
Estimating P: The Direct Linear Transformation (DLT) Algorithm
• xxi i == PXPXii is an equation involving homogeneous is an equation involving homogeneous
vectors, so vectors, so PXPXii and and xxi i need only be in the same need only be in the same
direction, not strictly equaldirection, not strictly equal
• We can specify “same directionality” by using a We can specify “same directionality” by using a cross product formulation:cross product formulation:
![Page 54: Computer Vision The 2D projective plane and it’s applications HZ Ch 2. In particular: Ch 2.1-4, 2.7, Szelisky: Ch 2.1.1, 2.1.2 Estimation:HZ: Ch 4.1-4.2.5,](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e8a5503460f94b8f917/html5/thumbnails/54.jpg)
DLT Camera Matrix Estimation: Preliminaries
•Let the image point Let the image point xxii = ( = (xxii, , yyii, , wwii))TT
(remember that (remember that XXii has 4 elements) has 4 elements)
•Denoting the Denoting the jjth row of th row of PP by by ppjTjT (a 4-element (a 4-element row vector), we have: row vector), we have:
![Page 55: Computer Vision The 2D projective plane and it’s applications HZ Ch 2. In particular: Ch 2.1-4, 2.7, Szelisky: Ch 2.1.1, 2.1.2 Estimation:HZ: Ch 4.1-4.2.5,](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e8a5503460f94b8f917/html5/thumbnails/55.jpg)
DLT Camera Matrix Estimation: Step 1
•Then by the definition of the cross product, Then by the definition of the cross product,
xxi i PXPXii is: is:
![Page 56: Computer Vision The 2D projective plane and it’s applications HZ Ch 2. In particular: Ch 2.1-4, 2.7, Szelisky: Ch 2.1.1, 2.1.2 Estimation:HZ: Ch 4.1-4.2.5,](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e8a5503460f94b8f917/html5/thumbnails/56.jpg)
DLT Camera Matrix Estimation: Step 2
•The dot product commutes, so The dot product commutes, so ppjTjT XXii ==
XXTTii ppjj, and we can rewrite the preceding as:, and we can rewrite the preceding as:
![Page 57: Computer Vision The 2D projective plane and it’s applications HZ Ch 2. In particular: Ch 2.1-4, 2.7, Szelisky: Ch 2.1.1, 2.1.2 Estimation:HZ: Ch 4.1-4.2.5,](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e8a5503460f94b8f917/html5/thumbnails/57.jpg)
DLT Camera Matrix Estimation: Step 3
• Collecting terms, this can be rewritten as a Collecting terms, this can be rewritten as a matrix product:matrix product:
where where 00TT = (0, 0, 0, 0) = (0, 0, 0, 0). This is a . This is a 3 x 12 matrix times a 12-element column 3 x 12 matrix times a 12-element column vector vector pp = ( = (pp11TT, , pp22TT, , pp33TT))TT
![Page 58: Computer Vision The 2D projective plane and it’s applications HZ Ch 2. In particular: Ch 2.1-4, 2.7, Szelisky: Ch 2.1.1, 2.1.2 Estimation:HZ: Ch 4.1-4.2.5,](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e8a5503460f94b8f917/html5/thumbnails/58.jpg)
What We Just Did
![Page 59: Computer Vision The 2D projective plane and it’s applications HZ Ch 2. In particular: Ch 2.1-4, 2.7, Szelisky: Ch 2.1.1, 2.1.2 Estimation:HZ: Ch 4.1-4.2.5,](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e8a5503460f94b8f917/html5/thumbnails/59.jpg)
DLT Camera Matrix Estimation: Step 4
• There are only two linearly independent rows here There are only two linearly independent rows here – The third row is obtained by adding The third row is obtained by adding xxii times the first row to times the first row to yyii times the times the
second and scaling the sum by second and scaling the sum by -1/-1/wwii
![Page 60: Computer Vision The 2D projective plane and it’s applications HZ Ch 2. In particular: Ch 2.1-4, 2.7, Szelisky: Ch 2.1.1, 2.1.2 Estimation:HZ: Ch 4.1-4.2.5,](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e8a5503460f94b8f917/html5/thumbnails/60.jpg)
DLT Camera Matrix Estimation: Step 4
• So we can eliminate one row to obtain the So we can eliminate one row to obtain the
following linear matrix equation for the following linear matrix equation for the iith pair th pair of corresponding points:of corresponding points:
• Write this as Write this as AAii pp = = 00
![Page 61: Computer Vision The 2D projective plane and it’s applications HZ Ch 2. In particular: Ch 2.1-4, 2.7, Szelisky: Ch 2.1.1, 2.1.2 Estimation:HZ: Ch 4.1-4.2.5,](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e8a5503460f94b8f917/html5/thumbnails/61.jpg)
DLT Camera Matrix Estimation: Step 5
• Remember that there are 11 unknowns which Remember that there are 11 unknowns which generate the 3 x 4 homogeneous matrix generate the 3 x 4 homogeneous matrix PP
(represented in vector form by (represented in vector form by pp))• Each point correspondence yields 2 equations Each point correspondence yields 2 equations
(the two row of (the two row of AAii)) We need at least 5 ½ point correspondences to We need at least 5 ½ point correspondences to
solve for solve for pp• Stack Stack AAii to get homogeneous linear system to get homogeneous linear system AA p p
= 0= 0
![Page 62: Computer Vision The 2D projective plane and it’s applications HZ Ch 2. In particular: Ch 2.1-4, 2.7, Szelisky: Ch 2.1.1, 2.1.2 Estimation:HZ: Ch 4.1-4.2.5,](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e8a5503460f94b8f917/html5/thumbnails/62.jpg)
Experiment:
![Page 63: Computer Vision The 2D projective plane and it’s applications HZ Ch 2. In particular: Ch 2.1-4, 2.7, Szelisky: Ch 2.1.1, 2.1.2 Estimation:HZ: Ch 4.1-4.2.5,](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e8a5503460f94b8f917/html5/thumbnails/63.jpg)
short and long focal length
Radial Distortion
![Page 64: Computer Vision The 2D projective plane and it’s applications HZ Ch 2. In particular: Ch 2.1-4, 2.7, Szelisky: Ch 2.1.1, 2.1.2 Estimation:HZ: Ch 4.1-4.2.5,](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e8a5503460f94b8f917/html5/thumbnails/64.jpg)
Radial Distortion
![Page 65: Computer Vision The 2D projective plane and it’s applications HZ Ch 2. In particular: Ch 2.1-4, 2.7, Szelisky: Ch 2.1.1, 2.1.2 Estimation:HZ: Ch 4.1-4.2.5,](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e8a5503460f94b8f917/html5/thumbnails/65.jpg)
Radial Distortion
![Page 66: Computer Vision The 2D projective plane and it’s applications HZ Ch 2. In particular: Ch 2.1-4, 2.7, Szelisky: Ch 2.1.1, 2.1.2 Estimation:HZ: Ch 4.1-4.2.5,](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e8a5503460f94b8f917/html5/thumbnails/66.jpg)
Correction of distortion
Choice of the distortion function and center
Computing the parameters of the distortion function(i) Minimize with additional unknowns(ii) Straighten lines(iii) …
Radial Distortion