scene planes and homographies class 16 multiple view geometry comp 290-089 marc pollefeys
Post on 19-Dec-2015
217 views
TRANSCRIPT
![Page 1: Scene Planes and Homographies class 16 Multiple View Geometry Comp 290-089 Marc Pollefeys](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d375503460f94a100e0/html5/thumbnails/1.jpg)
Scene Planes and Homographies
class 16
Multiple View GeometryComp 290-089Marc Pollefeys
![Page 2: Scene Planes and Homographies class 16 Multiple View Geometry Comp 290-089 Marc Pollefeys](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d375503460f94a100e0/html5/thumbnails/2.jpg)
Multiple View Geometry course schedule(subject to change)
Jan. 7, 9 Intro & motivation Projective 2D Geometry
Jan. 14, 16
(no class) Projective 2D Geometry
Jan. 21, 23
Projective 3D Geometry (no class)
Jan. 28, 30
Parameter Estimation Parameter Estimation
Feb. 4, 6 Algorithm Evaluation Camera Models
Feb. 11, 13
Camera Calibration Single View Geometry
Feb. 18, 20
Epipolar Geometry 3D reconstruction
Feb. 25, 27
Fund. Matrix Comp. Fund. Matrix Comp.
Mar. 4, 6 Rect. & Structure Comp.
Planes & Homographies
Mar. 18, 20
Trifocal Tensor Three View Reconstruction
Mar. 25, 27
Multiple View Geometry
MultipleView Reconstruction
Apr. 1, 3 Bundle adjustment Papers
Apr. 8, 10
Auto-Calibration Papers
Apr. 15, 17
Dynamic SfM Papers
Apr. 22, 24
Cheirality Project Demos
![Page 3: Scene Planes and Homographies class 16 Multiple View Geometry Comp 290-089 Marc Pollefeys](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d375503460f94a100e0/html5/thumbnails/3.jpg)
Two-view geometry
Epipolar geometry
3D reconstruction
F-matrix comp.
Structure comp.
![Page 4: Scene Planes and Homographies class 16 Multiple View Geometry Comp 290-089 Marc Pollefeys](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d375503460f94a100e0/html5/thumbnails/4.jpg)
Planar rectification
Bring two views Bring two views to standard stereo setupto standard stereo setup
(moves epipole to )(not possible when in/close to image)
(standard approach)
![Page 5: Scene Planes and Homographies class 16 Multiple View Geometry Comp 290-089 Marc Pollefeys](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d375503460f94a100e0/html5/thumbnails/5.jpg)
Polar re-parameterization around epipoles
Requires only (oriented) epipolar geometry
Preserve length of epipolar linesChoose so that no pixels are
compressed
original image rectified image
Polar rectification(Pollefeys et al. ICCV’99)
Works for all relative motionsGuarantees minimal image size
![Page 6: Scene Planes and Homographies class 16 Multiple View Geometry Comp 290-089 Marc Pollefeys](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d375503460f94a100e0/html5/thumbnails/6.jpg)
polar rectification: example
![Page 7: Scene Planes and Homographies class 16 Multiple View Geometry Comp 290-089 Marc Pollefeys](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d375503460f94a100e0/html5/thumbnails/7.jpg)
polar rectification: example
![Page 8: Scene Planes and Homographies class 16 Multiple View Geometry Comp 290-089 Marc Pollefeys](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d375503460f94a100e0/html5/thumbnails/8.jpg)
Example: Béguinage of Leuven
Does not work with standard Homography-based approaches
![Page 9: Scene Planes and Homographies class 16 Multiple View Geometry Comp 290-089 Marc Pollefeys](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d375503460f94a100e0/html5/thumbnails/9.jpg)
Stereo matching
• attempt to match every pixel• use additional constraints
![Page 10: Scene Planes and Homographies class 16 Multiple View Geometry Comp 290-089 Marc Pollefeys](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d375503460f94a100e0/html5/thumbnails/10.jpg)
Stereo matching
Optimal path(dynamic programming )
Similarity measure(SSD or NCC)
Constraints• epipolar
• ordering
• uniqueness
• disparity limit
• disparity gradient limit
Trade-off
• Matching cost (data)
• Discontinuities (prior)
(Cox et al. CVGIP’96; Koch’96; Falkenhagen´97; Van Meerbergen,Vergauwen,Pollefeys,VanGool IJCV‘02)
![Page 11: Scene Planes and Homographies class 16 Multiple View Geometry Comp 290-089 Marc Pollefeys](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d375503460f94a100e0/html5/thumbnails/11.jpg)
Disparity map
image I(x,y) image I´(x´,y´)Disparity map D(x,y)
(x´,y´)=(x+D(x,y),y)
![Page 12: Scene Planes and Homographies class 16 Multiple View Geometry Comp 290-089 Marc Pollefeys](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d375503460f94a100e0/html5/thumbnails/12.jpg)
Point reconstruction
PXx XP'x'
0X
p'p''
p'p''
pp
pp
2T3T
1T3T
2T3T
1T3T
y
x
y
x 222
211 XP,xXP,x dd
222
211 αl,xαl,x dd
![Page 13: Scene Planes and Homographies class 16 Multiple View Geometry Comp 290-089 Marc Pollefeys](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d375503460f94a100e0/html5/thumbnails/13.jpg)
Line reconstruction
P'l'Pl
T
T
L
doesn‘t work for epipolar plane
![Page 14: Scene Planes and Homographies class 16 Multiple View Geometry Comp 290-089 Marc Pollefeys](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d375503460f94a100e0/html5/thumbnails/14.jpg)
Scene planes and homographies
plane induces homography between two views
![Page 15: Scene Planes and Homographies class 16 Multiple View Geometry Comp 290-089 Marc Pollefeys](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d375503460f94a100e0/html5/thumbnails/15.jpg)
Homography given plane
a]|[AP' 0]|[IP
0XπT TT ,1vπ
Hxx' TavAH
0]X|[IPXx
TT ρ,xX
point on plane
TTT x,-vxproject in second view
xavAx' T
![Page 16: Scene Planes and Homographies class 16 Multiple View Geometry Comp 290-089 Marc Pollefeys](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d375503460f94a100e0/html5/thumbnails/16.jpg)
Homography given plane and vice-versa
TavAH T
111T vσuavHA
1T
T11
11TT
ua
vσuuaavHA
![Page 17: Scene Planes and Homographies class 16 Multiple View Geometry Comp 290-089 Marc Pollefeys](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d375503460f94a100e0/html5/thumbnails/17.jpg)
Calibrated stereo rig
0]|K[IPE t]|[RK'P'E
-1T K/tnRK'H d
dn/v
TT ) n(π d
![Page 18: Scene Planes and Homographies class 16 Multiple View Geometry Comp 290-089 Marc Pollefeys](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d375503460f94a100e0/html5/thumbnails/18.jpg)
homographies and epipolar geometry
points on plane also have to satisfy epipolar geometry!
x 0,FxHxFxHx TTT
HTF has to be skew-symmetric
0HFFH TT x ,x'e'Fx
x'x 0,'x'eHx TT
H'eF
![Page 19: Scene Planes and Homographies class 16 Multiple View Geometry Comp 290-089 Marc Pollefeys](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d375503460f94a100e0/html5/thumbnails/19.jpg)
e
(pick l =e’, since e’Te’≠0)Fxl'x π
homographies and epipolar geometry
πlle'
πlx Fxl
Fe'Hπ H'eF withcompare
![Page 20: Scene Planes and Homographies class 16 Multiple View Geometry Comp 290-089 Marc Pollefeys](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d375503460f94a100e0/html5/thumbnails/20.jpg)
Homography also maps epipole
Hee'
![Page 21: Scene Planes and Homographies class 16 Multiple View Geometry Comp 290-089 Marc Pollefeys](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d375503460f94a100e0/html5/thumbnails/21.jpg)
Homography also maps epipolar lines
eT
e l'Hl
![Page 22: Scene Planes and Homographies class 16 Multiple View Geometry Comp 290-089 Marc Pollefeys](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d375503460f94a100e0/html5/thumbnails/22.jpg)
Compatibility constraint
Hxx'Fxl'e
![Page 23: Scene Planes and Homographies class 16 Multiple View Geometry Comp 290-089 Marc Pollefeys](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d375503460f94a100e0/html5/thumbnails/23.jpg)
plane homography given F and 3 points correspondences
Method 1: reconstruct explicitly, compute plane through 3 points derive homography
Method 2: use epipoles as 4th correspondence to compute homography
![Page 24: Scene Planes and Homographies class 16 Multiple View Geometry Comp 290-089 Marc Pollefeys](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d375503460f94a100e0/html5/thumbnails/24.jpg)
degenerate geometry for an implicit computation of the homography
![Page 25: Scene Planes and Homographies class 16 Multiple View Geometry Comp 290-089 Marc Pollefeys](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d375503460f94a100e0/html5/thumbnails/25.jpg)
Estimastion from 3 noisy points (+F)
Consistency constraint: points have to be in exact epipolar correspodence
Determine MLE points given F and x↔x’
Use implicit 3D approach (no derivation here)
T1bMe'AH
2T e'x'/e'x'Ax'x iiiiib
Fe'A
![Page 26: Scene Planes and Homographies class 16 Multiple View Geometry Comp 290-089 Marc Pollefeys](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d375503460f94a100e0/html5/thumbnails/26.jpg)
plane homography given F, a point and a line
Tlμe'Fl'H
xle'x'
l'Fxx'e'x'μ
T2
T
![Page 27: Scene Planes and Homographies class 16 Multiple View Geometry Comp 290-089 Marc Pollefeys](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d375503460f94a100e0/html5/thumbnails/27.jpg)
application: matching lines(Schmid and Zisserman, CVPR’97)
![Page 28: Scene Planes and Homographies class 16 Multiple View Geometry Comp 290-089 Marc Pollefeys](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d375503460f94a100e0/html5/thumbnails/28.jpg)
epipolar geometry induces point homography on lines
![Page 29: Scene Planes and Homographies class 16 Multiple View Geometry Comp 290-089 Marc Pollefeys](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d375503460f94a100e0/html5/thumbnails/29.jpg)
Degenerate homographies
Fxl''x
![Page 30: Scene Planes and Homographies class 16 Multiple View Geometry Comp 290-089 Marc Pollefeys](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d375503460f94a100e0/html5/thumbnails/30.jpg)
plane induced parallax
![Page 31: Scene Planes and Homographies class 16 Multiple View Geometry Comp 290-089 Marc Pollefeys](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d375503460f94a100e0/html5/thumbnails/31.jpg)
Hx'xl
![Page 32: Scene Planes and Homographies class 16 Multiple View Geometry Comp 290-089 Marc Pollefeys](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d375503460f94a100e0/html5/thumbnails/32.jpg)
6-point algorithm
6655 Hx'xHx'xe'
x1,x2,x3,x4 in plane, x5,x6 out of plane
Compute H from x1,x2,x3,x4
He'F
![Page 33: Scene Planes and Homographies class 16 Multiple View Geometry Comp 290-089 Marc Pollefeys](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d375503460f94a100e0/html5/thumbnails/33.jpg)
Projective depth
ρe'Hxx'
TT ρ,xX
=0 on planesign of determines on which side of plane
![Page 34: Scene Planes and Homographies class 16 Multiple View Geometry Comp 290-089 Marc Pollefeys](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d375503460f94a100e0/html5/thumbnails/34.jpg)
Binary space partition
![Page 35: Scene Planes and Homographies class 16 Multiple View Geometry Comp 290-089 Marc Pollefeys](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d375503460f94a100e0/html5/thumbnails/35.jpg)
Two planes
1-12 HHH eHe iH[e]F
H has fixed point and fixed line
![Page 36: Scene Planes and Homographies class 16 Multiple View Geometry Comp 290-089 Marc Pollefeys](https://reader036.vdocuments.mx/reader036/viewer/2022062714/56649d375503460f94a100e0/html5/thumbnails/36.jpg)
Next class: The Trifocal Tensor