announcements project 3 due thursday by 11:59pm demos on friday; signup on cms prelim to be...
TRANSCRIPT
![Page 1: Announcements Project 3 due Thursday by 11:59pm Demos on Friday; signup on CMS Prelim to be distributed in class Friday, due Wednesday by the beginning](https://reader035.vdocuments.mx/reader035/viewer/2022070401/56649f1d5503460f94c34048/html5/thumbnails/1.jpg)
Announcements
• Project 3 due Thursday by 11:59pm
• Demos on Friday; signup on CMS
• Prelim to be distributed in class Friday, due Wednesday by the beginning of class– No late exams accepted
![Page 2: Announcements Project 3 due Thursday by 11:59pm Demos on Friday; signup on CMS Prelim to be distributed in class Friday, due Wednesday by the beginning](https://reader035.vdocuments.mx/reader035/viewer/2022070401/56649f1d5503460f94c34048/html5/thumbnails/2.jpg)
Lines and points
• See board
• Homographies map lines to lines• Homographies map points to points
• Is there anything that maps a point to a line?
![Page 3: Announcements Project 3 due Thursday by 11:59pm Demos on Friday; signup on CMS Prelim to be distributed in class Friday, due Wednesday by the beginning](https://reader035.vdocuments.mx/reader035/viewer/2022070401/56649f1d5503460f94c34048/html5/thumbnails/3.jpg)
A couple important things…
• Homogeneous points– How do you represent a point at infinity?– How many points at infinity are there?– What is a vanishing line?
• What are the parameters of a camera?– What does the K matrix mean?
• Another way to fit a line to a set of points?
![Page 4: Announcements Project 3 due Thursday by 11:59pm Demos on Friday; signup on CMS Prelim to be distributed in class Friday, due Wednesday by the beginning](https://reader035.vdocuments.mx/reader035/viewer/2022070401/56649f1d5503460f94c34048/html5/thumbnails/4.jpg)
Questions?
![Page 5: Announcements Project 3 due Thursday by 11:59pm Demos on Friday; signup on CMS Prelim to be distributed in class Friday, due Wednesday by the beginning](https://reader035.vdocuments.mx/reader035/viewer/2022070401/56649f1d5503460f94c34048/html5/thumbnails/5.jpg)
Lecture 20: Two-view geometry
CS4670 / 5670 : Computer VisionNoah Snavely
![Page 6: Announcements Project 3 due Thursday by 11:59pm Demos on Friday; signup on CMS Prelim to be distributed in class Friday, due Wednesday by the beginning](https://reader035.vdocuments.mx/reader035/viewer/2022070401/56649f1d5503460f94c34048/html5/thumbnails/6.jpg)
Readings
• Szeliski, Chapter 7.2• “Fundamental matrix song”
![Page 7: Announcements Project 3 due Thursday by 11:59pm Demos on Friday; signup on CMS Prelim to be distributed in class Friday, due Wednesday by the beginning](https://reader035.vdocuments.mx/reader035/viewer/2022070401/56649f1d5503460f94c34048/html5/thumbnails/7.jpg)
Back to stereo
• Where do epipolar lines come from?
![Page 8: Announcements Project 3 due Thursday by 11:59pm Demos on Friday; signup on CMS Prelim to be distributed in class Friday, due Wednesday by the beginning](https://reader035.vdocuments.mx/reader035/viewer/2022070401/56649f1d5503460f94c34048/html5/thumbnails/8.jpg)
Two-view geometry
• Where do epipolar lines come from?
epipolar plane
epipolar lineepipolar lineepipolar lineepipolar line
0
3d point lies somewhere along r
(projection of r)
Image 1 Image 2
![Page 9: Announcements Project 3 due Thursday by 11:59pm Demos on Friday; signup on CMS Prelim to be distributed in class Friday, due Wednesday by the beginning](https://reader035.vdocuments.mx/reader035/viewer/2022070401/56649f1d5503460f94c34048/html5/thumbnails/9.jpg)
Fundamental matrix
• This epipolar geometry of two views is described by a Very Special 3x3 matrix , called the fundamental matrix
• maps (homogeneous) points in image 1 to lines in image 2!• The epipolar line (in image 2) of point p is:
• Epipolar constraint on corresponding points:
epipolar plane
epipolar lineepipolar lineepipolar lineepipolar line
0
(projection of ray)
Image 1 Image 2
![Page 10: Announcements Project 3 due Thursday by 11:59pm Demos on Friday; signup on CMS Prelim to be distributed in class Friday, due Wednesday by the beginning](https://reader035.vdocuments.mx/reader035/viewer/2022070401/56649f1d5503460f94c34048/html5/thumbnails/10.jpg)
Fundamental matrix
• Two Special points: e1 and e2 (the epipoles): projection of one camera into the other
epipolar plane
epipolar lineepipolar lineepipolar lineepipolar line
0
(projection of ray)
![Page 11: Announcements Project 3 due Thursday by 11:59pm Demos on Friday; signup on CMS Prelim to be distributed in class Friday, due Wednesday by the beginning](https://reader035.vdocuments.mx/reader035/viewer/2022070401/56649f1d5503460f94c34048/html5/thumbnails/11.jpg)
Fundamental matrix
• Two special points: e1 and e2 (the epipoles): projection of one camera into the other
• All of the epipolar lines in an image pass through the epipole
0
![Page 12: Announcements Project 3 due Thursday by 11:59pm Demos on Friday; signup on CMS Prelim to be distributed in class Friday, due Wednesday by the beginning](https://reader035.vdocuments.mx/reader035/viewer/2022070401/56649f1d5503460f94c34048/html5/thumbnails/12.jpg)
Rectified case
• Images have the same orientation, t parallel to image planes
• Where are the epipoles?
![Page 13: Announcements Project 3 due Thursday by 11:59pm Demos on Friday; signup on CMS Prelim to be distributed in class Friday, due Wednesday by the beginning](https://reader035.vdocuments.mx/reader035/viewer/2022070401/56649f1d5503460f94c34048/html5/thumbnails/13.jpg)
Epipolar geometry demo
![Page 14: Announcements Project 3 due Thursday by 11:59pm Demos on Friday; signup on CMS Prelim to be distributed in class Friday, due Wednesday by the beginning](https://reader035.vdocuments.mx/reader035/viewer/2022070401/56649f1d5503460f94c34048/html5/thumbnails/14.jpg)
Relationship with homography?
Images taken from the same center of projection? Use a homography!
![Page 15: Announcements Project 3 due Thursday by 11:59pm Demos on Friday; signup on CMS Prelim to be distributed in class Friday, due Wednesday by the beginning](https://reader035.vdocuments.mx/reader035/viewer/2022070401/56649f1d5503460f94c34048/html5/thumbnails/15.jpg)
Fundamental matrix – uncalibrated case
0
the Fundamental matrix
: intrinsics of camera 1 : intrinsics of camera 2
: rotation of image 2 w.r.t. camera 1
![Page 16: Announcements Project 3 due Thursday by 11:59pm Demos on Friday; signup on CMS Prelim to be distributed in class Friday, due Wednesday by the beginning](https://reader035.vdocuments.mx/reader035/viewer/2022070401/56649f1d5503460f94c34048/html5/thumbnails/16.jpg)
Cross-product as linear operator
Useful fact: Cross product with a vector t can be represented as multiplication with a (skew-symmetric) 3x3 matrix
![Page 17: Announcements Project 3 due Thursday by 11:59pm Demos on Friday; signup on CMS Prelim to be distributed in class Friday, due Wednesday by the beginning](https://reader035.vdocuments.mx/reader035/viewer/2022070401/56649f1d5503460f94c34048/html5/thumbnails/17.jpg)
Fundamental matrix – calibrated case
0
: ray through p in camera 1’s (and world) coordinate system
: ray through q in camera 2’s coordinate system
{the Essential matrix
![Page 18: Announcements Project 3 due Thursday by 11:59pm Demos on Friday; signup on CMS Prelim to be distributed in class Friday, due Wednesday by the beginning](https://reader035.vdocuments.mx/reader035/viewer/2022070401/56649f1d5503460f94c34048/html5/thumbnails/18.jpg)
Properties of the Fundamental Matrix
• is the epipolar line associated with
• is the epipolar line associated with
• and
• is rank 2
• How many parameters does F have?18
T
![Page 19: Announcements Project 3 due Thursday by 11:59pm Demos on Friday; signup on CMS Prelim to be distributed in class Friday, due Wednesday by the beginning](https://reader035.vdocuments.mx/reader035/viewer/2022070401/56649f1d5503460f94c34048/html5/thumbnails/19.jpg)
Rectified case
![Page 20: Announcements Project 3 due Thursday by 11:59pm Demos on Friday; signup on CMS Prelim to be distributed in class Friday, due Wednesday by the beginning](https://reader035.vdocuments.mx/reader035/viewer/2022070401/56649f1d5503460f94c34048/html5/thumbnails/20.jpg)
Stereo image rectification
• reproject image planes onto a common• plane parallel to the line between optical centers• pixel motion is horizontal after this transformation• two homographies (3x3 transform), one for each input
image reprojection C. Loop and Z. Zhang. Computing Rectifying Homographies for Stereo Vision.
IEEE Conf. Computer Vision and Pattern Recognition, 1999.
![Page 21: Announcements Project 3 due Thursday by 11:59pm Demos on Friday; signup on CMS Prelim to be distributed in class Friday, due Wednesday by the beginning](https://reader035.vdocuments.mx/reader035/viewer/2022070401/56649f1d5503460f94c34048/html5/thumbnails/21.jpg)
Questions?
![Page 22: Announcements Project 3 due Thursday by 11:59pm Demos on Friday; signup on CMS Prelim to be distributed in class Friday, due Wednesday by the beginning](https://reader035.vdocuments.mx/reader035/viewer/2022070401/56649f1d5503460f94c34048/html5/thumbnails/22.jpg)
Estimating F
• If we don’t know K1, K2, R, or t, can we estimate F for two images?
• Yes, given enough correspondences
![Page 23: Announcements Project 3 due Thursday by 11:59pm Demos on Friday; signup on CMS Prelim to be distributed in class Friday, due Wednesday by the beginning](https://reader035.vdocuments.mx/reader035/viewer/2022070401/56649f1d5503460f94c34048/html5/thumbnails/23.jpg)
Estimating F – 8-point algorithm
• The fundamental matrix F is defined by
0Fxx'
for any pair of matches x and x’ in two images.
• Let x=(u,v,1)T and x’=(u’,v’,1)T,
333231
232221
131211
fff
fff
fff
F
each match gives a linear equation
0'''''' 333231232221131211 fvfuffvfvvfuvfufvufuu
![Page 24: Announcements Project 3 due Thursday by 11:59pm Demos on Friday; signup on CMS Prelim to be distributed in class Friday, due Wednesday by the beginning](https://reader035.vdocuments.mx/reader035/viewer/2022070401/56649f1d5503460f94c34048/html5/thumbnails/24.jpg)
8-point algorithm
0
1´´´´´´
1´´´´´´
1´´´´´´
33
32
31
23
22
21
13
12
11
222222222222
111111111111
f
f
f
f
f
f
f
f
f
vuvvvvuuuvuu
vuvvvvuuuvuu
vuvvvvuuuvuu
nnnnnnnnnnnn
• In reality, instead of solving , we seek f to minimize , least eigenvector of .
0Af
Af AA
![Page 25: Announcements Project 3 due Thursday by 11:59pm Demos on Friday; signup on CMS Prelim to be distributed in class Friday, due Wednesday by the beginning](https://reader035.vdocuments.mx/reader035/viewer/2022070401/56649f1d5503460f94c34048/html5/thumbnails/25.jpg)
8-point algorithm – Problem?• F should have rank 2• To enforce that F is of rank 2, F is replaced by F’ that
minimizes subject to the rank constraint. 'FF
• This is achieved by SVD. Let , where
, let
then is the solution.
VUF Σ
3
2
1
00
00
00
Σ
000
00
00
Σ' 2
1
VUF Σ''
![Page 26: Announcements Project 3 due Thursday by 11:59pm Demos on Friday; signup on CMS Prelim to be distributed in class Friday, due Wednesday by the beginning](https://reader035.vdocuments.mx/reader035/viewer/2022070401/56649f1d5503460f94c34048/html5/thumbnails/26.jpg)
8-point algorithm% Build the constraint matrix A = [x2(1,:)‘.*x1(1,:)' x2(1,:)'.*x1(2,:)' x2(1,:)' ... x2(2,:)'.*x1(1,:)' x2(2,:)'.*x1(2,:)' x2(2,:)' ... x1(1,:)' x1(2,:)' ones(npts,1) ]; [U,D,V] = svd(A); % Extract fundamental matrix from the column of V % corresponding to the smallest singular value. F = reshape(V(:,9),3,3)'; % Enforce rank2 constraint [U,D,V] = svd(F); F = U*diag([D(1,1) D(2,2) 0])*V';
![Page 27: Announcements Project 3 due Thursday by 11:59pm Demos on Friday; signup on CMS Prelim to be distributed in class Friday, due Wednesday by the beginning](https://reader035.vdocuments.mx/reader035/viewer/2022070401/56649f1d5503460f94c34048/html5/thumbnails/27.jpg)
8-point algorithm
• Pros: it is linear, easy to implement and fast• Cons: susceptible to noise
![Page 28: Announcements Project 3 due Thursday by 11:59pm Demos on Friday; signup on CMS Prelim to be distributed in class Friday, due Wednesday by the beginning](https://reader035.vdocuments.mx/reader035/viewer/2022070401/56649f1d5503460f94c34048/html5/thumbnails/28.jpg)
Results (ground truth)
![Page 29: Announcements Project 3 due Thursday by 11:59pm Demos on Friday; signup on CMS Prelim to be distributed in class Friday, due Wednesday by the beginning](https://reader035.vdocuments.mx/reader035/viewer/2022070401/56649f1d5503460f94c34048/html5/thumbnails/29.jpg)
Results (normalized 8-point algorithm)
![Page 30: Announcements Project 3 due Thursday by 11:59pm Demos on Friday; signup on CMS Prelim to be distributed in class Friday, due Wednesday by the beginning](https://reader035.vdocuments.mx/reader035/viewer/2022070401/56649f1d5503460f94c34048/html5/thumbnails/30.jpg)
What about more than two views?
• The geometry of three views is described by a 3 x 3 x 3 tensor called the trifocal tensor
• The geometry of four views is described by a 3 x 3 x 3 x 3 tensor called the quadrifocal tensor
• After this it starts to get complicated…
![Page 31: Announcements Project 3 due Thursday by 11:59pm Demos on Friday; signup on CMS Prelim to be distributed in class Friday, due Wednesday by the beginning](https://reader035.vdocuments.mx/reader035/viewer/2022070401/56649f1d5503460f94c34048/html5/thumbnails/31.jpg)
Large-scale structure from motion
Dubrovnik, Croatia. 4,619 images (out of an initial 57,845).Total reconstruction time: 23 hoursNumber of cores: 352