ee 7700 geometric transformations. bahadir k. gunturk2 geometric transformation translation scale...
TRANSCRIPT
![Page 1: EE 7700 Geometric Transformations. Bahadir K. Gunturk2 Geometric Transformation translation scale rotation & scale Rigid flow Rotation matrix Scale matrix](https://reader035.vdocuments.mx/reader035/viewer/2022062309/5697bf741a28abf838c7fce5/html5/thumbnails/1.jpg)
EE 7700
Geometric Transformations
![Page 2: EE 7700 Geometric Transformations. Bahadir K. Gunturk2 Geometric Transformation translation scale rotation & scale Rigid flow Rotation matrix Scale matrix](https://reader035.vdocuments.mx/reader035/viewer/2022062309/5697bf741a28abf838c7fce5/html5/thumbnails/2.jpg)
Bahadir K. Gunturk 2
Geometric Transformation
translation
scale
rotation & scale
2
1
43
21
b
b
y
x
aa
aa
y
x
Rigid flow
11001
'
'
243
121
y
x
baa
baa
y
x
cossin
sincos
s
s
0
0
Rotation matrix
Scale matrix
![Page 3: EE 7700 Geometric Transformations. Bahadir K. Gunturk2 Geometric Transformation translation scale rotation & scale Rigid flow Rotation matrix Scale matrix](https://reader035.vdocuments.mx/reader035/viewer/2022062309/5697bf741a28abf838c7fce5/html5/thumbnails/3.jpg)
Bahadir K. Gunturk 3
Affine Flow
2
1
43
21
b
b
y
x
aa
aa
yy
xx
v
u
![Page 4: EE 7700 Geometric Transformations. Bahadir K. Gunturk2 Geometric Transformation translation scale rotation & scale Rigid flow Rotation matrix Scale matrix](https://reader035.vdocuments.mx/reader035/viewer/2022062309/5697bf741a28abf838c7fce5/html5/thumbnails/4.jpg)
Bahadir K. Gunturk 4
Perspective Flow
1 2 3
7 8
'1
a x a y ax
a x a y
4 5 6
7 8
'1
a x a y ay
a x a y
'
'
u x x
v y y
![Page 5: EE 7700 Geometric Transformations. Bahadir K. Gunturk2 Geometric Transformation translation scale rotation & scale Rigid flow Rotation matrix Scale matrix](https://reader035.vdocuments.mx/reader035/viewer/2022062309/5697bf741a28abf838c7fce5/html5/thumbnails/5.jpg)
Bahadir K. Gunturk 5
![Page 6: EE 7700 Geometric Transformations. Bahadir K. Gunturk2 Geometric Transformation translation scale rotation & scale Rigid flow Rotation matrix Scale matrix](https://reader035.vdocuments.mx/reader035/viewer/2022062309/5697bf741a28abf838c7fce5/html5/thumbnails/6.jpg)
Bahadir K. Gunturk 6
![Page 7: EE 7700 Geometric Transformations. Bahadir K. Gunturk2 Geometric Transformation translation scale rotation & scale Rigid flow Rotation matrix Scale matrix](https://reader035.vdocuments.mx/reader035/viewer/2022062309/5697bf741a28abf838c7fce5/html5/thumbnails/7.jpg)
Bahadir K. Gunturk 7
![Page 8: EE 7700 Geometric Transformations. Bahadir K. Gunturk2 Geometric Transformation translation scale rotation & scale Rigid flow Rotation matrix Scale matrix](https://reader035.vdocuments.mx/reader035/viewer/2022062309/5697bf741a28abf838c7fce5/html5/thumbnails/8.jpg)
EE 7730
RANSAC: RANdom SAmple Consensus
![Page 9: EE 7700 Geometric Transformations. Bahadir K. Gunturk2 Geometric Transformation translation scale rotation & scale Rigid flow Rotation matrix Scale matrix](https://reader035.vdocuments.mx/reader035/viewer/2022062309/5697bf741a28abf838c7fce5/html5/thumbnails/9.jpg)
Bahadir K. Gunturk 9
Outliers
Consider the least squares fit for optical flow:
1 2 1
3 4 2
( , )
( , )
u x y a x a y b
v x y a x a y b
1 1 1 1
1 1 1 2
1
3
4
2
1 0 0 0
0 0 0 1
1 0 0 0
0 0 0 1n n n
n nn
u x y a
v x y a
b
a
u ax y
bx yv
If some of the values are wrong, it will degrade the estimation.
![Page 10: EE 7700 Geometric Transformations. Bahadir K. Gunturk2 Geometric Transformation translation scale rotation & scale Rigid flow Rotation matrix Scale matrix](https://reader035.vdocuments.mx/reader035/viewer/2022062309/5697bf741a28abf838c7fce5/html5/thumbnails/10.jpg)
Bahadir K. Gunturk 10
Outliers
It is best not to include outliers in the estimation.
Outliers
Best fit is degraded due to the outliers
Line Fitting Problem: Given (x1,y1), …, (xN,yN); find the line y=ax+b
1 1 1
1N N
y xa
by x
![Page 11: EE 7700 Geometric Transformations. Bahadir K. Gunturk2 Geometric Transformation translation scale rotation & scale Rigid flow Rotation matrix Scale matrix](https://reader035.vdocuments.mx/reader035/viewer/2022062309/5697bf741a28abf838c7fce5/html5/thumbnails/11.jpg)
Bahadir K. Gunturk 11
Robust Estimation
Two step process: Classify data points as outliers or inliers Use inliers only to fit a model
![Page 12: EE 7700 Geometric Transformations. Bahadir K. Gunturk2 Geometric Transformation translation scale rotation & scale Rigid flow Rotation matrix Scale matrix](https://reader035.vdocuments.mx/reader035/viewer/2022062309/5697bf741a28abf838c7fce5/html5/thumbnails/12.jpg)
Bahadir K. Gunturk 12
RANSAC
Repeat for k times: Randomly choose n points (the smallest number of points
required) from the data. Estimate the parameters using these points. For each data point other than these n points:
Check if the data point is within a threshold, t, distance of current model; if it is, the data point is consistent with current model.
The total number of data points that are consistent is model’s support.
If the support is larger than a predetermined number, d, then there is a good fit. Re-estimate the parameters using these inliers.
Choose the best fit with the smallest fitting error.
![Page 13: EE 7700 Geometric Transformations. Bahadir K. Gunturk2 Geometric Transformation translation scale rotation & scale Rigid flow Rotation matrix Scale matrix](https://reader035.vdocuments.mx/reader035/viewer/2022062309/5697bf741a28abf838c7fce5/html5/thumbnails/13.jpg)
Bahadir K. Gunturk 13
RANSAC
Two samplesand their supports
for line-fitting
![Page 14: EE 7700 Geometric Transformations. Bahadir K. Gunturk2 Geometric Transformation translation scale rotation & scale Rigid flow Rotation matrix Scale matrix](https://reader035.vdocuments.mx/reader035/viewer/2022062309/5697bf741a28abf838c7fce5/html5/thumbnails/14.jpg)
Bahadir K. Gunturk 14
Example
Find the perspective parameters
from Hartley & Zisserman
1 2 3
7 8
'1
a x a y ax
a x a y
4 5 6
7 8
'1
a x a y ay
a x a y
![Page 15: EE 7700 Geometric Transformations. Bahadir K. Gunturk2 Geometric Transformation translation scale rotation & scale Rigid flow Rotation matrix Scale matrix](https://reader035.vdocuments.mx/reader035/viewer/2022062309/5697bf741a28abf838c7fce5/html5/thumbnails/15.jpg)
Bahadir K. Gunturk 15
Example
Apply corner detectors to both images
from Hartley & Zisserman
![Page 16: EE 7700 Geometric Transformations. Bahadir K. Gunturk2 Geometric Transformation translation scale rotation & scale Rigid flow Rotation matrix Scale matrix](https://reader035.vdocuments.mx/reader035/viewer/2022062309/5697bf741a28abf838c7fce5/html5/thumbnails/16.jpg)
Bahadir K. Gunturk 16
Example
Find the best match within a search window.
from Hartley & Zisserman
![Page 17: EE 7700 Geometric Transformations. Bahadir K. Gunturk2 Geometric Transformation translation scale rotation & scale Rigid flow Rotation matrix Scale matrix](https://reader035.vdocuments.mx/reader035/viewer/2022062309/5697bf741a28abf838c7fce5/html5/thumbnails/17.jpg)
Bahadir K. Gunturk 17
Example
Initial match results
188 matched features in left image pointing to locations of corresponding right image features
from Hartley & Zisserman
![Page 18: EE 7700 Geometric Transformations. Bahadir K. Gunturk2 Geometric Transformation translation scale rotation & scale Rigid flow Rotation matrix Scale matrix](https://reader035.vdocuments.mx/reader035/viewer/2022062309/5697bf741a28abf838c7fce5/html5/thumbnails/18.jpg)
Bahadir K. Gunturk 18
Example
Inliers and outliers after RANSAC
89 outliers 99 inliersfrom Hartley & Zisserman
![Page 19: EE 7700 Geometric Transformations. Bahadir K. Gunturk2 Geometric Transformation translation scale rotation & scale Rigid flow Rotation matrix Scale matrix](https://reader035.vdocuments.mx/reader035/viewer/2022062309/5697bf741a28abf838c7fce5/html5/thumbnails/19.jpg)
Bahadir K. Gunturk 19
Panoramic Image Reconstruction
Application: Mosaic construction
Find featuresMatch featuresFit parametric model
![Page 20: EE 7700 Geometric Transformations. Bahadir K. Gunturk2 Geometric Transformation translation scale rotation & scale Rigid flow Rotation matrix Scale matrix](https://reader035.vdocuments.mx/reader035/viewer/2022062309/5697bf741a28abf838c7fce5/html5/thumbnails/20.jpg)
EE7730
Stereo Vision
![Page 21: EE 7700 Geometric Transformations. Bahadir K. Gunturk2 Geometric Transformation translation scale rotation & scale Rigid flow Rotation matrix Scale matrix](https://reader035.vdocuments.mx/reader035/viewer/2022062309/5697bf741a28abf838c7fce5/html5/thumbnails/21.jpg)
Bahadir K. Gunturk 21
Pinhole Camera
![Page 22: EE 7700 Geometric Transformations. Bahadir K. Gunturk2 Geometric Transformation translation scale rotation & scale Rigid flow Rotation matrix Scale matrix](https://reader035.vdocuments.mx/reader035/viewer/2022062309/5697bf741a28abf838c7fce5/html5/thumbnails/22.jpg)
Bahadir K. Gunturk 22
Review: Perspective Projection
' ' 'x y f
x y z
![Page 23: EE 7700 Geometric Transformations. Bahadir K. Gunturk2 Geometric Transformation translation scale rotation & scale Rigid flow Rotation matrix Scale matrix](https://reader035.vdocuments.mx/reader035/viewer/2022062309/5697bf741a28abf838c7fce5/html5/thumbnails/23.jpg)
Bahadir K. Gunturk 23
Stereoscene pointscene point
optical centeroptical center
image planeimage plane
p p’
p p’
![Page 24: EE 7700 Geometric Transformations. Bahadir K. Gunturk2 Geometric Transformation translation scale rotation & scale Rigid flow Rotation matrix Scale matrix](https://reader035.vdocuments.mx/reader035/viewer/2022062309/5697bf741a28abf838c7fce5/html5/thumbnails/24.jpg)
Bahadir K. Gunturk 24
Stereo Constraints
X1
Y1
Z1
O1
Image plane
Focal plane
M
p p’
Y2
X2
Z2O2
Epipolar Line
Epipole
![Page 25: EE 7700 Geometric Transformations. Bahadir K. Gunturk2 Geometric Transformation translation scale rotation & scale Rigid flow Rotation matrix Scale matrix](https://reader035.vdocuments.mx/reader035/viewer/2022062309/5697bf741a28abf838c7fce5/html5/thumbnails/25.jpg)
Bahadir K. Gunturk 25
From Geometry to Algebra
O O’
P
pp’
All vectors shown lie on the same plane.
![Page 26: EE 7700 Geometric Transformations. Bahadir K. Gunturk2 Geometric Transformation translation scale rotation & scale Rigid flow Rotation matrix Scale matrix](https://reader035.vdocuments.mx/reader035/viewer/2022062309/5697bf741a28abf838c7fce5/html5/thumbnails/26.jpg)
Bahadir K. Gunturk 26
From Geometry to Algebra
O O’
P
pp’
( , ,1)[ ( ')] 0 with
' ( ', ',1)
T
T
p u vp t Rp
p u v
![Page 27: EE 7700 Geometric Transformations. Bahadir K. Gunturk2 Geometric Transformation translation scale rotation & scale Rigid flow Rotation matrix Scale matrix](https://reader035.vdocuments.mx/reader035/viewer/2022062309/5697bf741a28abf838c7fce5/html5/thumbnails/27.jpg)
Bahadir K. Gunturk 27
Matrix form of cross product
20
0
0
y z z y z
z x x z z x
x y y z y x
a b a b a a
a b a b a b a a b a b
a b a b a a
( ) 0
( ) 0
a a b
b a b
a×b=|a||b|sin(η)u a=axi+ayj+azk
b=bxi+byj+bzk
![Page 28: EE 7700 Geometric Transformations. Bahadir K. Gunturk2 Geometric Transformation translation scale rotation & scale Rigid flow Rotation matrix Scale matrix](https://reader035.vdocuments.mx/reader035/viewer/2022062309/5697bf741a28abf838c7fce5/html5/thumbnails/28.jpg)
Bahadir K. Gunturk 28
The Essential Matrix
( , ,1)[ ( ')] 0 with
' ( ', ',1)
T
T
p u vp t Rp
p u v
' 0Tp Ep
' 0 with Tp Ep E t R Essential matrix
![Page 29: EE 7700 Geometric Transformations. Bahadir K. Gunturk2 Geometric Transformation translation scale rotation & scale Rigid flow Rotation matrix Scale matrix](https://reader035.vdocuments.mx/reader035/viewer/2022062309/5697bf741a28abf838c7fce5/html5/thumbnails/29.jpg)
Bahadir K. Gunturk 29
A Simple Stereo System
Zw=0
LEFT CAMERA
Left image:reference
Right image:target
RIGHT CAMERA
Elevation Zw
disparity
Depth Z
baseline
![Page 30: EE 7700 Geometric Transformations. Bahadir K. Gunturk2 Geometric Transformation translation scale rotation & scale Rigid flow Rotation matrix Scale matrix](https://reader035.vdocuments.mx/reader035/viewer/2022062309/5697bf741a28abf838c7fce5/html5/thumbnails/30.jpg)
Bahadir K. Gunturk 30
Parallel Cameras
ZT
fZxxTlr
OOll OOrr
PP
ppll pprr
TT
ZZ
xxll xxrr
ff
T is the stereo baseline
rlxx
TfZ
rlxxd Disparity:
![Page 31: EE 7700 Geometric Transformations. Bahadir K. Gunturk2 Geometric Transformation translation scale rotation & scale Rigid flow Rotation matrix Scale matrix](https://reader035.vdocuments.mx/reader035/viewer/2022062309/5697bf741a28abf838c7fce5/html5/thumbnails/31.jpg)
Bahadir K. Gunturk 31
Stereo View
Left View Right View
Disparity
![Page 32: EE 7700 Geometric Transformations. Bahadir K. Gunturk2 Geometric Transformation translation scale rotation & scale Rigid flow Rotation matrix Scale matrix](https://reader035.vdocuments.mx/reader035/viewer/2022062309/5697bf741a28abf838c7fce5/html5/thumbnails/32.jpg)
Bahadir K. Gunturk 32
Correlation Approach
For Each point (xl, yl) in the left image, define a window centered at the point
(xl, yl)LEFT IMAGE
![Page 33: EE 7700 Geometric Transformations. Bahadir K. Gunturk2 Geometric Transformation translation scale rotation & scale Rigid flow Rotation matrix Scale matrix](https://reader035.vdocuments.mx/reader035/viewer/2022062309/5697bf741a28abf838c7fce5/html5/thumbnails/33.jpg)
Bahadir K. Gunturk 33
Correlation Approach
… search its corresponding point within a search region in the right image
(xl, yl)RIGHT IMAGE
![Page 34: EE 7700 Geometric Transformations. Bahadir K. Gunturk2 Geometric Transformation translation scale rotation & scale Rigid flow Rotation matrix Scale matrix](https://reader035.vdocuments.mx/reader035/viewer/2022062309/5697bf741a28abf838c7fce5/html5/thumbnails/34.jpg)
Bahadir K. Gunturk 34
Correlation Approach
… the disparity (dx, dy) is the displacement when the correlation is maximum
(xl, yl)dx(xr, yr)RIGHT IMAGE
![Page 35: EE 7700 Geometric Transformations. Bahadir K. Gunturk2 Geometric Transformation translation scale rotation & scale Rigid flow Rotation matrix Scale matrix](https://reader035.vdocuments.mx/reader035/viewer/2022062309/5697bf741a28abf838c7fce5/html5/thumbnails/35.jpg)
Bahadir K. Gunturk 35
Stereo results
Ground truthScene
Data from University of Tsukuba
(Seitz)
![Page 36: EE 7700 Geometric Transformations. Bahadir K. Gunturk2 Geometric Transformation translation scale rotation & scale Rigid flow Rotation matrix Scale matrix](https://reader035.vdocuments.mx/reader035/viewer/2022062309/5697bf741a28abf838c7fce5/html5/thumbnails/36.jpg)
Bahadir K. Gunturk 36
Results with window correlation
Estimated depth of field Ground truth
(Seitz)
![Page 37: EE 7700 Geometric Transformations. Bahadir K. Gunturk2 Geometric Transformation translation scale rotation & scale Rigid flow Rotation matrix Scale matrix](https://reader035.vdocuments.mx/reader035/viewer/2022062309/5697bf741a28abf838c7fce5/html5/thumbnails/37.jpg)
Bahadir K. Gunturk 37
Results with better method
A state of the art methodBoykov et al., Fast Approximate Energy Minimization via Graph Cuts,
International Conference on Computer Vision, September 1999.
Ground truth
(Seitz)
![Page 38: EE 7700 Geometric Transformations. Bahadir K. Gunturk2 Geometric Transformation translation scale rotation & scale Rigid flow Rotation matrix Scale matrix](https://reader035.vdocuments.mx/reader035/viewer/2022062309/5697bf741a28abf838c7fce5/html5/thumbnails/38.jpg)
Bahadir K. Gunturk 38
Applications
courtesy of Sportvision
First-down line
![Page 39: EE 7700 Geometric Transformations. Bahadir K. Gunturk2 Geometric Transformation translation scale rotation & scale Rigid flow Rotation matrix Scale matrix](https://reader035.vdocuments.mx/reader035/viewer/2022062309/5697bf741a28abf838c7fce5/html5/thumbnails/39.jpg)
Bahadir K. Gunturk 39
ApplicationsVirtual advertising
courtesy of Princeton Video Image