lecture4 single view metrology - silvio savarese€¦ · lecture4_single_view_metrology author:...
TRANSCRIPT
![Page 1: lecture4 single view metrology - Silvio Savarese€¦ · lecture4_single_view_metrology Author: Christopher B. Choy Created Date: 1/16/2015 7:14:52 PM](https://reader034.vdocuments.mx/reader034/viewer/2022042908/5f39b375ace0ac5bd810b8fa/html5/thumbnails/1.jpg)
Lecture 4 - Silvio Savarese 14-Jan-15
Professor Silvio Savarese Computational Vision and Geometry
Lab
Lecture 4 Single View Metrology
![Page 2: lecture4 single view metrology - Silvio Savarese€¦ · lecture4_single_view_metrology Author: Christopher B. Choy Created Date: 1/16/2015 7:14:52 PM](https://reader034.vdocuments.mx/reader034/viewer/2022042908/5f39b375ace0ac5bd810b8fa/html5/thumbnails/2.jpg)
Lecture 4 - Silvio Savarese 14-Jan-15
• Review calibration and 2D transformations • Vanishing points and lines • Estimating geometry from a single image • Extensions
Reading: [HZ] Chapter 2 “Projective Geometry and Transformation in 2D” [HZ] Chapter 3 “Projective Geometry and Transformation in 3D” [HZ] Chapter 8 “More Single View Geometry” [Hoeim & Savarese] Chapter 2
Lecture 4 Single View Metrology
![Page 3: lecture4 single view metrology - Silvio Savarese€¦ · lecture4_single_view_metrology Author: Christopher B. Choy Created Date: 1/16/2015 7:14:52 PM](https://reader034.vdocuments.mx/reader034/viewer/2022042908/5f39b375ace0ac5bd810b8fa/html5/thumbnails/3.jpg)
Calibration Problem
jC
ii PMP → !"
#$%
&=→
i
ii v
up
In pixels
[ ]TRKM =
!!!
"
#
$$$
%
& −
=
100
0
cot
sin o
o
v
u
Kθ
β
θαα
World ref. system
![Page 4: lecture4 single view metrology - Silvio Savarese€¦ · lecture4_single_view_metrology Author: Christopher B. Choy Created Date: 1/16/2015 7:14:52 PM](https://reader034.vdocuments.mx/reader034/viewer/2022042908/5f39b375ace0ac5bd810b8fa/html5/thumbnails/4.jpg)
Calibration Problem
jC
ii PMP → !"
#$%
&=→
i
ii v
up
Need at least 6 correspondences11 unknown
[ ]TRKM =
In pixelsWorld ref. system
![Page 5: lecture4 single view metrology - Silvio Savarese€¦ · lecture4_single_view_metrology Author: Christopher B. Choy Created Date: 1/16/2015 7:14:52 PM](https://reader034.vdocuments.mx/reader034/viewer/2022042908/5f39b375ace0ac5bd810b8fa/html5/thumbnails/5.jpg)
Pinhole perspective projectionOnce the camera is calibrated...
[ ]TRKM =
COw
-Internal parameters K are known-R, T are known – but these can only relate C to the calibration rig
Pp
Can I estimate P from the measurement p from a single image?
No - in general ☹ [P can be anywhere along the line defined by C and p]
Line of sight
![Page 6: lecture4 single view metrology - Silvio Savarese€¦ · lecture4_single_view_metrology Author: Christopher B. Choy Created Date: 1/16/2015 7:14:52 PM](https://reader034.vdocuments.mx/reader034/viewer/2022042908/5f39b375ace0ac5bd810b8fa/html5/thumbnails/6.jpg)
C
p
Line of sight
Pinhole perspective projectionRecovering structure from a single view
Ow
P
unknown known Known/Partially known/unknown
![Page 7: lecture4 single view metrology - Silvio Savarese€¦ · lecture4_single_view_metrology Author: Christopher B. Choy Created Date: 1/16/2015 7:14:52 PM](https://reader034.vdocuments.mx/reader034/viewer/2022042908/5f39b375ace0ac5bd810b8fa/html5/thumbnails/7.jpg)
http://www.robots.ox.ac.uk/~vgg/projects/SingleView/models/hut/hutme.wrl
Recovering structure from a single view
![Page 8: lecture4 single view metrology - Silvio Savarese€¦ · lecture4_single_view_metrology Author: Christopher B. Choy Created Date: 1/16/2015 7:14:52 PM](https://reader034.vdocuments.mx/reader034/viewer/2022042908/5f39b375ace0ac5bd810b8fa/html5/thumbnails/8.jpg)
Transformation in 2D
-Isometries
-Similarities
-Affinity
-Projective
![Page 9: lecture4 single view metrology - Silvio Savarese€¦ · lecture4_single_view_metrology Author: Christopher B. Choy Created Date: 1/16/2015 7:14:52 PM](https://reader034.vdocuments.mx/reader034/viewer/2022042908/5f39b375ace0ac5bd810b8fa/html5/thumbnails/9.jpg)
Transformation in 2D
Isometries:!!!
"
#
$$$
%
&
=
!!!
"
#
$$$
%
&
!"
#$%
&=
!!!
"
#
$$$
%
&
1yx
H1yx
10tR
1'y'x
e
- Preserve distance (areas)- 3 DOF- Regulate motion of rigid object
[Euclideans]
[Eq. 4]
![Page 10: lecture4 single view metrology - Silvio Savarese€¦ · lecture4_single_view_metrology Author: Christopher B. Choy Created Date: 1/16/2015 7:14:52 PM](https://reader034.vdocuments.mx/reader034/viewer/2022042908/5f39b375ace0ac5bd810b8fa/html5/thumbnails/10.jpg)
Transformation in 2D
Similarities:!!!
"
#
$$$
%
&
=
!!!
"
#
$$$
%
&
!"
#$%
&=
!!!
"
#
$$$
%
&
1yx
H1yx
10tRs
1'y'x
s
- Preserve- ratio of lengths- angles
-4 DOF
[Eq. 5]
![Page 11: lecture4 single view metrology - Silvio Savarese€¦ · lecture4_single_view_metrology Author: Christopher B. Choy Created Date: 1/16/2015 7:14:52 PM](https://reader034.vdocuments.mx/reader034/viewer/2022042908/5f39b375ace0ac5bd810b8fa/html5/thumbnails/11.jpg)
Transformation in 2D
Affinities:!!!
"
#
$$$
%
&
=
!!!
"
#
$$$
%
&
!"
#$%
&=
!!!
"
#
$$$
%
&
1yx
H1yx
10tA
1'y'x
a
!"
#$%
&=
2221
1211
aaaa
A )(RD)(R)(R φφθ ⋅⋅−⋅= !"
#$%
&=
y
x
s00s
D
[Eq. 6]
[Eq. 7]
![Page 12: lecture4 single view metrology - Silvio Savarese€¦ · lecture4_single_view_metrology Author: Christopher B. Choy Created Date: 1/16/2015 7:14:52 PM](https://reader034.vdocuments.mx/reader034/viewer/2022042908/5f39b375ace0ac5bd810b8fa/html5/thumbnails/12.jpg)
Transformation in 2D
Affinities:!!!
"
#
$$$
%
&
=
!!!
"
#
$$$
%
&
!"
#$%
&=
!!!
"
#
$$$
%
&
1yx
H1yx
10tA
1'y'x
a
!"
#$%
&=
2221
1211
aaaa
A )(RD)(R)(R φφθ ⋅⋅−⋅= !"
#$%
&=
y
x
s00s
D
-Preserve:- Parallel lines- Ratio of areas- Ratio of lengths on collinear lines- others…
- 6 DOF
[Eq. 6]
[Eq. 7]
![Page 13: lecture4 single view metrology - Silvio Savarese€¦ · lecture4_single_view_metrology Author: Christopher B. Choy Created Date: 1/16/2015 7:14:52 PM](https://reader034.vdocuments.mx/reader034/viewer/2022042908/5f39b375ace0ac5bd810b8fa/html5/thumbnails/13.jpg)
Transformation in 2D
Projective:!!!
"
#
$$$
%
&
=
!!!
"
#
$$$
%
&
!"
#$%
&=
!!!
"
#
$$$
%
&
1yx
H1yx
bvtA
1'y'x
p
- 8 DOF- Preserve:
- cross ratio of 4 collinear points- collinearity - and a few others…
[Eq. 8]
![Page 14: lecture4 single view metrology - Silvio Savarese€¦ · lecture4_single_view_metrology Author: Christopher B. Choy Created Date: 1/16/2015 7:14:52 PM](https://reader034.vdocuments.mx/reader034/viewer/2022042908/5f39b375ace0ac5bd810b8fa/html5/thumbnails/14.jpg)
The cross ratio
1423
2413
PPPPPPPP
−−
−−
The cross-ratio of 4 collinear points
Can permute the point ordering
!!!!
"
#
$$$$
%
&
=
1i
i
i
i ZYX
P
3421
2431
PPPPPPPP
−−
−−
P1P2
P3P4
[Eq. 9]
![Page 15: lecture4 single view metrology - Silvio Savarese€¦ · lecture4_single_view_metrology Author: Christopher B. Choy Created Date: 1/16/2015 7:14:52 PM](https://reader034.vdocuments.mx/reader034/viewer/2022042908/5f39b375ace0ac5bd810b8fa/html5/thumbnails/15.jpg)
Lecture 4 - Silvio Savarese 14-Jan-15
• Review calibration and 2D transformations • Vanishing points and lines • Estimating geometry from a single image • Extensions
Reading: [HZ] Chapter 2 “Projective Geometry and Transformation in 2D” [HZ] Chapter 3 “Projective Geometry and Transformation in 3D” [HZ] Chapter 8 “More Single View Geometry” [Hoeim & Savarese] Chapter 2
Lecture 4 Single View Metrology
![Page 16: lecture4 single view metrology - Silvio Savarese€¦ · lecture4_single_view_metrology Author: Christopher B. Choy Created Date: 1/16/2015 7:14:52 PM](https://reader034.vdocuments.mx/reader034/viewer/2022042908/5f39b375ace0ac5bd810b8fa/html5/thumbnails/16.jpg)
Lines in a 2D plane
0cbyax =++
-c/b
-a/b
!!!
"
#
$$$
%
&
=
cba
l
If x = [ x1, x2]T ∈ l 0cba
1xx T
2
1
=
!!!
"
#
$$$
%
&
!!!
"
#
$$$
%
&
l
x
y
[Eq. 10]
![Page 17: lecture4 single view metrology - Silvio Savarese€¦ · lecture4_single_view_metrology Author: Christopher B. Choy Created Date: 1/16/2015 7:14:52 PM](https://reader034.vdocuments.mx/reader034/viewer/2022042908/5f39b375ace0ac5bd810b8fa/html5/thumbnails/17.jpg)
Lines in a 2D plane
Intersecting lines
llx !×= l
l!
Proof
lll ⊥"×lll !⊥!×
0l)ll( =⋅"×→0l)ll( =!⋅!×→
lx∈→
x
lx !∈→
→ x is the intersecting point
x
y
x
[Eq. 12][Eq. 13]
[Eq. 11]
![Page 18: lecture4 single view metrology - Silvio Savarese€¦ · lecture4_single_view_metrology Author: Christopher B. Choy Created Date: 1/16/2015 7:14:52 PM](https://reader034.vdocuments.mx/reader034/viewer/2022042908/5f39b375ace0ac5bd810b8fa/html5/thumbnails/18.jpg)
2D Points at infinity (ideal points)
0x,xxx
x 3
3
2
1
≠
"""
#
$
%%%
&
'
=!!!
"
#
$$$
%
&
=
cba
l
!!!
"
#
$$$
%
&
'
='
cba
l ''
!!!
"
#
$$$
%
&
−∝)×→
0ab
llLet’s intersect two parallel lines:
• In Euclidian coordinates this point is at infinity• Agree with the general idea of two lines intersecting at infinity
l
l!!!!
"
#
$$$
%
&
=∞
0''
2
1
xx
x
'/'/ baba −=−
∞= x Eq.13]
![Page 19: lecture4 single view metrology - Silvio Savarese€¦ · lecture4_single_view_metrology Author: Christopher B. Choy Created Date: 1/16/2015 7:14:52 PM](https://reader034.vdocuments.mx/reader034/viewer/2022042908/5f39b375ace0ac5bd810b8fa/html5/thumbnails/19.jpg)
2D Points at infinity (ideal points)
0x,xxx
x 3
3
2
1
≠
"""
#
$
%%%
&
'
=!!!
"
#
$$$
%
&
=
cba
l
!!!
"
#
$$$
%
&
'
='
cba
l ''
[ ] 00
lT =
!!!
"
#
$$$
%
&
−=∞ ab
cbax
Note: the line l = [a b c]T pass trough the ideal point
So does the line l’ since a b’ = a’ b
l
l!
'/'/ baba −=−
∞x
[Eq. 15]
![Page 20: lecture4 single view metrology - Silvio Savarese€¦ · lecture4_single_view_metrology Author: Christopher B. Choy Created Date: 1/16/2015 7:14:52 PM](https://reader034.vdocuments.mx/reader034/viewer/2022042908/5f39b375ace0ac5bd810b8fa/html5/thumbnails/20.jpg)
Lines infinity ∞lSet of ideal points lies on a line called the line at infinityHow does it look like?
∞l
!!!
"
#
$$$
%
&
=∞
100
l
T
2
1
0xx
!!!
"
#
$$$
%
&
0100=
!!!
"
#
$$$
%
&Indeed:
A line at infinity can be thought of the set of “directions” of lines in the plane
'∞x
!!!
"
#
$$$
%
&
−=
0''ab
!!!
"
#
$$$
%
&
−=
0''''ab
''∞x
![Page 21: lecture4 single view metrology - Silvio Savarese€¦ · lecture4_single_view_metrology Author: Christopher B. Choy Created Date: 1/16/2015 7:14:52 PM](https://reader034.vdocuments.mx/reader034/viewer/2022042908/5f39b375ace0ac5bd810b8fa/html5/thumbnails/21.jpg)
Projective transformation of a point at infinity
pHp ='
!"
#$%
&=
bvtA
H
?=∞pH!!!
"
#
$$$
%
&
=
!!!
"
#
$$$
%
&
!"
#$%
&=
z
y
x
ppp
bvtA
'''
011
is it a point at infinity?
…no!
?=∞pHA!!!
"
#
$$$
%
&
=
!!!
"
#
$$$
%
&
!"
#$%
&=
0''
011
0 y
x
pp
btA
An affine transformation of a point at infinity is still a point at infinity
[Eq. 17]
[Eq. 18]
![Page 22: lecture4 single view metrology - Silvio Savarese€¦ · lecture4_single_view_metrology Author: Christopher B. Choy Created Date: 1/16/2015 7:14:52 PM](https://reader034.vdocuments.mx/reader034/viewer/2022042908/5f39b375ace0ac5bd810b8fa/html5/thumbnails/22.jpg)
Projective transformation of a line (in 2D)
lHl T−="
!"
#$%
&=
bvtA
H
?lH T =∞−
!!!
"
#
$$$
%
&
=
!!!
"
#
$$$
%
&
!"
#$%
&=
−
btt
100
bvtA
y
xT
is it a line at infinity?
…no!
?lH TA =∞−
!!!
"
#
$$$
%
&
=
!!!
"
#
$$$
%
&
!"
#$%
&
−=
!!!
"
#
$$$
%
&
!"
#$%
&=
−
−−
100
100
10
100
10 TT
TT
AtAtA
[Eq. 19]
[Eq. 20]
[Eq. 21]
![Page 23: lecture4 single view metrology - Silvio Savarese€¦ · lecture4_single_view_metrology Author: Christopher B. Choy Created Date: 1/16/2015 7:14:52 PM](https://reader034.vdocuments.mx/reader034/viewer/2022042908/5f39b375ace0ac5bd810b8fa/html5/thumbnails/23.jpg)
Points and planes in 3D
!!!!
"
#
$$$$
%
&
=
1xxx
x3
2
1
!!!!
"
#
$$$$
%
&
=Π
dcba
0dczbyax =+++
x
y
z
x∈ Π↔ xTΠ = 0
How about lines in 3D? • Lines have 4 degrees of freedom - hard to represent in 3D-space• Can be defined as intersection of 2 planes
[Eq. 23][Eq. 22]
![Page 24: lecture4 single view metrology - Silvio Savarese€¦ · lecture4_single_view_metrology Author: Christopher B. Choy Created Date: 1/16/2015 7:14:52 PM](https://reader034.vdocuments.mx/reader034/viewer/2022042908/5f39b375ace0ac5bd810b8fa/html5/thumbnails/24.jpg)
Points at infinity in 3DPoints where parallel lines intersect in 3D
!!!!
"
#
$$$$
%
&
=∞
03
2
1
xxx
xworld
Parallel lines
point at infinity
![Page 25: lecture4 single view metrology - Silvio Savarese€¦ · lecture4_single_view_metrology Author: Christopher B. Choy Created Date: 1/16/2015 7:14:52 PM](https://reader034.vdocuments.mx/reader034/viewer/2022042908/5f39b375ace0ac5bd810b8fa/html5/thumbnails/25.jpg)
Vanishing pointsThe projective projection of a point at infinity into the image plane defines a vanishing point.
M
!!!!
"
#
$$$$
%
&
=∞
03
2
1
xxx
x
!!!
"
#
$$$
%
&
=∞
3
2
1
ppp
p
world
p∞
Parallel lines
point at infinity
= direction of the line in 3D
![Page 26: lecture4 single view metrology - Silvio Savarese€¦ · lecture4_single_view_metrology Author: Christopher B. Choy Created Date: 1/16/2015 7:14:52 PM](https://reader034.vdocuments.mx/reader034/viewer/2022042908/5f39b375ace0ac5bd810b8fa/html5/thumbnails/26.jpg)
[ ]!!!
"
#
$$$
%
&
=
!!!!
"
#
$$$$
%
&
== ∞
cba
cba
K0IKXMv
0
dv K=
d = direction of the line = [a, b, c]T
d
C
v
!!!!
"
#
$$$$
%
&
=∞
0cba
x M
vvd
1
1
−
−
=KK
[Eq. 24] [Eq. 25]
Proof:
Vanishing points and directions
![Page 27: lecture4 single view metrology - Silvio Savarese€¦ · lecture4_single_view_metrology Author: Christopher B. Choy Created Date: 1/16/2015 7:14:52 PM](https://reader034.vdocuments.mx/reader034/viewer/2022042908/5f39b375ace0ac5bd810b8fa/html5/thumbnails/27.jpg)
Vanishing (horizon) line
horizon
∞
−
= lHlT
Phor
∞l
π
Projective transformation M
Image
[Eq. 26]
![Page 28: lecture4 single view metrology - Silvio Savarese€¦ · lecture4_single_view_metrology Author: Christopher B. Choy Created Date: 1/16/2015 7:14:52 PM](https://reader034.vdocuments.mx/reader034/viewer/2022042908/5f39b375ace0ac5bd810b8fa/html5/thumbnails/28.jpg)
Are these two lines parallel or not?
- Recognize the horizon line- Measure if the 2 lines meet at the horizon- if yes, these 2 lines are // in 3D
•Recognition helps reconstruction!•Humans have learnt this
![Page 29: lecture4 single view metrology - Silvio Savarese€¦ · lecture4_single_view_metrology Author: Christopher B. Choy Created Date: 1/16/2015 7:14:52 PM](https://reader034.vdocuments.mx/reader034/viewer/2022042908/5f39b375ace0ac5bd810b8fa/html5/thumbnails/29.jpg)
Vanishing points and planes
C
n
horizTK ln =
…
horizl
πl∞
[Eq. 27]
![Page 30: lecture4 single view metrology - Silvio Savarese€¦ · lecture4_single_view_metrology Author: Christopher B. Choy Created Date: 1/16/2015 7:14:52 PM](https://reader034.vdocuments.mx/reader034/viewer/2022042908/5f39b375ace0ac5bd810b8fa/html5/thumbnails/30.jpg)
Planes at infinity
x
y
z
• A set of 2 or more lines at infinity defines the plane at infinity
∞Π…
!!!!
"
#
$$$$
%
&
=Π∞
1000
plane at infinity …
∞Π
• Parallel planes intersect at infinity in a common line – the line at infinity
![Page 31: lecture4 single view metrology - Silvio Savarese€¦ · lecture4_single_view_metrology Author: Christopher B. Choy Created Date: 1/16/2015 7:14:52 PM](https://reader034.vdocuments.mx/reader034/viewer/2022042908/5f39b375ace0ac5bd810b8fa/html5/thumbnails/31.jpg)
Angle between 2 vanishing points
2T21
T1
2T1
vvvv
vvcos
ωω
ωθ =
C
d1v2
v1d2
θ
If 90=θ 0vv 2T1 =→ ω
1T )KK( −=ω[Eq. 28]
x1∞
x2∞
[Eq. 29]Scalar equation
![Page 32: lecture4 single view metrology - Silvio Savarese€¦ · lecture4_single_view_metrology Author: Christopher B. Choy Created Date: 1/16/2015 7:14:52 PM](https://reader034.vdocuments.mx/reader034/viewer/2022042908/5f39b375ace0ac5bd810b8fa/html5/thumbnails/32.jpg)
Projective transformation of ∞Ω
1)( −= KKT
It is not function of R, T
!!!
"
#
$$$
%
&
=
654
532
421
ωωω
ωωω
ωωω
ω symmetric and known up scale
02 =ω zero-skew 31 ωω = square pixel02 =ω
1.
2.
3. 4.
M = K R T!"
#$=M −TΩ∞M
−1
Absolute conic
[Eq. 30]
![Page 33: lecture4 single view metrology - Silvio Savarese€¦ · lecture4_single_view_metrology Author: Christopher B. Choy Created Date: 1/16/2015 7:14:52 PM](https://reader034.vdocuments.mx/reader034/viewer/2022042908/5f39b375ace0ac5bd810b8fa/html5/thumbnails/33.jpg)
Why is this useful?
• To calibrate the camera• To estimate the geometry of the 3D world
![Page 34: lecture4 single view metrology - Silvio Savarese€¦ · lecture4_single_view_metrology Author: Christopher B. Choy Created Date: 1/16/2015 7:14:52 PM](https://reader034.vdocuments.mx/reader034/viewer/2022042908/5f39b375ace0ac5bd810b8fa/html5/thumbnails/34.jpg)
Lecture 4 - Silvio Savarese 14-Jan-15
• Review calibration • Vanishing points and line • Estimating geometry from a single image • Extensions
Reading: [HZ] Chapter 2 “Projective Geometry and Transformation in 3D” [HZ] Chapter 3 “Projective Geometry and Transformation in 3D” [HZ] Chapter 8 “More Single View Geometry” [Hoeim & Savarese] Chapter 2
Lecture 4 Single View Metrology
![Page 35: lecture4 single view metrology - Silvio Savarese€¦ · lecture4_single_view_metrology Author: Christopher B. Choy Created Date: 1/16/2015 7:14:52 PM](https://reader034.vdocuments.mx/reader034/viewer/2022042908/5f39b375ace0ac5bd810b8fa/html5/thumbnails/35.jpg)
90=θ
0vv 2T1 =ω
v1v2
1T )KK( −=ωDo we have enough constraints to estimate K?K has 5 degrees of freedom and Eq.29 is a scalar equation ☹
Single view calibration - example
![Page 36: lecture4 single view metrology - Silvio Savarese€¦ · lecture4_single_view_metrology Author: Christopher B. Choy Created Date: 1/16/2015 7:14:52 PM](https://reader034.vdocuments.mx/reader034/viewer/2022042908/5f39b375ace0ac5bd810b8fa/html5/thumbnails/36.jpg)
Single view calibration - examplev2
v3
0vv 2T1 =ω
0vv 3T1 =ω
0vv 3T2 =ω
!!!
"
#
$$$
%
&
=
654
532
421
ωωω
ωωω
ωωω
ω
31 ωω =
02 =ω ! Compute :ω
known up to scale
1T )KK( −=ω K(Cholesky factorization; HZ pag 582)
v2
Once ω is calculated, we get K:
[Eqs
. 31]
![Page 37: lecture4 single view metrology - Silvio Savarese€¦ · lecture4_single_view_metrology Author: Christopher B. Choy Created Date: 1/16/2015 7:14:52 PM](https://reader034.vdocuments.mx/reader034/viewer/2022042908/5f39b375ace0ac5bd810b8fa/html5/thumbnails/37.jpg)
Single view reconstruction - example
lh
knownK horizTK ln =→ = Scene plane orientation in
the camera reference system
Select orientation discontinuities
![Page 38: lecture4 single view metrology - Silvio Savarese€¦ · lecture4_single_view_metrology Author: Christopher B. Choy Created Date: 1/16/2015 7:14:52 PM](https://reader034.vdocuments.mx/reader034/viewer/2022042908/5f39b375ace0ac5bd810b8fa/html5/thumbnails/38.jpg)
Single view reconstruction - example
Recover the structure within the camera reference systemNotice: the actual scale of the scene is NOT recovered
C
•Recognition helps reconstruction!•Humans have learnt this
![Page 39: lecture4 single view metrology - Silvio Savarese€¦ · lecture4_single_view_metrology Author: Christopher B. Choy Created Date: 1/16/2015 7:14:52 PM](https://reader034.vdocuments.mx/reader034/viewer/2022042908/5f39b375ace0ac5bd810b8fa/html5/thumbnails/39.jpg)
Lecture 4 - Silvio Savarese 14-Jan-15
• Review calibration • Vanishing points and lines • Estimating geometry from a single image • Extensions
Reading: [HZ] Chapter 2 “Projective Geometry and Transformation in 3D” [HZ] Chapter 3 “Projective Geometry and Transformation in 3D” [HZ] Chapter 8 “More Single View Geometry” [Hoeim & Savarese] Chapter 2
Lecture 4 Single View Metrology
![Page 40: lecture4 single view metrology - Silvio Savarese€¦ · lecture4_single_view_metrology Author: Christopher B. Choy Created Date: 1/16/2015 7:14:52 PM](https://reader034.vdocuments.mx/reader034/viewer/2022042908/5f39b375ace0ac5bd810b8fa/html5/thumbnails/40.jpg)
Criminisi & Zisserman, 99
http://www.robots.ox.ac.uk/~vgg/projects/SingleView/models/merton/merton.wrl
![Page 41: lecture4 single view metrology - Silvio Savarese€¦ · lecture4_single_view_metrology Author: Christopher B. Choy Created Date: 1/16/2015 7:14:52 PM](https://reader034.vdocuments.mx/reader034/viewer/2022042908/5f39b375ace0ac5bd810b8fa/html5/thumbnails/41.jpg)
Criminisi & Zisserman, 99
http://www.robots.ox.ac.uk/~vgg/projects/SingleView/models/merton/merton.wrl
![Page 42: lecture4 single view metrology - Silvio Savarese€¦ · lecture4_single_view_metrology Author: Christopher B. Choy Created Date: 1/16/2015 7:14:52 PM](https://reader034.vdocuments.mx/reader034/viewer/2022042908/5f39b375ace0ac5bd810b8fa/html5/thumbnails/42.jpg)
La Trinita' (1426) Firenze, Santa Maria Novella; by Masaccio (1401-1428)
![Page 43: lecture4 single view metrology - Silvio Savarese€¦ · lecture4_single_view_metrology Author: Christopher B. Choy Created Date: 1/16/2015 7:14:52 PM](https://reader034.vdocuments.mx/reader034/viewer/2022042908/5f39b375ace0ac5bd810b8fa/html5/thumbnails/43.jpg)
La Trinita' (1426) Firenze, Santa Maria Novella; by Masaccio (1401-1428)
![Page 44: lecture4 single view metrology - Silvio Savarese€¦ · lecture4_single_view_metrology Author: Christopher B. Choy Created Date: 1/16/2015 7:14:52 PM](https://reader034.vdocuments.mx/reader034/viewer/2022042908/5f39b375ace0ac5bd810b8fa/html5/thumbnails/44.jpg)
http://www.robots.ox.ac.uk/~vgg/projects/SingleView/models/hut/hutme.wrl
![Page 45: lecture4 single view metrology - Silvio Savarese€¦ · lecture4_single_view_metrology Author: Christopher B. Choy Created Date: 1/16/2015 7:14:52 PM](https://reader034.vdocuments.mx/reader034/viewer/2022042908/5f39b375ace0ac5bd810b8fa/html5/thumbnails/45.jpg)
Manually select:• Vanishing points and lines;• Planar surfaces;• Occluding boundaries;• Etc..
Single view reconstruction - drawbacks
![Page 46: lecture4 single view metrology - Silvio Savarese€¦ · lecture4_single_view_metrology Author: Christopher B. Choy Created Date: 1/16/2015 7:14:52 PM](https://reader034.vdocuments.mx/reader034/viewer/2022042908/5f39b375ace0ac5bd810b8fa/html5/thumbnails/46.jpg)
Automatic Photo Pop-upHoiem et al, 05
![Page 47: lecture4 single view metrology - Silvio Savarese€¦ · lecture4_single_view_metrology Author: Christopher B. Choy Created Date: 1/16/2015 7:14:52 PM](https://reader034.vdocuments.mx/reader034/viewer/2022042908/5f39b375ace0ac5bd810b8fa/html5/thumbnails/47.jpg)
Automatic Photo Pop-upHoiem et al, 05…
![Page 48: lecture4 single view metrology - Silvio Savarese€¦ · lecture4_single_view_metrology Author: Christopher B. Choy Created Date: 1/16/2015 7:14:52 PM](https://reader034.vdocuments.mx/reader034/viewer/2022042908/5f39b375ace0ac5bd810b8fa/html5/thumbnails/48.jpg)
Automatic Photo Pop-upHoiem et al, 05…
http://www.cs.uiuc.edu/homes/dhoiem/projects/software.html
Software:
![Page 49: lecture4 single view metrology - Silvio Savarese€¦ · lecture4_single_view_metrology Author: Christopher B. Choy Created Date: 1/16/2015 7:14:52 PM](https://reader034.vdocuments.mx/reader034/viewer/2022042908/5f39b375ace0ac5bd810b8fa/html5/thumbnails/49.jpg)
Training
Image Depth
Prediction
Planar Surface Segmentation
Plane Parameter MRF
Connectivity Co-Planarity
Make3D
youtube
Saxena, Sun, Ng, 05…
![Page 50: lecture4 single view metrology - Silvio Savarese€¦ · lecture4_single_view_metrology Author: Christopher B. Choy Created Date: 1/16/2015 7:14:52 PM](https://reader034.vdocuments.mx/reader034/viewer/2022042908/5f39b375ace0ac5bd810b8fa/html5/thumbnails/50.jpg)
A software: Make3D“Convert your image into 3d model”
Single Image Depth ReconstructionSaxena, Sun, Ng, 05…
http://make3d.stanford.edu/http://make3d.cs.cornell.edu/
![Page 51: lecture4 single view metrology - Silvio Savarese€¦ · lecture4_single_view_metrology Author: Christopher B. Choy Created Date: 1/16/2015 7:14:52 PM](https://reader034.vdocuments.mx/reader034/viewer/2022042908/5f39b375ace0ac5bd810b8fa/html5/thumbnails/51.jpg)
Coherent object detection and scene layout estimation from a single image
Y. Bao, M. Sun, S. Savarese, CVPR 2010, BMVC 2010
Y. BaoM. Sun
![Page 52: lecture4 single view metrology - Silvio Savarese€¦ · lecture4_single_view_metrology Author: Christopher B. Choy Created Date: 1/16/2015 7:14:52 PM](https://reader034.vdocuments.mx/reader034/viewer/2022042908/5f39b375ace0ac5bd810b8fa/html5/thumbnails/52.jpg)
Next lecture:
Multi-view geometry (epipolar geometry)