eecs 442 computer vision single view · pdf fileeecs 442 – computer vision single view...

EECS 442 – Computer vision

Single view metrology

• Review calibration

• Lines and planes at infinity

• Absolute conic

• Estimating geometry from a single image

• Extensions

Reading: [HZ] Chapters 2,3,8

Calibration Problem

jC

ii PMP

i

i

iv

up

In pixels

TRKM

100

0

cot

sino

o

v

u

K

World ref. system

Calibration Problem

jC

ii PMP

i

i

iv

up

Need at least 6 correspondences

11 unknown

TRKM

In pixels World ref. system

Pinhole perspective projection Once the camera is calibrated...

TRKM

C

Ow

-Internal parameters K are known

-R, T are known – but these can only relate C to the calibration rig

P p

Can I estimate P from the measurement p from a single image?

No - in general [P can be anywhere along the line defined by C and p]

Pinhole perspective projection Recovering structure from a single view

C

Ow

P p

unknown known Known/

Partially known/

unknown

http://www.robots.ox.ac.uk/~vgg/projects/SingleView/models/hut/hutme.wrl

Recovering structure from a single view



• Absolute conic


• Examples

Lines in a 2D plane

0cbyax

-c/b

-a/b

c

b

a

l

If x = [ x1, x2]T l 0

c

b

a

1

x

xT

2

1

l

x

y

Lines in a 2D plane

Intersecting lines

llx l

l

Proof

lll

lll

0l)ll(

0l)ll(

lx

x

lx

x is the intersecting point

x

y

x

Points and planes in 3D

1

x

x

x

x3

2

1

d

c

b

a

0dczbyax

x

y

z

0xx T

How about lines in 3D? • Lines have 4 degrees of freedom - hard to represent in 3D-space

• Can be defined as intersection of 2 planes

2D Points at infinity (ideal points)

0x,

x

x

x

x 3

3

2

1

c

b

a

l

c

b

a

l '

'

0

1

/

)(

ab

ccllLet’s intersect two parallel lines:

Agree with the general idea of two lines intersecting at infinity

l

l

0

2

1

x

x

x

'/'/ baba

Lines infinity l

Set of ideal points lies on a line called the line at infinity

How does it look like?

l

1

0

0

l

T

2

1

0

x

x

0

1

0

0

Indeed:

Projective transformation of a point at infinity

pHp '

bv

tAH

?pH

z

y

x

p

p

p

bv

tA

'

'

'

0

1

1

is it a point at infinity?

…no!

?pH A

0

'

'

0

1

1

0y

x

p

p

b

tA

An affine

transformation

of a point

at infinity is

still a point at

infinity

Projective transformation of a line (in 2D)

lHl T

bv

tAH

?lH T

b

t

t

1

0

0

bv

tAy

xT

is it a line at infinity?

…no!

?lH T

A

1

0

0

1

0

0

1

0

1

0

0

10 TT

TT

At

AtA

The horizon line

horizon

- Recognize the horizon line

- Measure if the 2 lines meet

at the horizon

- if yes, these 2 lines are // in 3D •Recognition helps reconstruction!

•Humans have learnt this

Are these two lines parallel or not?

lHl T

hor

Vanishing points (= ideal points in 2D)

Points where parallel lines

intersect in 3D

Vanishing points and their image

TRKM dv K

d=direction of the line

d

C

v

Vanishing points - example

C2

v2

C1

v1

R,T

star

1

1

1

1

1K

K

v

vd

2

1

2

1

2K

K

v

vd

21 R dd R

v1, v2: measurements

K = known and constant

Can I compute R?

1d2d

In 2D

1

1

1

1

1K

K

v

vd

2

1

2

1

2K

K

v

vd

R

Planes at infinity & vanishing lines

1

0

0

0

x

y

z

• Parallel planes intersect at the plane at infinity

• 2 planes are parallel iff their intersections

is a line that belongs to

• Parallel planes intersect the plane at infinity in a

common line – the vanishing line (horizon)

Vanishing lines and their images

C

n

Parallel planes intersect the plane at infinity

in a common line – the vanishing line (horizon)

horiz

TK ln

horizl



• Absolute conic


• Examples

Conics in 2D

- If a point x C

0feydxcybxyax 22

In homogeneous coordinates:

0xCxT

f2/e2/d

2/ec2/b

2/d2/ba

C

1

x

x

x 2

1

0xCxT

- If a line l is tangent to a conic in x xCl

1T PCPC - Projective transformation of conics:

Circular points

l

0

i

1

pi

0

i

1

p j

Circular points (point at infinity)

Circular points are fixed under

similarity transformation

0

i

1

0

i

1

100

tcosssins

tsinscoss

pH y

x

is

i2=-1

Conic C

Circular points define a degenerate

conic called ; any C

0x

0xx

3

2

2

2

1

Cx

000

010

001

C0xCxT

l

0

i

1

pi

0

i

1

p j

Circular points (point at infinity)

is fixed under similarity transformation

C

In 3D: absolute conic

Cis a

Any x satisfies:

0

1

1

1

0xxT

0x

0xxx

4

2

3

2

2

2

1

is fixed under similarity transformation

Projective transformation of

1T )KK(

It is not function of R, T

654

532

421

symmetric

02 zero-skew 31

square pixel 02

1.

2.

3. 4.

TRKP

Projective transformation of

1T )KK(

Why this is useful?



• Absolute conic


• Examples

Angle between 2 vanishing points

2

T

21

T

1

2

T

1

vvvv

vvcos

C

d1 v1

dv Kv2

d2

If 90 0vv 2

T

1

Angle between 2 scene lines

90

0vv 2

T

1

v1 v2

1T )KK(

Constraint on K

Single view calibration - example

v2

v3

0vv 2

T

1

0vv 3

T

1

0vv 3

T

2

654

532

421

31

02

Compute :6 unknown

5 constraints

known up to scale

1T )KK( K(Cholesky factorization; HZ pag 582)

v2

Once is calculated, we get K:

Single view reconstruction - example

lh

known Khoriz

TK ln = Scene plane orientation in

the camera reference system

Select orientation discontinuities

Single view reconstruction - example

Recover the structure within the camera reference system

Notice: the actual scale of the scene is NOT recovered

C

- Recognize the horizon line

- Measure if the 2 lines meet

at the horizon

- if yes, these 2 lines are // in 3D

•Recognition helps reconstruction!

•Humans have learnt this

Are these two lines parallel or not?

Criminisi & Zisserman, 99

http://www.robots.ox.ac.uk/~vgg/projects/SingleView/models/merton/merton.wrl

La Trinita' (1426)

Firenze, Santa Maria

Novella; by Masaccio

(1401-1428)

http://www.robots.ox.ac.uk/~vgg/projects/SingleView/models/hut/hutme.wrl

Manually select: • Vanishing points and lines;

• Planar surfaces;

• Occluding boundaries;

• Etc..

Single view reconstruction - drawbacks

Automatic Photo Pop-up

Hoiem et al, 05


Hoiem et al, 05…


Hoiem et al, 05…

http://www.cs.uiuc.edu/homes/dhoiem/projects/software.html

Software:

Training

Image Depth

Prediction

Planar Surface

Segmentation

Plane Parameter MRF

Connectivity Co-Planarity

Make3D

youtube

Saxena, Sun, Ng, 05…

http://www.youtube.com/watch?v=33oHekQipTg

A software: Make3D

“Convert your image into 3d model”

Single Image Depth Reconstruction Saxena, Sun, Ng, 05…

http://make3d.stanford.edu/

http://make3d.stanford.edu/images/view3D/185

http://make3d.stanford.edu/images/view3D/931?noforward=true



http://make3d.stanford.edu/images/view3D/931?noforward=true

Coherent object detection and scene

layout estimation from a single image Y. Bao, M. Sun, S. Savarese, CVPR 2010,

BMVC 2010

Y. Bao M. Sun

Next lecture:

Multi-view geometry (epipolar geometry)

eecs 442 computer vision single view · pdf fileeecs 442 – computer vision single view...

Documents