linear cameras what is a perspective camera? reguli linear congruences direct and inverse...

Linear cameras

• What is a perspective camera?• Reguli• Linear congruences• Direct and inverse projections• Multi-view geometry

Présentations mercredi 26 mai de 9h30 à midi, salle U/V

Planches :

– http://www.di.ens.fr/~ponce/geomvis/lect6.ppt

– http://www.di.ens.fr/~ponce/geomvis/lect6.pdf

П1

Chasles’ absolute conic: x12+x2

2+x32 = 0, x4 = 0.

Kruppa (1913); Maybank & Faugeras (1992)

Triggs (1997);Pollefeys et al. (1998,2002)

, u0, v0

The absolute quadric u0 = v0 = 0The absolute quadratic complex 2 = 2, = 0

u0

v0

kl

f

x’ ¼ P ( H H-1 ) xH = [ X y ]

Relation between K, , and *

» = u1 »1 + u2 »2 + u3 »3

Line bundles

c

x

3

1

2

» = u1 »1 + u2 »2 + u3 »3

y = u1 y1 + u2 y2 + u3 y3

y2

c

r

x

y

y1

3

1

2

y3

Line bundles

» = X u , where X2R6£3, u2R3

y = Y u , where Y2R4£3, u2R3

y2

c

r

x

y

y1

3

1

2

y3

Line bundles

u = Yzy y = Yz

y [(c Ç x) Æ r]

y2

c

r

x

y

y1

3

1

2

y3

Line bundles

Note:

u = Yzy y = Yz

y [(c Ç x) Æ r]

y2

c

r

x

y

y1

3

1

2

y3

Line bundles

Note:(c Ç x) Æ r = [c x – x c ] rT T

u = Yzy y = Yz

y [(c Ç x) Æ r] = P x when z = c

y2

c

r

x

y

y1

3

1

2

y3

Line bundles

c

r

x

y

u ¼ P x ¼ P*

p ¼ P T ¼ P T y

Perspective projection

z

p’

c

r

x

y

y ¼ P x ¼ P*

p ¼ P T ¼ P T y

Perspective projection

z

p’

Note: y and u haveare identified here

П1

Chasles’ absolute conic: x12 + x2

2 + x32 = 0, x4 = 0.

The absolute quadratic complex: T diag(Id,0) = | u |2 = 0.

Perspective projection

c

r

x

x’

c

r

x

x’

x’¼P x’¼P*

p’¼P T ’ ¼P T x’

x’¼P x ¼P*

p’¼P T ¼P T

The AQC general equation:T = 0, with = X*TX*

Proposition:T’ ¼ û¢ û’

Proposition :P P T ¼

’p

y’

’

y’

’

Proposition :P * P T ¼ *

Triggs (1997);Pollefeys et al. (1998)

epT = H ppT

ex = H-1 px

e = p

*

*

Z

X

Relation between K, , and *

What is a camera?What is a camera?(Ponce, CVPR’09)(Ponce, CVPR’09)

x

c

ξ

ry

x

x

c

ξ

ry

c

x

c

ξ

ry

x

c

ξ

x

c

ξ

ry

x

c

ξ

ξ

x

c

ξ

ry

x

ξ

ry

Linear familyof lines

x

ξ

x

c

ξ

ξ

ξ

Lines linearly dependent on 2 or 3 lines

(Veblen & Young, 1910)

Then go on recursively for general linear dependence

© H. Havlicek, VUT

What a camera is

Definition: A camera is a two-parameter linear family of lines – that is, a degenerate regulus, or a non-degenerate linear congruence.

Rank-3 families:Reguli

Line fields ≡ epipolar plane images(Bolles, Baker, Marimont, 1987)

Line bundles

Rank-4 (nondegenerate) families:Linear congruences

Figures © H. Havlicek, VUT

x

ξ

yr

x

yr

ξ

Hyperbolic linear congruences

Crossed-slit cameras(Zomet et al., 2003)

Linear pushbroom cameras(Gupta & Hartley, 1997)

© E

. M

olz

can

© L

eic

a

Hyperbolic linear congruences

© T. Pajdla, CTU

Elliptic linear congruences

Linear oblique cameras (Pajdla, 2002)Bilinear cameras (Yu & McMillan, 2004)Stereo panoramas / cyclographs (Seitz & Kim, 2002)

Parabolic linear congruences

Pencil cameras (Yu & McMillam, 2004)Axial cameras (Sturm, 2005)

Plücker coordinates and the Klein quadric

line

screw

the Klein quadric= x Ç y =

uv[ ]

x

y

Note: u . v = 0

P5

Pencils of screws and linear congruences

line

s

P5

the Klein quadric

Reciprocal screws:(s | t) = 0

Screw ≈ linear complex:s ≈ { ± | ( s | ± ) = 0 }

line

s

P5

the Klein quadric

tl

Pencils of screws and linear congruences

Reciprocal screws:(s | t) = 0

Screw ≈ linear complex:s ≈ { ± | ( s | ± ) = 0 }

Pencil of screws: l = { ¸ s + ¹k t ; ¸k¹2R }

The carrier of l is alinear congruence

P5

e

hp

Reciprocal screws:(s | t) = 0

Screw ≈ linear complex:s ≈ { ± | ( s | ± ) = 0 }

Pencil of screws: l = { ¸ s + ¹k t ; ¸k¹2R }

The carrier of l is alinear congruence

Pencils of screws and linear congruences

x

±2

Hyperbolic linear congruences

»

±1

x»1

p1

±1

±2

p2

Hyperbolic linear congruences

» = (xT[ p1 p2T]x) »1 + (xT[ p1 p2

T]x) »2

+ (xT[ p1 p2T]x) »3 + (xT[ p1 p2

T]x) »4

»2

»3 »4

»

x»1

p1

±1

±2

p2

Hyperbolic linear congruences

» = (yT[ p1 p2T] y) »1 + (yT[ p1 q2

T] y) »2

+ (yT[ q1 p2T] y) »3 + (yT[ q1 q2

T] y) »4

y = u1 y1 + u2 y2 + u3 y3 = Y u

»2

»3 »4

»

y

x»1

p1

±1

±2

p2

Hyperbolic linear congruences

» = (uT[ ¼1¼2T]u) »1 + (uT[ ¼1 ρ½2

T]u) »2

+ (uT[ ρ½1¼2T]u) »3 + (uT[ρ½1 ρ½2

T]u) »4

= X û , where X2R6£4 and û2R4

»2

»3 »4

»

y

x ξ

±

a2

p1

z

p2

p

a1

Parabolic linear congruences

±

s°

T

» = X û , where X2R6£5 and û2R5

Elliptic linear congruences

x

»y

» = X û , where X2R6£4 and û2R4

x

»1

y1

»2

y2

Epipolar geometry

(»1 | »2 ) = 0 or û1TF û2 = 0, where F = X1

TX2 2R4£4

Feldman et al. (2003): 6£6 F for crossed-slit cameras

Gupta & Hartley (1997): 4£4 F for linear pushbroom cameras

Trinocular geometry

Di (»1 , »2 , »3 ) = 0 or Ti (û1 , û2 , û3 ) = 0, for i = 1,2,3,4

1 1 1

2 2 2

3 3 3

4 4 4

5 5 5

6 6 6

δ

η φ

x

The essential map(Oblique cameras, Pajdla, 2002)

x

ξ

Ax

x ! ξ = x Ç Ax

B

H

P

E

Canonical forms of essential maps A ( = matrices with quadratic minimal poylnomial)

(Batog, Goaoc, Ponce, 2009)

Alternative geometric characterization of linearcongruences

A new elliptic camera? (Batog, Goaoc, Ponce, 2010)

SMOOTH SURFACES AND THEIR OUTLINES

• Elements of Differential Geometry• What are the Inflections of the Contour?• Koenderink’s Theorem•The second fundamental form• Koenderink’s Theorem• Aspect graphs• More differential geometry• A catalogue of visual events• Computing the aspect graph

• http://www.di.ens.fr/~ponce/geomvis/lect6.ppt • http://www.di.ens.fr/~ponce/geomvis/lect6.pdf

Smooth Shapes and their Outlines

Can we say anything about a 3D shapefrom the shape of its contour?

What are the contour stable features??

folds cusps T-junctions

Shadows are likesilhouettes..

Reprinted from “Computing Exact Aspect Graphs of Curved Objects: Algebraic Surfaces,” by S. Petitjean,J. Ponce, and D.J. Kriegman, the International Journal of ComputerVision, 9(3):231-255 (1992). 1992Kluwer Academic Publishers.

Reprinted from “Solid Shape,” by J.J. Koenderink,MIT Press (1990). 1990 by the MIT.

Differential geometry: geometry in the small

A tangent is the limitof a sequence ofsecants.

The normal to a curveis perpendicular to thetangent line.

What can happen to a curve in the vicinity of a point?

(a) Regular point;

(b) inflection;

(c) cusp of the first kind;

(d) cusp of the second kind.

The Gauss Map

• It maps points on a curve onto points on the unit circle.

• The direction of traversal of the Gaussian image revertsat inflections: it folds there.

The curvature

C

• C is the center of curvature;

• R = CP is the radius of curvature;

• = lim s = 1/R is the curvature.

Closed curves admit a canonical orientation..

> 0

<0

= d / ds à derivative of the Gauss map!

Twisted curves are more complicated animals..

A smooth surface, its tangent plane and its normal.

Normal sections and normal curvatures

Principal curvatures:minimum value maximum value

Gaussian curvature:K = 1 1

22

The differential of the Gauss map

dN (t)= lim s ! 0

Second fundamental form:II( u , v) = uT dN ( v )

(II is symmetric.)

• The normal curvature is t = II ( t , t ).• Two directions are said to be conjugated when II ( u , v ) = 0.

The local shape of a smooth surface

Elliptic point Hyperbolic point

Parabolic point

K > 0 K < 0

K = 0

Reprinted from “On ComputingStructural Changes in Evolving Surfaces and their Appearance,”By S. Pae and J. Ponce, theInternational Journal of ComputerVision, 43(2):113-131 (2001). 2001 Kluwer AcademicPublishers.

The parabolic lines marked on the Apollo Belvedere by Felix Klein

N . v = 0 ) II( t , v )=0

Asymptotic directions:

The contour cusps whenwhen a viewing ray grazesthe surface along an asymptotic direction.

II(u,u)=0

The Gauss map

The Gauss map folds at parabolic points.Reprinted from “On ComputingStructural Changes in Evolving Surfaces and their Appearance,”By S. Pae and J. Ponce, theInternational Journal of ComputerVision, 43(2):113-131 (2001). 2001 Kluwer AcademicPublishers.K = dA’/dA

Smooth Shapes and their Outlines

Can we say anything about a 3D shapefrom the shape of its contour?

Theorem [Koenderink, 1984]: the inflections of the silhouetteare the projections of parabolic points.

Koenderink’s Theorem (1984)

K = r c

Note: > 0.r

Corollary: K and havethe same sign!

c

Proof: Based on the idea that,given two conjugated directions,

K sin2 = u v

What are the contour stable features??

folds T-junctionscusps

How does the appearance of an object change with viewpoint?

Reprinted from “Computing Exact Aspect Graphs of Curved Objects: Algebraic Surfaces,” by S. Petitjean,J. Ponce, and D.J. Kriegman, the International Journal of ComputerVision, 9(3):231-255 (1992). 1992Kluwer Academic Publishers.

Imaging in Flatland: Stable Views

Visual Event: Change in Ordering of Contour Points

Transparent ObjectOpaque Object

Visual Event: Change in Number of Contour Points

Transparent ObjectOpaque Object

Exceptional and Generic Curves

The Aspect GraphIn Flatland

The Geometry of the Gauss Map

Cusp ofGauss

Gutterpoint

Concavefold

Convexfold

Gausssphere

Image ofparaboliccurve

Movinggreatcircle

Reprinted from “On ComputingStructural Changes in Evolving Surfaces and their Appearance,”By S. Pae and J. Ponce, theInternational Journal of ComputerVision, 43(2):113-131 (2001). 2001 Kluwer AcademicPublishers.

Asymptotic directions at ordinary hyperbolic points

The integral curves of the asymptoticdirections form two families ofasymptotic curves (red and blue)

Asymptotic curves

Parabolic curve Fold

Asymptotic curves’ images

Gaussmap

• Asymptotic directions are self conjugate: a . dN ( a ) = 0

• At a parabolic point dN ( a ) = 0, so for any curve t . dN ( a ) = a . dN ( t ) = 0

• In particular, if t is the tangent to the parabolic curve itself dN ( a ) ¼ dN ( t )

The Lip Event

Reprinted from “On ComputingStructural Changes in Evolving Surfaces and their Appearance,”By S. Pae and J. Ponce, theInternational Journal of ComputerVision, 43(2):113-131 (2001). 2001 Kluwer AcademicPublishers.

v . dN (a) = 0 ) v ¼ a

The Beak-to-Beak Event

Reprinted from “On ComputingStructural Changes in Evolving Surfaces and their Appearance,”By S. Pae and J. Ponce, theInternational Journal of ComputerVision, 43(2):113-131 (2001). 2001 Kluwer AcademicPublishers.

v . dN (a) = 0 ) v ¼ a

Ordinary Hyperbolic Point

Flecnodal Point

Reprinted from “On ComputingStructural Changes in Evolving Surfaces and their Appearance,”By S. Pae and J. Ponce, theInternational Journal of ComputerVision, 43(2):113-131 (2001). 2001 Kluwer AcademicPublishers.

Red asymptotic curves

Red flecnodal curve

Asymptoticsphericalmap

Red asymptotic curves

Red flecnodal curve

Cusp pairs appear or disappear as one crosses the fold of theasymptotic spherical map.This happens at asymptotic directions along parabolic curves,and asymptotic directions along flecnodal curves.

The Swallowtail Event

Flecnodal Point

Reprinted from “On Computing Structural Changes in Evolving Surfaces and their Appearance,” by S. Pae and J. Ponce, theInternational Journal of Computer Vision, 43(2):113-131 (2001). 2001 Kluwer Academic Publishers.

The Bitangent Ray Manifold:

Ordinarybitangents..

..and exceptional(limiting) ones.

P

P’

P”

limiting bitangent line

unodeReprinted from “Toward a Scale-Space Aspect Graph: Solids ofRevolution,” by S. Pae and J. Ponce, Proc. IEEE Conf. on ComputerVision and Pattern Recognition (1999). 1999 IEEE.

The Tangent Crossing Event

Reprinted from “On Computing Structural Changes in Evolving Surfaces and their Appearance,” by S. Pae and J. Ponce, theInternational Journal of Computer Vision, 43(2):113-131 (2001). 2001 Kluwer Academic Publishers.

The Cusp Crossing Event

After “Computing Exact Aspect Graphs of Curved Objects: Algebraic Surfaces,” by S. Petitjean, J. Ponce, and D.J. Kriegman, the International Journal of Computer Vision, 9(3):231-255 (1992). 1992 Kluwer Academic Publishers.

The Triple Point Event

After “Computing Exact Aspect Graphs of Curved Objects: Algebraic Surfaces,” by S. Petitjean, J. Ponce, and D.J. Kriegman, the International Journal of Computer Vision, 9(3):231-255 (1992). 1992 Kluwer Academic Publishers.

X0

X1

E1

S1

S2

E3

S1

S2

Tracing Visual Events

P1(x1,…,xn)=0…Pn(x1,…,xn)=0

F(x,y,z)=0

Computing the Aspect Graph

• Curve Tracing

• Cell Decomposition

After “Computing Exact Aspect Graphs of Curved Objects: Algebraic Surfaces,” by S. Petitjean, J. Ponce, and D.J. Kriegman, the International Journal of Computer Vision, 9(3):231-255 (1992). 1992 Kluwer Academic Publishers.

An Example

Approximate Aspect Graphs (Ikeuchi & Kanade, 1987)

Reprinted from “Automatic Generation of Object Recognition Programs,” by K. Ikeuchi and T. Kanade, Proc. of the IEEE, 76(8):1016-1035 (1988). 1988 IEEE.

Approximate Aspect Graphs II: Object Localization(Ikeuchi & Kanade, 1987)

Reprinted from “Precompiling a GeometricalModel into an Interpretation Tree for ObjectRecognition in Bin-Picking Tasks,” by K. Ikeuchi,Proc. DARPA Image Understanding Workshop,1987.