EECS 274 Computer Vision

Geometry of Multiple Views

Geometry of Multiple Views

• Epipolar geometry– Essential matrix– Fundamental matrix

• Trifocal tensor• Quadrifocal tensor• Reading: FP Chapter 10, S Chapter 7

• Epipolar plane defined by P, O, O’, p and p’

• Epipoles e, e’

• Epipolar lines l, l’

• Baseline OO’

Epipolar geometry

p’ lies on l’ where the epipolar plane intersects with image plane π’

l’ is epipolar line associated with p and intersects baseline OO’ on e’

e’ is the projection of O observed from O’

• Potential matches for p have to lie on the corresponding epipolar line l’

• Potential matches for p’ have to lie on the corresponding epipolar line l

Epipolar constraint

Epipolar Constraint: Calibrated Case

Essential Matrix(Longuet-Higgins, 1981)3 ×3 skew-symmetric

matrix: rank=2

M’=(R t)

• E is defined by 5 parameters (3 for rotation and 2 for translation)

• E p’ is the epipolar line associated with p’

• E T p is the epipolar line associated with p

• Can write as l .p = 0

• The point p lies on the epipolar line associated with the vectorE p’

Properties of essential matrix

Properties of essential matrix (cont’d)

• E e’=0 and ETe=0 (E e’=-RT[tx]e=0 )

• E is singular• E has two equal non-zero singular

values (Huang and Faugeras, 1989)

Epipolar Constraint: Small MotionsTo First-Order:

Pure translation: Focus of Expansion e

The motion field at every point in the image points to focus of expansion

Epipolar Constraint: Uncalibrated Case

Fundamental Matrix(Faugeras and Luong, 1992)are normalized image coordinate pp ˆ,ˆ

• F has rank 2 and is defined by 7 parameters

• F p’ is the epipolar line associated with p’ in the 1st image

• F T p is the epipolar line associated with p in the 2nd image

• F e’=0 and F T e=0

• F is singular

Properties of fundamental matrix

Rank-2 constraint

• F admits 7 independent parameter• Possible choice of parameterization

using e=(α,β)T and e’=(α’,β’)T and epipolar transformation

• Can be written with 4 parameters: a, b, c, d

''''''''

badcacbd

dccd

baab

F

Weak calibration

• In theory: – E can be estimated with 5 point

correspondences– F can be estimated with 7 point

correspondences– Some methods estimate E and F matrices from

a minimal number of parameters

• Estimating epipolar geometry from a redundant set of point correspondences with unknown intrinsic parameters

The Eight-Point Algorithm (Longuet-Higgins, 1981)

|F | =1.

Minimize:

under the constraint2

Homogenous system, set F33=1

Least-squares minimization

• Error function:

|F | =1

Minimize:

under the constraint

factor scale :',

),'(')',(')',(

ppdppdppppe TT FFF

Non-Linear Least-Squares Approach (Luong et al., 1993)

Minimize

with respect to the coefficients of F , using an appropriate rank-2 parameterization (4 parameters instead of 8)

8 point algorithm with least-squares minimizationignores the rank 2 propertyFirst use least squares to find epipoles e and e’ that minimizes |FT e|2 and |Fe’|2

The Normalized Eight-Point Algorithm (Hartley, 1995)

• Estimation of transformation parameters suffer form poor numerical condition problem

• Center the image data at the origin, and scale it so themean squared distance between the origin and the data points is 2 pixels: q = T p , q’ = T’ p’

• Use the eight-point algorithm to compute F from thepoints q and q’

• Enforce the rank-2 constraint

• Output T F T’

T

i i i i

i i

Weak calibration experiment

Trinocular Epipolar Constraints

These constraints are not independent!

Optical centers O1O2O3 defines a trifocal plane

Generally, P does not lie on trifocal plane formed

Trifocal plane intersects retinas along t1, t2, t3

Each line defines two epipoles, e.g., t2 defines e12, e32, wrt O1 and O3

Trinocular Epipolar Constraints: Transfer

Given p1 and p2 , p3 can be computedas the solution of linear equations.

Geometrically, p1 is found as the intersection of epipolar lines associated with p2 and p3

Trifocal Constraints

The set of points that project onto an image line l is the plane L that contains the line and pinhole

Point P in L is projected onto p on line l (l=(a,b,c)T)

)(

1

tR

Pp z

KM

M

Recall

P

Trifocal Constraints

All 3×3 minorsmust be zero!

Calibrated Case

Trifocal Tensorline-line-line correspondence

P

Trifocal Constraints

Calibrated Case

Given 3 point correspondences, p1, p2, p3 of the same point P, and two lines l2, l3, (passing through p2, and p3), O1p1 must intersect the line l, where the planes L2 and L3 intersect

point-line-line correspondence

011 lpT

Trifocal Constraints

Uncalibrated Case

P

Trifocal Constraints

Uncalibrated Case

Trifocal Tensor

Trifocal Constraints: 3 Points

Pick any two lines l and l through p and p .

Do it again.2 3 2 3

T( p , p , p )=01 2 3

Properties of the Trifocal Tensor

Estimating the Trifocal Tensor

• Ignore the non-linear constraints and use linear least-squaresa posteriori• Impose the constraints a posteriori

• For any matching epipolar lines, l G l = 0 • The matrices G are singular• Each triple of points 4 independent equations• Each triple of lines 2 independent equations• 4p+2l ≥ 26 need 7 triples of points or 13 triples of lines • The coefficients of tensor satisfy 8 independent constraints in the uncalibrated case (Faugeras and Mourrain, 1995) Reduce the number of independent parameters from 26 to 18

2 1 3T i

1

i

Multiple Views (Faugeras and Mourrain, 1995)

All 4 × 4 minors have zero determinants

Two Views

Epipolar Constraint

6 minors

Three Views

Trifocal Constraint

48 minors

Four Views

Quadrifocal Constraint(Triggs, 1995)

16 minors

3,...,1,,,,1,Det

l4

k3

j2

i1

lkjiijklijkl

M

M

M

M

Geometrically, the four rays must intersect in P..

Quadrifocal Tensorand Lines

Given 4 point correspondences, p1, p2, p3, p4 of the same point P, and 3 lines l2, l3, l4 (passing through p2, and p3, p4), O1p1 must intersect the line l, where the planes L2 , L3, and L4

Scale-Restraint Condition from Photogrammetry

Trinocular constraints in the presence of calibration or measurement errors