camera parameters and calibration. camera parameters from last time…

30
Camera Parameters and Calibration

Post on 22-Dec-2015

238 views

Category:

Documents


1 download

TRANSCRIPT

Page 1: Camera Parameters and Calibration. Camera parameters From last time…

Camera Parameters and Calibration

Page 2: Camera Parameters and Calibration. Camera parameters From last time…

Camera parameters

U

V

W

⎜ ⎜ ⎜

⎟ ⎟ ⎟ =

Transformation

representing

intrinsic parameters

⎜ ⎜ ⎜

⎟ ⎟ ⎟

1 0 0 0

0 1 0 0

0 0 1 0

⎜ ⎜ ⎜

⎟ ⎟ ⎟

Transformation

representing

extrinsic parameters

⎜ ⎜ ⎜

⎟ ⎟ ⎟

X

Y

Z

T

⎜ ⎜ ⎜ ⎜ ⎜

⎟ ⎟ ⎟ ⎟ ⎟

• From last time….

Page 3: Camera Parameters and Calibration. Camera parameters From last time…

Homogeneous Coordinates (Again)

Page 4: Camera Parameters and Calibration. Camera parameters From last time…

Extrinsic Parameters: Characterizing Camera position

Chasles's theorem: Any motion of a solid body can be composed of a translation and a rotation.

Page 5: Camera Parameters and Calibration. Camera parameters From last time…

3D Rotation Matrices

QuickTime™ and aGIF decompressorare needed to see this picture.

QuickTime™ and aGIF decompressorare needed to see this picture.

QuickTime™ and aGIF decompressorare needed to see this picture.

Page 6: Camera Parameters and Calibration. Camera parameters From last time…

3D Rotation Matrices cont’dEuler’s Theorem: An arbitrary rotation can be described by 3 rotation parameters

R = For example:

More Generally:

Most General:

Page 7: Camera Parameters and Calibration. Camera parameters From last time…

Rodrigue’s FormulaTake any rotation axis a and angle : What is the matrix?

with

R = eA

Page 8: Camera Parameters and Calibration. Camera parameters From last time…

Rotations can be represented as points in space (with care)

Turn vector length into angle, direction into axis:

Useful for generating random rotations, understanding angular errors, characterizing angular position, etc.

Problem: not uniqueNot commutative

Page 9: Camera Parameters and Calibration. Camera parameters From last time…

Other Properties of rotations

NOT CommutativeR1*R2 ≠ R2*R1

Page 10: Camera Parameters and Calibration. Camera parameters From last time…

Rotations

x'=

x1

x2

x3

⎢ ⎢ ⎢

⎥ ⎥ ⎥=

x1 y1 z1

x2 y2 z2

x3 y3 z3

⎢ ⎢ ⎢

⎥ ⎥ ⎥

1

0

0

⎢ ⎢ ⎢

⎥ ⎥ ⎥

To form a rotation matrix, you can plug in the columns of new coordinate points

For Example: The unit x-vectorgoes to x’:

Page 11: Camera Parameters and Calibration. Camera parameters From last time…

Other Properties of Rotations

Inverse: R-1 = RT

rows, columns are othonormal

riT rj = 0 if i≠j, else ri

T ri = 1

Determinant:

det( R ) = 1

The effect of a coordinate rotation on a function:

x’ = R x

F( x’ ) = F( R-1 x )

Page 12: Camera Parameters and Calibration. Camera parameters From last time…

Extrinsic Parametersp’ = R p + t

R = rotation matrix

t = translation vector

In Homogeneous coordinates,

p’ = R p + t =>

x '

y '

z'

1

⎢ ⎢ ⎢ ⎢

⎥ ⎥ ⎥ ⎥

=

r11 r12 r13 tx

r21 r22 r23 ty

r31 r32 r33 tz

0 0 0 1

⎢ ⎢ ⎢ ⎢

⎥ ⎥ ⎥ ⎥

x

y

z

1

⎢ ⎢ ⎢ ⎢

⎥ ⎥ ⎥ ⎥

Page 13: Camera Parameters and Calibration. Camera parameters From last time…

Intrinsic Parameters

u * Su

v * Sv

1

⎢ ⎢ ⎢

⎥ ⎥ ⎥=

Su 0 0

0 Sv 0

0 0 1

⎢ ⎢ ⎢

⎥ ⎥ ⎥

u

v

1

⎢ ⎢ ⎢

⎥ ⎥ ⎥

Differential Scaling

u + u0

v + v0

1

⎢ ⎢ ⎢

⎥ ⎥ ⎥=

1 0 u0

0 1 v0

0 0 1

⎢ ⎢ ⎢

⎥ ⎥ ⎥

u

v

1

⎢ ⎢ ⎢

⎥ ⎥ ⎥

Camera OriginOffset

Page 14: Camera Parameters and Calibration. Camera parameters From last time…

10101001

• Rotation –90o

• Scaling *2

• Translation

1yx

p =

House Points

2D Example

Page 15: Camera Parameters and Calibration. Camera parameters From last time…

The Whole (Linear) Transformation

U

V

W

⎢ ⎢ ⎢

⎥ ⎥ ⎥=

− f ⋅su 0 u0

0 − f ⋅sv v0

0 0 1

⎢ ⎢ ⎢

⎥ ⎥ ⎥

1

0

0

0

1

0

0

0

1

0

0

0

⎢ ⎢ ⎢

⎥ ⎥ ⎥

r11 r12 r13 tx

r21 r22 r23 ty

r31 r32 r33 tz

0 0 0 1

⎢ ⎢ ⎢ ⎢

⎥ ⎥ ⎥ ⎥

x

y

z

T

⎢ ⎢ ⎢ ⎢

⎥ ⎥ ⎥ ⎥

U

V

W

⎜ ⎜ ⎜

⎟ ⎟ ⎟ =

Transformation

representing

intrinsic parameters

⎜ ⎜ ⎜

⎟ ⎟ ⎟

1 0 0 0

0 1 0 0

0 0 1 0

⎜ ⎜ ⎜

⎟ ⎟ ⎟

Transformation

representing

extrinsic parameters

⎜ ⎜ ⎜

⎟ ⎟ ⎟

X

Y

Z

T

⎜ ⎜ ⎜ ⎜ ⎜

⎟ ⎟ ⎟ ⎟ ⎟

u =U/W

v =U/W

Final image coordinates

Page 16: Camera Parameters and Calibration. Camera parameters From last time…

Non-linear distortions (not handled by our treatment)

Page 17: Camera Parameters and Calibration. Camera parameters From last time…

Camera Calibration

• You need to know something beyond what you get out of the camera– Known World points (object)

• Traditional camera calibration• Linear and non-linear parameter estimation

– Known camera motion• Camera attached to robot • Execute several different motions

– Multiple Cameras

Page 18: Camera Parameters and Calibration. Camera parameters From last time…
Page 19: Camera Parameters and Calibration. Camera parameters From last time…

Classical Calibration

Page 20: Camera Parameters and Calibration. Camera parameters From last time…

Camera Calibration

Take a known set of points.

Typically 3 orthogonal planes.

Treat a point in the object as the World origin

Points x1, x2, x3,

Project to y1,y2,y3

Page 21: Camera Parameters and Calibration. Camera parameters From last time…

Calibration Patterns

Page 22: Camera Parameters and Calibration. Camera parameters From last time…

Classical CalibrationPut the set of points known object points xi into a matrix

X = x1 L xn[ ]

And projected points ui into a matrix

U = u1 L un[ ]

U = Π X

UX t = Π (XX t )

UX t (XX t )−1 = Π (XX t )(XX t )−1

UX t (XX t )−1 = Π

Odd derivation of the Least Squares solution:

Solve for the transformation matrix:

Next extract extrinsic and intrinsic parameters from

Note this is only for instructional purposes. It does not work as a real procedure.

Page 23: Camera Parameters and Calibration. Camera parameters From last time…

Real Calibration

• Real calibration procedures look quite different– Weight points by correspondence quality– Nonlinear optimization– Linearizations– Non-linear distortions– Etc.

Page 24: Camera Parameters and Calibration. Camera parameters From last time…

Camera Motion

Page 25: Camera Parameters and Calibration. Camera parameters From last time…

Calibration Example (Zhang, Microsoft)

8 Point matches manually pickedMotion algorithm used to calibrate camera

Page 26: Camera Parameters and Calibration. Camera parameters From last time…

ApplicationsImage based rendering: Light field -- Hanrahan (Stanford)

Page 27: Camera Parameters and Calibration. Camera parameters From last time…

Virtualized Reality

QuickTime™ and aYUV420 codec decompressorare needed to see this picture.

Page 28: Camera Parameters and Calibration. Camera parameters From last time…

Projector-based VRUNC Chapel Hill

Page 29: Camera Parameters and Calibration. Camera parameters From last time…

Shader Lamps

QuickTime™ and aSorenson Video decompressorare needed to see this picture.

Page 30: Camera Parameters and Calibration. Camera parameters From last time…

Shape recovery without calibration

Fitzgibbons, ZissermanFully automatic procedure: Converting the images to 3D models is a black-box filter: Video in, VRML out. * We don't require that the motion be regular: the angle between views can vary, and it doesn't have to be known. Recovery of the angle is automatic, and accuracy is about 40 millidegrees standard deviation. In golfing terms, that's an even chance of a hole in one. * We don't use any calibration targets: features on the objects themselves are used to determine where the camera is, relative to the turntable. Aside from being easier, this means that there is no problem with the setup changing between calibration and acquisition, and that anyone can use the software without special equipment.

For example, this dinosaur sequence was supplied to us by the University of Hannover without any other information. (Actually, we do have the ground-truth angles so that we can make the accuracy claims above, but of course these are not used in the reconstruction).