calibration

CalibrationCalibration

Camera CalibrationCamera Calibration

• GeometricGeometric– Intrinsics: Focal length, principal point, distortionIntrinsics: Focal length, principal point, distortion

– Extrinsics: Position, orientationExtrinsics: Position, orientation

• RadiometricRadiometric– Mapping between pixel value and scene radianceMapping between pixel value and scene radiance

– Can be nonlinear at a pixel (gamma, etc.)Can be nonlinear at a pixel (gamma, etc.)

– Can vary between pixels (vignetting, cosCan vary between pixels (vignetting, cos44, etc.), etc.)

– Dynamic range (calibrate shutter speed, etc.)Dynamic range (calibrate shutter speed, etc.)

Geometric Calibration IssuesGeometric Calibration Issues

• Camera ModelCamera Model– Orthogonal axes?Orthogonal axes?

– Square pixels?Square pixels?

– Distortion?Distortion?

• Calibration TargetCalibration Target– Known 3D points, noncoplanarKnown 3D points, noncoplanar

– Known 3D points, coplanarKnown 3D points, coplanar

– Unknown 3D points (structure from motion)Unknown 3D points (structure from motion)

– Other features (e.g., known straight lines)Other features (e.g., known straight lines)

Geometric Calibration IssuesGeometric Calibration Issues

• Optimization methodOptimization method– Depends on camera model, available dataDepends on camera model, available data

– Linear vs. nonlinear modelLinear vs. nonlinear model

– Closed form vs. iterativeClosed form vs. iterative

– Intrinsics only vs. extrinsics only vs. bothIntrinsics only vs. extrinsics only vs. both

– Need initial guess?Need initial guess?

Caveat - 2D Coordinate SystemsCaveat - 2D Coordinate Systems

• yy axis up axis up vs. vs. yy axis down axis down

• Origin at center vs. Origin at center vs. cornercorner

• Will often write (Will often write (uu,, vv) for image ) for image coordinatescoordinates

uu

vv

uu

uu

vvvv

Camera Calibration – Example 1Camera Calibration – Example 1

• Given:Given:– 3D 3D 2D correspondences 2D correspondences

– General perspective camera model (no distortion)General perspective camera model (no distortion)

• Don’t care about “Don’t care about “zz” after transformation” after transformation

• Homogeneous scale ambiguity Homogeneous scale ambiguity 11 free 11 free parametersparameters

lkzjyix

hgzfyexlkzjyix

dczbyax

z

y

x

lkji

hgfe

dcbashomogeneou

divide

1

lkzjyix

hgzfyexlkzjyix

dczbyax

z

y

x

lkji

hgfe

dcbashomogeneou

divide

1


• Write equations:Write equations:

1

111

111

1111

111

vlkzjyix

hgzfyex

ulkzjyix

dczbyax

1

111

111

1111

111

vlkzjyix

hgzfyex

ulkzjyix

dczbyax


• Linear equationLinear equation

• Overconstrained (more equations than unknowns)Overconstrained (more equations than unknowns)

• Underconstrained (rank deficient matrix – any Underconstrained (rank deficient matrix – any multiple of a solution, including 0, is also a multiple of a solution, including 0, is also a solution)solution)

0

l

c

b

a

uzuyuxuzyx

uzuyuxuzyx

uzuyuxvzyx

uzuyuxuzyx

2222222222

2222222222

1111111111

1111111111

10000

00001

10000

00001

0

l

c

b

a

uzuyuxuzyx

uzuyuxuzyx

uzuyuxvzyx

uzuyuxuzyx

2222222222

2222222222

1111111111

1111111111

10000

00001

10000

00001


• Standard linear least squares methods Standard linear least squares methods forforAx=0 will give the solution x=0Ax=0 will give the solution x=0

• Instead, look for a solution with |x|= 1Instead, look for a solution with |x|= 1

• That is, minimize |Ax|That is, minimize |Ax|22 subject to |x| subject to |x|22=1=1


• Minimize |Ax|Minimize |Ax|22 subject to |x| subject to |x|22=1=1

• |Ax||Ax|22 = (Ax) = (Ax)TT(Ax) = (x(Ax) = (xTTAATT)(Ax) = x)(Ax) = xTT(A(ATTA)xA)x

• Expand x in terms of eigenvectors of Expand x in terms of eigenvectors of AATTA:A:

x = x = 11ee11+ + 22ee22+…+…

x xTT(A(ATTA)x = A)x = 111122++2222

22+…+…

|x||x|22 = = 1122++22

22+…+…


• To minimizeTo minimize1111

22++222222+…+…

subject tosubject to 11

22++2222+… = 1+… = 1

set set minmin= 1 and all other = 1 and all other ii=0=0

• Thus, least squares solution is Thus, least squares solution is eigenvector corresponding to minimum eigenvector corresponding to minimum (non-zero) eigenvalue of A(non-zero) eigenvalue of ATTAA


• Incorporating additional constraints intoIncorporating additional constraints intocamera modelcamera model– No shear (No shear (uu, , vv axes orthogonal) axes orthogonal)

– Square pixelsSquare pixels

– etc.etc.

• Doing minimization in image spaceDoing minimization in image space

• All of these impose All of these impose nonlinearnonlinear constraints constraints ononcamera parameterscamera parameters


• Option 1: nonlinear least squaresOption 1: nonlinear least squares– Usually “gradient descent” techniquesUsually “gradient descent” techniques

– e.g. Levenberg-Marquardte.g. Levenberg-Marquardt

• Option 2: solve for general perspective Option 2: solve for general perspective model, find closest solution that model, find closest solution that satisfies constraintssatisfies constraints– Use closed-form solution as initial guess forUse closed-form solution as initial guess for

iterative minimizationiterative minimization

Radial DistortionRadial Distortion

• Radial distortion can not be representedRadial distortion can not be representedby matrixby matrix

• ((ccuu, , ccvv) is image center,) is image center,

uu**imgimg= = uuimgimg– – ccuu, , vv**

imgimg= = vvimgimg– – ccvv,,

is first-order radial distortion coefficient is first-order radial distortion coefficient

)(1

)(12*2**

2*2**

imgimgimgvimg

imgimgimguimg

vuvcv

vuucu

)(1

)(12*2**

2*2**

imgimgimgvimg

imgimgimguimg

vuvcv

vuucu


• Incorporating radial distortionIncorporating radial distortion

• Option 1:Option 1:– Find distortion first (e.g., straight lines inFind distortion first (e.g., straight lines in

calibration target)calibration target)

– Warp image to eliminate distortionWarp image to eliminate distortion

– Run (simpler) perspective calibrationRun (simpler) perspective calibration

• Option 2: nonlinear least squaresOption 2: nonlinear least squares

Calibration TargetsCalibration Targets

• Full 3D (nonplanar)Full 3D (nonplanar)– Can calibrate with one imageCan calibrate with one image

– Difficult to constructDifficult to construct

• 2D (planar)2D (planar)– Can be made more accuracteCan be made more accuracte

– Need multiple viewsNeed multiple views

– Better constrained than full SFM problemBetter constrained than full SFM problem

Calibration TargetsCalibration Targets

• Identification of featuresIdentification of features– ManualManual

– Regular array, manually seededRegular array, manually seeded

– Regular array, automatically seededRegular array, automatically seeded

– Color coding, patterns, etc.Color coding, patterns, etc.

• Subpixel estimation of locationsSubpixel estimation of locations– Circle centersCircle centers

– Checkerboard cornersCheckerboard corners

Calibration Target w. CirclesCalibration Target w. Circles

3D Target w. Circles3D Target w. Circles

Planar Checkerboard TargetPlanar Checkerboard Target

[Bouguet][Bouguet]

Coded CirclesCoded Circles

[Marschner et al.][Marschner et al.]

Concentric Coded CirclesConcentric Coded Circles

[Gortler et al.][Gortler et al.]

Color Coded CirclesColor Coded Circles

[Culbertson][Culbertson]

Calibrating ProjectorCalibrating Projector

• Calibrate cameraCalibrate camera

• Project pattern onto a known objectProject pattern onto a known object(usually plane)(usually plane)– Can use time-coded structured lightCan use time-coded structured light

• Form (Form (uuprojproj, , vvprojproj, , xx, , yy, , zz) tuples) tuples

• Use regular camera calibration codeUse regular camera calibration code

• Typically lots of keystoning relative to Typically lots of keystoning relative to camerascameras

Multi-Camera GeometryMulti-Camera Geometry

• Epipolar geometry – relationship Epipolar geometry – relationship between observed positions of points in between observed positions of points in multiple camerasmultiple cameras

• Assume:Assume:– 2 cameras2 cameras

– Known intrinsics and extrinsicsKnown intrinsics and extrinsics

Epipolar GeometryEpipolar Geometry

PP

CC11 CC22

pp22pp11


PP

CC11 CC22

pp22pp11

ll22


PP

CC11 CC22

pp22pp11

ll22

Epipolar lineEpipolar line

EpipolesEpipoles


• Goal: derive equation for lGoal: derive equation for l22

• Observation: P, CObservation: P, C11, C, C22 determine a plane determine a plane

PP

CC11 CC22

pp22pp11

ll22


• Work in coordinate frame of CWork in coordinate frame of C11

• Normal of plane is T Normal of plane is T Rp Rp22, where T is , where T is

relative translation, R is relative rotationrelative translation, R is relative rotation

PP

CC11 CC22

pp22pp11

ll22


• pp11 is perpendicular to this normal: is perpendicular to this normal:

p p11 (T (T Rp Rp22) = 0) = 0

PP

CC11 CC22

pp22pp11

ll22


• Write cross product as matrix Write cross product as matrix multiplicationmultiplication

PP

CC11 CC22

pp22pp11

ll22

0

0

0

, **

xy

xz

yz

TT

TT

TT

xxT TT

0

0

0

, **

xy

xz

yz

TT

TT

TT

xxT TT


• pp11 T T* * R pR p22 = 0 = 0 p p11TT EE pp22 = 0 = 0

• E is the E is the essential matrixessential matrix

PP

CC11 CC22

pp22pp11

ll22

Essential MatrixEssential Matrix

• E depends only on camera geometryE depends only on camera geometry

• Given E, can derive equation for line lGiven E, can derive equation for line l22

PP

CC11 CC22

pp22pp11

ll22

Fundamental MatrixFundamental Matrix

• Can define Can define fundamental matrixfundamental matrix F F analogously, operating on pixel analogously, operating on pixel coordinates instead of camera coordinates instead of camera coordinatescoordinates

uu11TT FF uu22 = 0 = 0

• Advantage: can sometimes estimate F Advantage: can sometimes estimate F without knowing camera calibrationwithout knowing camera calibration

calibration

Documents