Optics and Lasers in Engineering 49 (2011) 1245–1250

Contents lists available at ScienceDirect

Optics and Lasers in Engineering

0143-81

doi:10.1

n Corr

E-m

journal homepage: www.elsevier.com/locate/optlaseng

Review

A calibration method for stereo vision sensor with large FOV basedon 1D targets

Junhua Sun, Qianzhe Liu, Zhen Liu, Guangjun Zhang n

Key Laboratory of Precision Opt-mechatronics Technology, Ministry of Education, Beihang University, Beijing 100191, China

a r t i c l e i n f o

Article history:

Received 22 October 2010

Received in revised form

12 June 2011

Accepted 19 June 2011Available online 23 July 2011

Keywords:

Stereo vision

Calibration

1D target

Large FOV

66/$ - see front matter & 2011 Elsevier Ltd. A

016/j.optlaseng.2011.06.011

esponding author. Tel.: þ86 1082338768; fa

ail addresses: gjzhang@buaa.edu.cn, sjh@buaa

a b s t r a c t

Large FOV (field of view) stereo vision sensor is of great importance in the measurement of large free-

form surface. Before using it, the intrinsic and structure parameters of cameras should be calibrated.

Traditional methods are mainly based on planar or 3D targets, which are usually expensive and difficult

to manufacture especially for large dimension ones. Compared to that the method proposed in this

paper is based on 1D (one dimensional) targets, which are easy to operate and with high efficiency. First

two 1D targets with multiple feature points are placed randomly, and the cameras acquire multiple

images of the targets from different angles of view. With the fixed angle between vectors defined by the

two 1D targets we can establish the objective function with intrinsic parameters, which can be later

solved by the optimization method. Then the stereo vision sensor with two calibrated cameras is set up,

which acquire multiple images of another 1D target with two feature points in unrestrained motion.

The initial values of the structure parameters are estimated by the linear method for the known

distance between two feature points on the 1D target, while the optimal ones and intrinsic parameters

of the stereo vision sensor are estimated with non-linear optimization method by establishing the

minimizing function involving all the parameters. The experimental results show that the measure-

ment precision of the stereo vision sensor is 0.046 mm with the working distance of about 3500 mm

and the measurement scale of about 4000 mm�3000 mm. The method in this paper is proved suitable

for calibration of stereo vision sensor of large-scale measurement field for its easy operation and high

efficiency.

& 2011 Elsevier Ltd. All rights reserved.

Contents

1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1246

2. Mathematical model of stereo vision . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1246

2.1. Camera model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1246

2.2. Model of stereo vision . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1246

3. Calibration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1247

3.1. Calibrating intrinsic parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1247

3.1.1. Collecting distortion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1247

3.1.2. Estimating intrinsic parameters. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1247

3.1.3. Estimating extrinsic parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1247

3.1.4. Optimizing all the intrinsic and extrinsic parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1248

3.2. Calibrating structure parameter of stereo vision sensor. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1248

4. Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1248

4.1. Calibrating intrinsic parameters of the cameras . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1248

4.2. Calibrating structure parameters of stereo vision sensor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1249

5. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1249

Acknowledgment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1250

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1250

ll rights reserved.

x: þ86 1082316930.

.edu.cn (J. Sun).

P

ol

oro

z

vl

ul

zr

or

ur

vrxy

yr

xr

Fig. 1. Model of stereo vision.

J. Sun et al. / Optics and Lasers in Engineering 49 (2011) 1245–12501246

1. Introduction

Large dimensional 3D vision measurement technique is difficultbut important in the industrial measurement, while calibrationtechnique is the key to the precision of measurement involvingintrinsic parameters and structure parameters. The structure para-meters of stereo vision sensor can be calibrated only after the sensoris set up, for the parameters will change as the relative positionbetween two cameras changes. Traditional calibration methods canobtain higher precision among the existing methods, but theyusually rely on the target too much. Mainstream methods oncalibration are based on 2D or 3D targets [1–4]. These methods ofcalibration have high accuracy but require targets to cover the wholeFOV, which are difficult and expensive to manufacture. Zhang [5]was the first to propose an intrinsic parameters calibration methodbased on 1D target that is characterized by free from self-occlusion,easy to manufacture, economical, portable and suitable for large-scale camera. He proved that it is impossible to calibrate with a 1Dtarget moving without any constraint. So, in his method, an endpointof the 1D target is fixed. However, it is hard to realize in practice.Other methods based on 1D target especially for intrinsic calibrationare also limited in practice. Wu et al.[6] proposed a calibrationmethod based on 1D target with planar motion that relies on precisemotion platform. Wang and Wu [7] proposed a multiple cameracalibration method based on 1D target with unknown motion, whichcannot be applied to a single camera calibration. Structure para-meters calibration is used to determine the relative position betweentwo cameras. Its mainstream methods include calibration based on3D target with the 3D coordinates of feature points known, which isdifficult to manufacture, calibration based on planar target withknown motion, which relies on precise motion platform, andcalibration based on planar target with unknown motion, where itis hard to guarantee the quality of points projected from the featurepoints on the targets. Faugeras et al. [8] first proposed the theory ofself-camera calibration that does not need any target. The methodcalibrates stereo vision sensor with constraints of parametersbetween cameras. However, it is difficult to achieve highaccuracyhere. Ozturk et al. [9] proposed a method that analyzed the samepoint in different images taken by stereo vision to reconstruct 3Dface. Zhou et al. [10] proposed a structure parameters calibrationmethod based on 1D target that is easier to manufacture and knowndistance between two feature points, but the precision needs furtherimprovement for the application of large-scale measurement.

In this paper, a method of calibrating stereo vision sensor isproposed. The intrinsic parameters can be calibrated with two 1Dtargets, and the structure ones can be calibrated with another 1Dtarget in unknown motion. The method can obtain a satisfactoryprecision and high efficiency of calibration. The paper is organizedas follows. Section 2 analyzes the mathematical model of stereovision. Section 3 describes in detail the calibration method with1D target. Section 4 provides experimental results. Finally, Section5 concludes the paper.

2. Mathematical model of stereo vision

2.1. Camera model

A camera is modeled by the usual pinhole. Note that the skew oftwo image axes is not considered in this paper. The relationshipbetween a 3D point P(Xw,Yw,Zw) and its image projection p(uu, vu) isgiven by

s

uu

vu

1

264

375¼

fx 0 u0 0

0 fy v0 0

0 0 1 0

264

375 R T

0T 1

� � Xw

Yw

Zw

1

26664

37775, ð1Þ

where s is a nonzero scale factor. There are four intrinsic para-meters: fx and fy are the scale factors along the image axes u and v,and (u0,v0) is the principal point. R and T, called the extrinsicparameters, are the rotation matrix and the translation vector fromworld coordinate frame to camera coordinate frame, respectively.A camera usually exhibits lens distortion, especially radial distor-tion. In this paper, we consider only the first two terms of radialdistortion. Let p(uu,vu) and pd(ud,vd) be the distortion-free and thedistorted normalized image coordinates, respectively. We have

ud ¼ ðuu�u0Þð1þk1r2uþk2r4

uÞ

vd ¼ ðvu�v0Þð1þk1r2uþk2r4

uÞ,

(ð2Þ

where r2u ¼ u2

uþv2u; k1 and k2 are the coefficients of radial distortion.

The task of camera calibration is to determine the four intrinsicparameters and the two distortion coefficients.

2.2. Model of stereo vision

The model of stereo vision is shown in Fig. 1. Let the leftcamera coordinate frame ol�xlylzl be the world coordinate frameo–xyz, and the image frame of left camera be ol�ulvl. The rightcamera coordinate frame and corresponding image coordinateframe are or�xryrzr and or�urvr, respectively. The origins ofol�ulvl and or�urvr superpose the principal points. Pl(x,y,z) andPr(xr,yr,zr) are coordinates of a 3D point P(x,y,z) in o�xyz andor�xryrzr with projected image points pl(ul,vl) and pr(ur,vr),respectively. The transformation from o–xyz to or–xryrzrcan berepresented by R and t. According to the camera model, we have

sl

ul

vl

1

264

375¼

fxl 0 0

0 fyl 0

0 0 1

264

375

x

y

z

264375¼ Al

1 0 0 0

0 1 0 0

0 0 1 0

264

375

x

y

z

1

26664

37775, ð3Þ

sr

ur

vr

1

264

375¼

fxr 0 0

0 fyr 0

0 0 1

264

375

xr

yr

zr

264

375¼Ar R t

� � x

y

z

1

26664

37775, ð4Þ

where Al and Ar are the intrinsic parameter matrices of left andright camera, respectively.

According to Eqs. (3) and (4), we have

sr

ur

vr

1

264

375¼

fxrr1 fxrr2 fxrr3 fxrtx

fyrr4 fyrr5 fyrr6 fyrty

r7 r8 r9 tz

264

375

zul=fxl

zvl=fyl

z

1

26664

37775: ð5Þ

J. Sun et al. / Optics and Lasers in Engineering 49 (2011) 1245–1250 1247

From Eq. (5), we can derive 3D coordinate of P(x y z) in o�xyz

as follows:

x¼ zul=fxl

y¼ zvl=fyl

z¼fxlfylðfxr tx�ur tzÞ

ur ðr7ulfylþ r8vlfxlþ r9 fxl fylÞ�fxr ðr1ulfylþ r2vlfxlþ r3 fxlfylÞ

¼fxlfylðfyr ty�vr tzÞ

vr ðr7ulfylþ r8vlfxlþ r9fxl fylÞ�fyr ðr4ulfylþ r5vlfxlþ r6fxlfylÞ

:

8>>>>>><>>>>>>:

ð6Þ

Let

Ml ¼Al

1 0 0 0

0 1 0 0

0 0 1 0

264

375¼

ml11 ml

12 ml13 ml

14

ml21 ml

22 ml23 ml

24

ml31 ml

32 ml33 ml

34

2664

3775, ð7Þ

and

Mr ¼Ar ½R9T� ¼

mr11 mr

12 mr13 mr

14

mr21 mr

22 mr23 mr

24

mr31 mr

32 mr33 mr

34

264

375: ð8Þ

According to Eqs. (3), (4), (7) and (8), the measurement model[11] can also be given by

ulml31�ml

11 ulml32�ml

12 ulml33�ml

13

vlml31�ml

21 vlml32�ml

22 vlml33�ml

23

urmr31�mr

11 urmr32�mr

12 urmr33�mr

13

vrmr31�mr

21 vrmr32�mr

22 vrmr33�mr

23

266664

377775

x

y

z

264375¼

ml14�ulm

l34

ml24�vlm

l34

mr14�urmr

34

mr24�vrmr

34

266664

377775:

ð9Þ

In order to reconstruct the 3D coordinates of P, linear system(9) can be solved by the least-square method.

3. Calibration

3.1. Calibrating intrinsic parameters

3.1.1. Collecting distortion

In order to get the linear camera model, the distortion coeffi-cients should be obtained first. For the feature points in the targetthat are collinear, distortion coefficients can be solved by the non-linear optimization method by setting up minimization function ofthe straightness of the feature points in images [12].

3.1.2. Estimating intrinsic parameters

The model of two 1D targets is shown in Fig. 2. Two 1D targetsare placed without any constraint in the visible area floating in 3Dspace. The un-calibrated camera acquires multiple images fromdifferent viewpoints. The feature points in the images areextracted, and the vanishing points [13] on the images are foundwith at least 3 collinear feature points, the distances betweenevery two of which are known. Let v1¼(x1,y1,1)T and v2¼(x2,y2,1)T

be the vanishing point homogeneous coordinates of two 1D

Fig. 2. Model of two 1D targets.

targets in the image and y be the angle between vectors definedby the two targets; we have

cos y¼vT

1A�TA�1v2ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiðvT

1A�TA�1v1ÞðvT2A�TA�1v2Þ

q,

ð10Þ

where A¼

fx a u0

0 fy v0

0 0 1

264

375 is the intrinsic parameters matrix of

camera. For fxE fy, aE0, u0E0.5Nu, v0E0.5Nv, where Nu and Nv

are the number of pixels in the direction of u and v, respectively,we can translate the origin of the frame to the principal point, and

the intrinsic parameters matrix A becomes

f 0 u

0 f v

0 0 1

264

375, named as

~A Then Eq. (10) becomes

cos y¼~vT

1~A�T ~A�1

~v2ffiffið

p~vT

1~A�T ~A�1 ~v1Þð ~vT

2~A�T ~A�1 ~v2Þ,

ð11Þ

where~v1 ¼ ðx1�u0,y1�v0,1ÞT

~v2 ¼ ðx2�u0,y2�v0,1ÞT

(.

Let ~vm,1, ~vm,2 and ~vn,1, ~vn,2 be the vanishing point coordinates oftwo 1D targets in the mth image and the nth image, respectively.Since the relative position between two 1D targets is constant, wehave cos2 ym ¼ cos2 yn; namely

ð ~vTm,1

~A�T ~A

�1~vm,2Þ

2

ð ~vTm,1

~A�T ~A

�1~vm,1Þð ~v

Tm,2

~A�T ~A

�1~vm,2Þ

¼ð ~vT

n,1~A�T ~A

�1~vn,2Þ

2

ð ~vTn,1~A�T ~A

�1~vn,1Þð ~v

Tn,2~A�T ~A

�1~v

n,2Þ

:

ð12Þ

Eq. (12) can be reformed into a simple cubic equation of f�2.We can find f in the positive root by solving this equation. Thenthe minimization function of all the parameters in the intrinsicparameter matrix A is established as follows:

minXM

m ¼ 1

:cos ym�1

M

XMi ¼ 1

cos yi:2, ð13Þ

where m¼1,2, y,M and M is the number of the images. Let fx¼ f,fy¼ f, u0¼0.5Nu, v0¼0.5Nv be the initial value of the intrinsicparameters; then A can be estimated by the Levenberg–Marquardt Algorithm.

3.1.3. Estimating extrinsic parameters

For each target in each image, we have

s1p1 ¼ AP1

s2p2 ¼ AP2

JP1�P2J¼ L

ðp1�p2Þ � v¼ 0

,

8>>>><>>>>:

ð14Þ

where P1¼(xc1,yc1,zc1)T and P2¼(xc2,yc2,zc2)T are the coordinatesin Oc–xcyczc projected from the two feature points on the 1Dtarget; p1¼(u1,v1,1)T and p2¼(u2,v2,1)T are the correspondingimage points. v¼(x,y,1)T is the coordinate of the vanishing pointof the line on which the feature points lie. L is the spatial distancebetween the two feature points on the 1D target and s1 and s2 arenonzero constants. With Eq. (14), we can obtain the coordinatesof P1 and P2 on the target in Oc–xcyczc.

Let Cm be the camera coordinate frame at the mth viewpoint,m, r and j be the indexes of the image, target and feature point,respectively, and Pm,r,j be 3D coordinate of the jth real point onthe rth target in Cm, where r¼1,2,j¼1,2,y,N. N is the numberof feature points on each target. According to Eq. (14), we canobtain Pm,r,j. Let C1 be the world coordinate frame. We can use

J. Sun et al. / Optics and Lasers in Engineering 49 (2011) 1245–12501248

P1,1,1, P1,1,2, P1,2,1 and P1,2,2, and Pm,1.1, Pm,1.2, Pm,2.1 and Pm,2.2 toestimate the transformation Rm and tm from Cm to C1.

3.1.4. Optimizing all the intrinsic and extrinsic parameters

In the above process of estimating intrinsic and extrinsicparameters we use only the two feature points on each targetand the constraint that the angle between vectors defined by twotargets is kept constant, yet not use all the feature points on each1D target. So the results of intrinsic and extrinsic parameters arenot optimal and seriously affected by noise.

We establish the objective function to minimize the squaresum of the error between the real and reprojecting imagecoordinates of all the feature points as follows:

minX

m

Xr

Xj

Jxm,r,j�x̂ðA,k1,k2,Rm,tm,P1,r,jÞJ2, ð15Þ

where x̂ðA,k1,k2,Rm,tm,P1,r,jÞ is the feature point’s reprojectingpoint on the image calculated through the estimated cameramodel, and xm,r,j is the real image point. Let the intrinsic andextrinsic parameters estimated above be the initial value; theoptimal solution of the parameters can be obtained by solvingEq. (15) with the levenberg–Marquardt Algorithm.

Fig. 3. Images of two 1D targets for calibrating intrinsic parameters.

3.2. Calibrating structure parameter of stereo vision sensor

The structure parameters of stereo vision sensor includerotation matrix R and translation vector t. We calibrate R and tby placing a 1D target arbitrarily in the field of view. There areonly two feature points on the 1D target, which is different fromthe 1D targets with multiple feature points (at least 3) used forcalibrating intrinsic parameters mentioned above, and the dis-tance L between the points is exactly known.

Let E be the essential matrix and F be the fundamental matrixof a stereo vision sensor; we have

F ¼A�Tr EA�1

l : ð16Þ

Let pl and pr be the projected image points of P on the imageplanes of left and right cameras, respectively; we have

pTr Fpl ¼ 0: ð17Þ

At least 7 pairs of corresponding projected points are needed tofind F, which can be obtained by the 8-point algorithm proposedby Hartley [14], and E can be found by solving Eq. (16). After that,we can obtain two rotation matrices R and a translation vector t’with a nonzero factor k by operating SVD on E.

Suppose Jt0J¼ 1, and we can obtain a proportional distance L0

between two feature points with the R and t0. With the knowndistance L, we can find the factor k [10].

Because of noise, the 3D coordinates of feature pointsreconstructed by the structure parameters obtained before haveerror. Let pi

1 and pi2 be the 3D coordinates of the two endpoints of

the 1D target reconstructed by the stereo vision sensor usingthe ith left and right images, and d be the distance between pi

1

and pi2; then the error between L and d can be denoted as

follows:

e1ðxÞ ¼ 9L�dðpi1,pi

2Þ9 ð18Þ

where x¼[rx,ry,rz,tx,ty,tz]T; rx ry rz

h iTis the vector form of the

rotation matrix R.Let pl and pr be the projected image points of P on the image

planes of left and right cameras, respectively; we have

e2ðxÞ ¼ 9piTr Fpi

l9¼ 9piTr A�T

r R½t��A�1l pi

l9 ð19Þ

where t½ �� ¼

0 �tz ty

tz 0 �tx

�ty tx 0

264

375. With Eqs. (18) and (19), the

minimization function is established as follows:

eðxÞ ¼ r1

XN

i ¼ 1

½L�dðpi1,pi

2Þ�2þr2

X2N

i ¼ 1

½piTr A�T

r R½t��A�1l pi

l�2, ð20Þ

where r1 and r2 are importance factors, which can usually be setas r1¼10, and r2¼0.1. The target has to be placed at least 4 timesbecause solving the fundamental matrix F needs at least 4 pairs ofpoints. Let the structure parameters obtained by the linearmethod above be the initial value; the optimized solution of thestructure parameter of the stereo vision sensor can be obtained bythe Levenberg–Marquardt Algorithm.

After the stereo vision sensor is set up, the intrinsic parametersof two cameras obtained above are not optimal. So we proposed anovel minimization function to optimize all the intrinsic and thestructure parameters.

We can see from Eq. (6) that z has two types of expression,which should be equal in theory. But they are not equal inpractice, and the error is denoted as ez(x); namely

ez ¼fxlfylðfxrtx�urtzÞ

urðr7ulfylþr8vlfxlþr9fxlfylÞ�fxrðr1ulfylþr2vlfxlþr3fxlfylÞ

�fxlfylðfyrty�vrtzÞ

vrðr7ulfylþr8vlfxlþr9fxlfylÞ�fyrðr4ulfylþr5vlfxlþr6fxlfylÞ: ð21Þ

Let

e3ðxÞ ¼ 9ez9 ð22Þ

The new minimization function becomes

f ðxÞ ¼ r1

XN

i ¼ 1

½e1ðxÞ�2þr2

X2N

i ¼ 1

½e2ðxÞ�2þr3

X2N

i ¼ 1

½e3ðxÞ�2: ð23Þ

More optimal structure parameters and intrinsic parametersfor stereo sensor can be obtained by solving Eq. (23) with theLevenberg–Marquardt Algorithm.

4. Experiments

4.1. Calibrating intrinsic parameters of the cameras

Two Canon EOS-5D cameras with the resolution of 4368pixels�2912pixels, equipped with Canon 35 mm F/2 lens, are used to setup a stereo vision sensor. We first calibrate intrinsic parameters ofthe two cameras at work distance of about 1500 mm, and measure-ment range of about 1200 mm�800 mm. Two 1D targets areproduced by two 19’’ LCD displayers in which both displays animage of 10 co-linear feature points with an interval of 38.67 mm.Fig. 3 shows the two 1D targets.

Fig. 4. 1D target.

Fig. 5. Structure parameters calibration scene.

Fig. 6. Images of 1D target for calibrating structure parameters.

Table 1Calibration result of the camera intrinsic parameters.

Camera fx fy u0 v0 k1 k2 Error (pixel)

Left 4364.673 4370.393 2183.117 1454.640 �9.091�10–2 1.145�10�1 0.228

Right 4361.656 4367.530 2185.994 1467.322 �9.110�10–2 1.122�10�1 0.240

J. Sun et al. / Optics and Lasers in Engineering 49 (2011) 1245–1250 1249

Thirty three images from different viewpoints are used tocalibrate cameras, and the result was shown in Table 1.

The re-projection error is defined as Root Mean Square (RMS)of the projected image points of 660 feature points with thecalibrated parameters and the related real image points.

4.2. Calibrating structure parameters of stereo vision sensor

The stereo vision sensor was set up with the two calibratedcameras. Its working distance was about 3500 mm and the rangeof measurement was about 4000 mm�3000 mm. The 1D target,shown in Fig. 4, has two features with a 1218.64 mm interval. Thecalibration scene is shown in Fig. 5.

Let the frame of left camera be the world coordinate frame;place the target in the measurement area randomly 22 times andobtain 44 observation image points. Fig. 6 shows some 1D targetimages taken by left camera and right camera.

First, the initial value of structure parameters can be obtainedby the linear method. After the optimization with 6 variables x¼

(rx,ry,rz,tx,ty,tz), we get

Mo1 ¼

8:716� 10-1�6:101� 10�3 4:901� 10�1

�2:063� 103

�2:791� 10�4 9:999� 10�1 1:294� 10�2 6:946� 100

�4:901� 10�1�1:142� 10�2 8:716� 10�1 4:323� 102

264

375

The RMS of the error between the real distance and themeasured distance between the two feature points on the targetin all the images calibrated is 0.154 mm.

After the optimization with 10 variables x¼ ðrx,ry,rz,tx,ty,tz,f lx,f l

y,f rx ,f r

y Þ, we get

Mo2 ¼

8:718� 10�1�6:088� 10�3 4:898� 10�1

�2:060� 103

�4:695� 10�4 9:999� 10�1 1:326� 10�2 7:904� 100

�4:899� 10�1�1:179� 10�2 8:717� 10�1 4:371� 102

264

375

The RMS error of real distance and measured distance betweenthe two feature points on the target is 0.043 mm.

After the optimization, the intrinsic parameters of the twocameras are revised as follows:

Aol ¼

4358:277 0 2183:117

0 4357:246 1454:640

0 0 1

264

375,

Aor ¼

4363:552 0 2185:994

0 4362:842 1467:322

0 0 1

264

375

In order to evaluate the measurement precision of the stereovision sensor, the 1D target is randomly placed 10 times atdifferent positions in the measurement area and the distancebetween two endpoints on the target is measured. From themeasurement data listed in Table 2 we can see that the RMS errorsolved by optimization with 6 structure parameters is 0.132 mm,and the one with 10 structure parameters is 0.046.

5. Conclusion

We have presented a calibration technique for stereo visionbased on 1D targets. It includes two main steps: calibratingintrinsic parameters with two 1D targets, which have multiplefeature points, and calibrating structure parameters of stereovision with another 1D target, which has two feature points.Comparing the existing techniques, there are some advantages inthis method. First the approach is suitable to calibrate a stereovision sensor applied in a complicated space, because 1D targetoccupies much less space. Second the 1D target is easier to buildthan stereo target or planar target, and easy to take for calibration

Table 2Measurement data and error of the stereo vision sensor.

Image coordinates

of left camera

Image coordinates

of right camera

Reconstructed 3D

coordinates

6 variable

Distance (mm)

10 variable

Distance (mm)

u (pixel) v (pixel) u (pixel) v (pixel) x (mm) y (mm) z (mm) Error (mm) Error (mm)

1 625.005 987.748 434.560 1114.158 �1123.587 �336.566 3142.828 1218.350 1218.582

2125.449 1671.482 1631.355 1730.624 �44.978 169.198 3399.240 �0.290 �0.058

2 588.354 1001.245 397.375 1126.891 �1141.544 �324.450 3119.674 1218.597 1218.731

2081.283 1634.101 1667.672 1693.337 �81.716 144.047 3497.272 �0.043 0.091

3 1581.746 1104.886 1058.863 1186.495 �438.256 �254.882 3176.133 1218.707 1218.625

3027.186 1557.255 2540.532 1634.227 681.443 82.839 3518.572 0.067 �0.015

4 1929.640 1116.859 1381.631 1186.454 �190.022 �253.251 3267.215 1218.719 1218.654

3323.870 1556.136 2907.508 1639.769 937.831 83.411 3583.009 0.079 0.014

5 2319.466 1128.942 1757.217 1185.270 104.835 �250.486 3350.962 1218.740 1218.681

3653.898 1552.158 3356.065 1643.121 1235.541 81.828 3661.209 0.100 0.041

6 2885.508 1149.116 2350.477 1186.418 556.848 �242.326 3455.192 1218.701 1218.648

4168.542 1541.855 4096.242 1643.228 1700.405 74.553 3732.623 0.061 0.008

7 2243.764 1555.927 1652.123 1616.662 45.943 76.782 3301.609 1218.752 1218.672

3554.620 1156.580 3252.598 1167.691 1157.467 �251.726 3678.118 0.112 0.032

8 1916.526 1539.052 1384.410 1594.984 �201.118 63.708 3287.902 1218.812 1218.706

3339.791 1160.533 2911.491 1181.703 946.046 �240.727 3564.633 0.172 0.066

9 1403.645 1506.122 922.918 1557.612 �565.377 37.429 3161.204 1218.770 1218.601

2931.670 1180.639 2387.659 1219.389 591.878 �216.736 3446.070 0.130 �0.039

10 830.341 1508.706 588.864 1555.318 �992.920 39.782 3198.920 1218.717 1218.651

2371.176 1175.025 1930.653 1229.486 151.934 �225.974 3521.088 0.077 0.011

RMS error (mm) 0.132 0.046

J. Sun et al. / Optics and Lasers in Engineering 49 (2011) 1245–12501250

on field. Third, the calibration process is simple for the 1D targetcan be placed arbitrarily. Finally, by establishing the objectivefunction of all the intrinsic parameters and structure parameters,we optimize them by the Levenberg–Marquardt algorithm, whichhas a marked improvement in the precision of the stereo visionsensor. The experimental results show that the RMS error of thereal distance and the measured distance between the two featurepoints on the target is upgraded to 0.046 mm from 0.154 mmafter the global optimization. So our approach can offer aneffective solution for calibrating stereo vision with large FOV.

Acknowledgment

This research has been supported by the National NaturalScience Foundation of China under Grant 60804060, and Specialfund for the Doctoral Program of Higher Education of China underGrant 200800061003.

References

[1] Tsai RY. A versatile camera calibration technique for high-accuracy 3Dmachine vision metrology using off-the-shelf TV cameras and lenses. IEEETransactions on Robotics and Automation 1987;3:323–44.

[2] Zhang ZY. A flexible new technique for camera calibration. IEEE Transactionson Pattern Analysis and Machine Intelligence 2000;22(11):1330–4.

[3] Xu L, Zhang ZhJ. Calibration in structured light system using coplanar target.Journal of Optoelectronics Laser 2009;20(8):1063–9.

[4] Liu Z, Zhang GJ, Wei. ZZ. Global calibration of multi-sensor vision system-based on two planar targets. Journal of Mechanical Engineering 2009;45(7):228–32.

[5] Zhang ZY. Camera calibration with one-dimensional objects. IEEE Transac-tions on Pattern Analysis and Machine Intelligence 2004;26(7):892–9.

[6] Wu FC, Hu ZY, Zhu. HJ. Camera calibration with moving one-dimensionalobjects. Pattern Recognition 2005;38(5):755–65.

[7] Wang L, Wu FC. Multi-camera calibration based on 1D calibration object.AC Automatica Sinica 2007;33(3):225–31.

[8] Faugeras O, Luong Q, Maybank S. Camera self-calibration: theory andexperiments. In: Proceedings of the second ECCV 1992:231–4. p..

[9] OzturkA.O., HaliciU, UlusoyI. 3D face reconstruction using stereo images andstructured light. In: Proceedings of the 16th IEEE signal processing andcommunications applications conference Didim, Turkey: IEEE; 2008.

[10] Zhou FQ, Zhang GJ, Wei ZZ. Calibration binocular vision sensor with one-dimensional target of unknown motion. Chinese Journal of Mechanical

Engineering 2006;42(6):92–6.[11] Ma SD, Zhang ZY. Computer vision. Science Press; 1998.[12] Devernay F, Faugeras O. Straight lines have to be straight: automatic

calibration and removal of distortion from scenes of structured environ-

ments. Machine Vision and Application 2001:14–24.[13] Hartley R, Zisserman Andrew. A multiple view geometry in computer

vision.Cambridge University Press; 2000.[14] Hartley R. In defense of the eight-point algorithm. IEEE Transactions on

Pattern Analysis And Machine Intelligence 1997;19(6):580–93.