calibration of a vision-based system for displacement measurement in planetary exploration space...

Calibration of a vision-based system for

displacement measurement in planetary

exploration space missions

Marco Pertile1, Marina Magnabosco2, Stefano Debei1

1 Dept. of Mechanical Engineering, University of Padova,

Via Venezia 1, 35131 Padova (PD), Italy.

2 Cranfield University, UK.

IMEKO 2010 London, UK, 1-3 September 2010

PURPOSE

To perform a detailed uncertainty analysis and calibration of a

measurement system of displacement based on a stereo camera

and applied to a simulated planetary scene.


METHOD

Experimental tests

using a reference

displacement

instrument, to obtain

a calibration curve

Experimental tests to

evaluate the uncertainty

of input quantities used

to calculate the output

quantity (displacement)

Probabilistic

propagation to

evaluate the

uncertainty of an

indirect measurement

performed by a

mathematical model

INTRODUCTION


In planetary exploration space missions, the position of a vehicle on the planet surface

can not be measured in a easy way:

1. An odometry system (e.g. optical encoders), that measures the rotation of the

wheels, has wide uncertainty due to slippage of wheels on a natural, often sandy or

slippery, surface.

2. GPS-like positioning systems are accurate but are not yet available on

extraterrestrial planets.

3. Inertial navigation sensors exhibit unacceptable drifts.

Need of a reliable and accurate displacement instrument is particularly relevant.

In this presentation a measurement system based on a stereo camera is described

MEASUREMENT ALGORITHM


Camera model

A pin-hole camera model is

assumed:

-X, Y, Z defines the 3D position of

landmarks (3D points);

-x, y are the coordinates of the

projection of a landmark using an

ideal camera, aligned like the real

camera but with a focal length

equal to 1 (in length units).

• Intrinsic parameters: define the functional relationship between projection x, y,

expressed in length units, and projection x’, y’, expressed in pixels; they comprise: the

pixel densities along the two axes, the focal length, the position in the image of the

principal point (intersection of the optical axis with the sensor), distortion parameters.

• Extrinsic parameters: the relative position and rotation between the two cameras.



The measurand is the displacement of a calibrated stereo system and the measurement

is performed using the images acquired in an initial position and in a second one.

Indirect measurement

Output:

a numerical evaluation of the displacement

Input :

1. Positions in pixels of the projections on the image plane of 3D points (landmarks);

2. Intrinsic and extrinsic parameters of each camera



1. Positions of 2D keypoints on the image plane

a) A detector locally analyses the images and finds out regions that are projections of

landmarks and can be used as features. The Hessian-Affine detector is selected.

b) A descriptor provides representations of the detected regions. Thus, the descriptor

allows to search corresponding features (regions that are projections of the same

landmark) in the acquired images and to perform their matching:

• between the two cameras;

• between images acquired by the same camera but in two different positions.

This matching phase is required

to measure the 3D position of

features and then the position of

the stereo system.

The SIFT (Scale Invariant

Feature Transform) descriptor is

selected. Image 1 Image 2



2. Computation of 3D position of landmarks

Once the features are identified in images and matched between the two cameras, the

3D positions of landmarks are computed by the middle point triangulation method.

3. Stereo system displacement

Once the 3D landmarks are evaluated for two positions of the stereo system, the

displacement is computed as the rigid translation that makes the corresponding 3D

points overlap.

CALIBRATION


Determination of intrinsic and extrinsic parameters of the stereo system

The first step to calibrate the whole measurement system is the determination of intrinsic

and extrinsic parameters of the stereo system.

The known Zhang method is selected:

1. It uses a plurality of images of a chessboard acquired in different positions and

orientations;

2. It comprises:

a) a first analytical evaluation of parameters for both cameras;

b) a nonlinear optimization technique based on the maximum likelihood criterion

(Levenberg-Marquardt algorithm); lens distortions, especially radial distortion,

are taken into account the for each camera.

CALIBRATION


Calibration of the whole measurement system

The stereo system is mounted on a mechanical slide having 2 degrees of freedom and

is aimed at a simulated planetary scene obtained with crumpled brown paper.

The whole system is calibrated

separately along two orthogonal

directions: a first one

substantially aligned with the

optical axes of the cameras (axial

displacement) and the second

one substantially orthogonal to

the optical axes (transverse

displacement).

For each direction the stereo system is moved from an initial position to a final one with

11 steps, while the displacement is measured by the laser interferometer and by the

stereo system in order to build a calibration curve for each direction of translation.

UNCERTAINTY ANALYSIS


The uncertainty associated with the following quantities are analyzed and evaluated:

1. intrinsic and extrinsic parameters of the stereo system, whose uncertainties are

evaluated during the stereo calibration by applying a Monte Carlo simulation to the

Zhang method; in this sub-problem the input quantities are the chessboard

dimensions, and the positions of the chessboard points on the image plane;

2. FEATURE POSITIONS in the image plane, which have two main contributions:

1. the sensor and reading electronics uncertainty;

2. the lighting angular variation of the scene.


2D position of features on the image plane


A. uncertainty (commonly referred to

as image noise in computer vision)

associated with the image sensor

and its electronics

B. Variation of lighting conditions

Sensor and reading electronics

uncertainty depends on the acquired

scene

A simulated rocky scene

is employed

Sensor and reading electronics

uncertainty is assumed of random

type

200 images of the same scene were

acquired with both cameras in fixed

positions and in the same shooting

conditions

A.





On a planet surface, the sun angular position may change between two acquisition

positions.

Time tm allows to estimate a maximum angular variation (between two consecutive

image acquisitions) of the lighting if the planet (e.g. Mars) rotation and orbit is known

Time tm required for the stereo system to move from an initial position to a final one

is evaluated assuming a moving velocity of a rover on a planetary surface


Dedicated experimental tests


slide

camera

lamp

Simulated

scene

PC





The angular variation of lighting

between two images of the same

scene make the shadows move and

yields a translation of features on the

image plane

20 images of the same scene are

acquired varying the lamp position

to simulate the maximum angular

variation of sun on the planet

surface.

RESULTS


Measured AXIAL displacement: stereo system vs. laser interferometer.

RESULTS


Measured TRANSVERSE displacement: stereo system vs. laser interferometer.

RESULTS


Difference between measurements of

the stereo system and the

interferometer

Axial direction

Transverse direction

RESULTS


Observations:

1. The uncertainty evaluated for axial displacements is larger than that obtained for

transverse displacements.

Explanations:

a) the uncertainty of 3D points acquired by the stereo system is much wider along

the axial direction than along a transverse direction, due to the small distance

between the two cameras.

b) This disadvantage along axial direction is partially compensated by the fact

that the number of image features correctly matched in case of axial

displacement is generally greater than the number of matched features in case of

transverse displacement.

2. Along both directions, the evaluated uncertainty increases with the measured

displacement.

Explanation: the larger the displacement and the fewer the features correctly matched

among images (the averaging effect associated with a large number of matched

features decreases along the displacement direction).

CONCLUSIONS

1. A vision-based displacement instrument was described and calibrated

using a simulated planetary rocky scene.

2. Dedicated experimental tests were performed to evaluate the most

significant uncertainty sources using the simulated scene. Particular

attention was dedicated to the uncertainty contributions of the feature

detector and of lighting conditions.

3. Two different motion directions were analyzed and the evaluated

uncertainty were compared.


APPENDIXES




Computation of 3D position of landmarks

The 3D position of landmarks is computed by the middle point triangulation method:

For the two projections of the same 3D landmark, the algorithm finds the 3D points with

the minimum distance, belonging respectively to the preimage lines of cameras 1 and 2.

This points define a segment orthogonal to the two skew preimage lines. The middle

point of this segment is selected as the measured 3D point of the landmark (feature).

Stereo system displacement

To compute the displacement, the following steps are performed:

1. The 3D position is calculated for all features (landmarks) detected by both cameras

when the vision system is in an initial position P1.

2. The vision system is moved (cameras are rigidly connected) from the initial position

P1 to a second position P2, the same procedure is used to compute the 3D positions

of the features detected by both cameras in the second position P2.

3. Common features detected in both positions P1 and P2 are identified.

4. The displacement is computed as the rigid translation that makes the corresponding

features overlap.




The probability density function (PDF) of the uncertainty contribution is evaluated by

the frequency histogram of the normalized grey levels of all pixels of all images.

A. Contribution of the sensor and reading electronics uncertainty

For each pixel: the mean value calculated using all 200 images is subtracted from the

same pixel of all images (normalization)

The scene acquired for uncertainty evaluation is not uniform

PDF of the normalized grey levels

grey levels




A.

The sensor and

reading electronics

uncertainty

contribution is added

to all pixels

detector

and

descriptor

Corresponding

features

Distance between the features

found in both images

Monte Carlo simulation




A. Contribution of the sensor and reading electronics uncertainty

All the distances obtained from all iterations are used to build a frequency histogram

along the x and y axes

This histogram is used to evaluate the PDF of the 1D displacements of image features

due to the sensor and reading electronics uncertainty

The evaluated PDF

are substantially

equal along the two x

and y axes

PDF of feature displacements along x axis [pixels]





Comparing two images at a time,

displacements of all common

features are calculated.

The PDFs of feature displacements

are evaluated by the frequency

histograms along x and y axes.

Along x Along y

PDF of feature displacements along x axis [pixels] PDF of feature displacements along y axis [pixels]

calibration of a vision-based system for displacement measurement in planetary exploration space...

Documents