1 long-term image-based motion estimation dennis strelow

1

Long-term image-based motion estimation

Dennis Strelow

2

Problems (1)

micro air vehicle (MAV) navigation

AeroVironment Black Widow AeroVironment Microbat

3

Problems (2)

mars rover navigation

Mars Exploration Rovers (MER) Hyperion

4

Problems (3)

robotic search and rescue

RhexCenter for Robot-Assisted Search and Rescue, U. of South Florida

5

Problems (4)

NASA ISS personal satellite assistant

6

Problems (5)

Each of these problems requires:

six degree of freedom motion

in unknown environments

without GPS or other absolute positioning

over the long term

…and some of the problems require:

small, light, and cheap sensors

7

Existing mage-based approaches (1)

Monocular image-based motion estimation is a good candidate given these requirements

In particular, simultaneous estimation of:

multiframe motion

sparse scene structure

is the most promising approach

8

Existing image-based approaches (2)

Algorithms exist to estimate camera motion and sparse scene structure:

SVD-based factorization (1992)

Bundle adjustment (1950’s)

Kalman filtering (1990)

Variable state dimension filter (~1994)

9


Hotel sequence here

10


These often generate good results:

11

Sensitivity of motion estimation (4)

But the resulting estimates can be very sensitive to:

incorrect or insufficient image feature tracking

camera modeling and calibration errors

poor prior assumptions on the motion

poor approximations in error modeling

outlier detection thresholds

12


REL sequence here

13


14

Long-term motion estimation (1)

For applications like micro air vehicles…

…the situation is really desperate

15


Each tracked point is only visible in a small percentage of the image sequence

(Example video here of going over the wall)

16


So, we’re no longer estimating our motion with respect to a single point…

…the motion estimation essentially becomes integration, just as in odometry

17


But, harder than odometry:

odometry measurements are direct measurements of the incremental motion

whereas, as we’ve seen:

sparse image measurements can produce very poor estimates of the incremental motion

18


And since we’re essentially integrating incremental motions:

one gross error in the estimated motion finishes you

one mild qualitative error may quickly compound into a gross error, finishing you

19


And since we’re essentially integrating incremental motions:

one gross error in the estimated motion finishes you

one mild qualitative error may quickly compound into a gross error, finishing you

Even with no gross or “mild qualitative” errors the integrated motion will always drift

20

Approach (1)

Two areas:

improved 6 DOF estimates

improved tracking

21

Approach (2)

most of our work has been on improved 6 DOF estimates

22

Improved 6 DOF estimates (1)

Batch estimation:

uses all of the observations at once

all observations must be available before computation begins

Online estimation:

observations are incorporated as they arrive

many reasons why the filter’s prior distribution might be inaccurate

23


Image measurements only

often the only option

Image and inertial measurements

can disambiguate image-only estimates

more calibration, unknowns

24


Conventional images:

most common case

Omnidirectional images:

requires a more complex projection model, additional calibration

generally better motion

25

Improved tracking (1)

For long-term image-based motion estimation, high-quality feature tracking is critical

A. Robustness to harsh overall motion

B. Robustness to poor image texture

C. Uniform image coverage

D. Squeeze every feature for all it’s worth

26


A. Robustness to harsh image motion

large overall image motions

highly discontinuous 2D motion fields from nonplanar scenes

27


B. Robustness to poor image texture

low texture

repetitive texture

one-dimensional texture

28


C. Uniform image coverage

features should span the entire image

features that have become clumped together are redundant

29


D. Squeeze every feature for all it’s worth

features should be tracked:

despite changes in appearance due to:

even if they are near the image boundary

distance

relative angle

specularities

30


smalls correlation image feature tracker

Picture of l.s. here

31


Leonard Smalls smalls

tracker and lone biker of the apocalypse

correlation tracker

especially hard on the little things

safe for use with the little things

mama didn’t love him not applicable

all the powers of hell at his command

maybe in 2.0

32


Eliminates the heuristics normally used for…

handling large motions

determining when a point has:

extracting features

been mistracked

become occluded

left the image

33


…and instead:

constrains tracking to epipolar lines

uses only 3D geometric consistency for determining when a point has:

chooses features based on image coverage

been mistracked

become occluded

left the image

34


smalls uses…

SIFT keypoint extraction and matching

RANSAC

two-frame SFM

…to determine…

consistent matches between the images

35


…and thence:

epipolar geometry

center of search range along epipolar lines

36

Hyperion (1)

37

Hyperion (2)

38

Hyperion (3)

39

Hyperion (4)

40

Hyperion (4)

41

Hyperion (5)

42

Hyperion (6)

43

Experiments (7): CMU crane

Crane capable of translating a platform…

…through x, y, z…

…through a workspace of about 10 x 10 x 5 m

44

Experiments (8): CMU crane, cont.

y tr

ansl

atio

n (m

eter

s)

x translation (meters)3.00.0-3.0

-3.0

0.0

3.0

(x, y) translation ground truth

45

Experiments (9): CMU crane, cont.z

(m)

time

3.5

4.5

z translation ground truth

No change in rotation

46


47


Hard sequence:

• Each image contains an average of 56.0 points

• Each point appears in an average of 62.3 images (4.4% of sequence)

• Image-and-inertial online algorithm applied

• 40 images used in batch initialization

48


49


Estimated z camera translations

50


6 DOF errors, after scaled rigid alignment:

Rotation: 0.14 radians average

Translation: 31.5 cm average (0.9% of distance traveled)

Global scale error: -3.4%

51

Future work (1)

Closing the loop to deal with drift:

(1) recognizing revisited features

(2) exploiting revisited features in the estimation

1 long-term image-based motion estimation dennis strelow

Documents