multi-objekt-tracking und grid-mapping für die … · 2017. 9. 22. · seite 2 tracking and...

1

Multi-Objekt-Tracking und Grid-Mapping für die

Fahrzeugumfelderfassung unter Verwendung von Random-Finite-Sets

Stephan Reuter, Karl Granström, Dominik Nuss, Alexander Scheel

Tracking and Grid-Mapping Using RFS | S. Reuter | 2017Seite 2

Motivation

Multi-object tracking and grid mapping are widely used approaches for

environment perception

Complexity increases with level of automation

Requirements

Real-time capability

Integrated existence estimation for objects / obstacles

High object density

level of automation

2


Sensors for Environment Perception

Mono-Camera

Lidar

Radar

Versuchsträgerfahrzeuge der Universität Ulm

Stereo-Camera


Bayes Filter

Implementation

Kalman Filter

Sequential Monte-Carlo Methods (Particle Filter)

motion

state space

measurement space

xk

xk+1

zk

zk+1

updateprediction

3


Multi-object tracking often realized using several independent Kalman filters

Heuristics between the individual steps lead to loss of information

Joint Probabilistic Data Association (JPDA) filter

Based on computing marginal data association probabilities

Multi-Hypothesis Tracker (MHT)

Based on maintaining different hypotheses about data association.

Pruning and merging of hypotheses

Classical Approach: Multi-Instance Kalman Filter

Raw data Tracks

? ?

?

?


Multi-object tracking often realized using several independent Kalman filters

Heuristics between the individual steps lead to loss of information

Joint Probabilistic Data Association (JPDA) filter

Based on computing marginal data association probabilities

Multi-Hypothesis Tracker (MHT)

Based on maintaining different hypotheses about data association.

Pruning and merging of hypotheses

Classical Approach: Multi-Instance Kalman Filter

Raw data Tracks

Random-Finite-Set

Approach

4


Random Finite Set

Set-valued random variable

Realization represents multi-object state

Modelling of object interdependencies possible

Random Finite Sets (RFS)

state vectors RFS


Multi-Object BayesFilter

updateprediction

state space

measurement space

XkXk+1

Zk Zk+1

k k+1

Motion

Disappearance

Appearance

multi-object measurement model

Detection

Missed detection

Clutter

multi-object Markov density

5


Multi-Object Bayes Filter – Implementation

Sequential Monte-Carlo Methods

(Vo 2005)

Probability Hypothesis Density (PHD) Filter

(Mahler 2003)

Cardinalized PHD Filter

(Mahler 2007)

Cardinality Balanced Multi-Bernoulli Filter

(Vo 2009)

-Generalized Labeled Multi-Bernoulli Filter

(Vo 2011)

Labeled Multi-Bernoulli Filter

(Reuter 2014)

Filter Algorithm

Moment

Approximation

Parameter

Approximation


Multi-Object Bayes Filter: Moment-Approximations

PHD-Filter (Mahler2003): Approximation using first statistical moment

CPHD-Filter (Mahler2007): first statistical moment & cardinality distribution

PHD or

intensity function

Cardinality

distribution

pdf over number of

objects

first moment of

multi-object distribution

vkvk-1

rkrk-1

6


Parameter-Approximation: Labeled Random Finite Sets

Labeled Multi-Bernoulli (LMB) Random Finite Set

r=0.99 r=0.05r=0.79r=0.79existence

probability

Track-IDs

spatial

distribution

-Generalized Labeled Multi-Bernoulli (-GLMB) Random Finite Set

w=0.2 w=0.74 w=0.05

Transformation Approximation


Multi-Sensor Multi-Object Tracking

Sensor-independent fusion using Labeled-Multi-Bernoulli-Filter

Facilitates sensor replacement

Classification of objects (car, bike, pedestrian,…)

Generic

Sensor-

Inte

rface

LMB Filter

-

LMB Prediction

Gating

Birth model

Track-Management

LMB -GLMB

-GLMB-Update

-GLMB LMB

Camera Preprocessing

Lidar Preprocessing

Radar Preprocessing

7


Video Urban Traffic


Extended Object Tracking

State of the Art: point-target tracking using heuristic preprocessing

algorithms

Extended Object Point-Object Unresolved Objects

Raw data Tracks

Random-Finite-Set

Approach:

Labeled-Multi-

Bernoulli-Filter

8


Extended Object Tracking

State of the Art: point-target tracking using heuristic preprocessing

algorithms

Extended Object Point-Object Unresolved Objects

Raw data Tracks

Random-Finite-Set-Filter for Extended Objects


Modeling of Extended Objects

Complexity

Point Basic Shape Arbitrary Shape

Possible Reflection Centers (in 2D)

Contour Surface

9


Gamma Gaussian Inverse Wishart Model

Elliptical shape

Assumption: Measurements are generated by a multi-variateGaussian distribution with mean at the objects‘ position andcovariance matrix

Number of measurements isPoisson with mean

Combined state vector

Gaussian distributed kinematic state


Results

LMB filter with joint prediction and update implementation (red)

Computation times (Intel i7 2600, C++, no parallelization)

Average of ~100 targets

Mean: 7.0 ms

Max: approx. 25 ms

Comparison: standard

implementation using

Murty’s algorithm (blue)

Mean: 622.0 ms

Max: aprox. 10000 ms

10


Modeling of Extended Objects

Complexity

Point Basic Shape Arbitrary Shape

Possible Reflection Centers (in 2D)

Contour Surface


Gaussian Process Object Model

• Use Gaussian process to describe the radius function

• Consider combined state vector for kinematics and radius functiong mit

EKF

Combined

State

Kinematic

State

11


Gaussian-Process

Definition:

with first two moments

• Mean function

• Covariance function:

Representation as a multivariate Gaussian:


Gaussian Process

3-Sigma-Bounds

12


Recursive Gaussian Process

3-Sigma-Bounds


Extended Object Tracking Using Gaussian Processes

13


Intersection Scenario Using Gaussian Processes


Intersection Scenario Using Gaussian Processes

14


Challenges

Precise shape representation requires high number of inputs (sampling

points)

Matrix dimension: number of kinematical states + number of inputs

Numerical issues during calculation of matrix inverse

Rao-Blackwellized implementation

Representation of kinematic state using particles (approx. 1000)

Each particle holds a GP or a mixture of GPs for the objects’ extent

Parallelization is required


Environment Representation using Grid Maps

Separation of environment into discrete cells

For each cell: infer on cell state (occupied or free) using sensor data

Assumption: statistically independent cells

[1] Thrun, S.: Probabilistic Robotic. The MIT Press, 2005.

[2] Elfes, A.: Using occupancy grids for mobile robot perception and navigation. IEEE Computer, 1989, Vol. 22.6, pp. 46-57.

15


Static Grid Map

Assumes stationary environment

Dynamic objects lead to artefacts in grid map


Robust estimation of dynamic objects without error-prone clustering

Independent of object shape

and class

Fusion of heterogenous sensor

data possible

Based on PHD and Bernoulli filter

Dynamic Grid Map

Laser Raw Data

Dynamic Grid Mapping

Clustering

Object-Tracking

Radar Raw Data

16


Dynamic Grid Map – Principle

Occupancy for current timerepresented by particles

“Motion” of occupancy inprediction step,uncertainty about actualmovement leads to blurring

Update of particle weightsusing measurement grid,new cells may be occupiedafterwards


Current Implementation

Random-Finite-Set based particle filter implementation

Parallelization using CUDA (GTX970+)

Huge amount of particles required (3-10M)

Prediction step facilitates embarrassingly parallel computation

Update step

Association of particles to cells using sorting algorithm

Update of particles in cell embarrassingly parallel

Resampling based on sorting

Real-time capable for up to 10M particles

17


Dynamic Grid Map – Video


Conclusions

Random Finite Sets provide a rigorous mathematical framework for object

tracking and grid mapping

Multi-Object Tracking

Very efficient implementation of point-target tracking and extended object

tracking with basic shapes

Tracking of arbitrarily shaped objects requires parallelization and

numerical issues need to be considered

Dynamic Grid Mapping

Well suited to represent arbitrarily shaped objects

Parallelization required to handle huge number of particles

18


Additional References / Matlab-Code

B.-N. Vo et. al: „Multitarget Tracking“, http://ba-ngu.vo-au.com/vo/VMBCOMV_MTT_WEEE15.pdf

Multi-Object-Tracking using Random Finite Sets:

R. Mahler: „Statistical Multi-Source Multi-Target Information Fusion“

R. Mahler: „Statistics 101“ / „Statistics 102“

S. Reuter: „Multi-Object Tracking Using Random Finite Sets“

https://oparu.uni-ulm.de/xmlui/handle/123456789/3231

K. Granström, M. Baum, S.Reuter: „Extended Object Tracking: Introduction, Overview and

Applications“, https://arxiv.org/abs/1604.00970

S. Hörmann, M. Bach, K. Dietmayer: „Dynamic Occupancy Grid Prediction for Urban Autonomous

Driving: A Deep Learning Approach with Fully Automatic Labeling”,

https://arxiv.org/abs/1705.08781

D. Nuss et. al: „A Random Finite Set Approach for Dynamic Occupancy Grid Maps with Real-Time

Application”, https://arxiv.org/abs/1605.02406

Matlab Code Multi-Objekt-Tracking: http://ba-tuong.vo-au.com/codes.html



multi-objekt-tracking und grid-mapping für die … · 2017. 9. 22. · seite 2 tracking and...

Documents