Loris Bazzani*, Marco Cristani*†, Vittorio Murino*†

Speaker: Diego Tosato*

*Computer Science Department, University of Verona, Italy

†Istituto Italiano di Tecnologia (IIT), Genova, Italy

Collaborative Particle Filters for Group Tracking

This research is founded by the EU-Project FP7 SAMURAI,grant FP7-SEC- 2007-01 No. 217899

Analysis of the problem (1)

2

Multi-Target Tracking: Estimate the trajectories of objects of

interest, keeping their identification over the time

Well-investigated problemState-of-the-art methods are very effective

and efficient

Multi-Group Tracking:Estimate the trajectories of the groups of

objects, keeping their identification over the time

Not Well-investigated problemFew methods in the State of the art

Analysis of the problem (2)

Why it is a hard taskMethods for multi-target tracking failsGroups are highly structured entityHard to model the complex dynamicsStrong appearance variations over the timeIntra- and inter-group occlusions phenomenaWhat is a group?

Motivation:Highlighting social behaviors among

individuals3

Outline

4

Overview of the proposed methodParticle Filtering Multi-Object Tracking (MOT)Multi-Group Tracking (MGT)Collaborative Particle Filters (Co-

PF)ResultsConclusions

Overview of the proposed method

5

Two separate particle filtersMulti-object tracker (MOT) models each individual

separatelyMulti-group tracker (MGT) focuses on groups as

atomic entitiesCoupling of the two processes in a formal

probabilistic framework

Co-PF Model

Particle Filtering for Target Tracking

Recursively calculating the posterior distribution

is defined by

The dynamical modelThe observation modelThe first frame distribution Monte Carlo approximation by a set of

weighted particles6

Multi-Object TrackingExtension to Multi-target

Hybrid Joint-Separable (HJS) Filter [Lanz 2006]Approximation to decompose the joint state space in

single state spaces

HJS is efficient and models the interactions among targets

We “just” need to defineSingle-object dynamical and the single-object observation

models

7 [Lanz 2006] O. Lanz, “Approximate bayesian multibody tracking,” IEEETPAMI, 28(9):1436–1449, 2006.

Multi-Group TrackingUse HJS filterState of the group: Gaussian modelObservation model

Projection of the cylinder into the image

Histogram-based feature as descriptor

Dynamical model : linear motion, perturbed by

Gaussian noise : Gaussian perturbation of its

principal axes, i.e., by varying its eigenvalues and eigenvectors

8

Collaborative Particle FiltersInject the information collected by the MOT into the

MGTMarginalization over the MOT state space

After some approximations, we end up with

It is a combination of MOT and MGT posteriors at time (t-1)

9

MOT posterior at time (t-1)

MGT posterior at time (t-1)

Linking probability

Collaborative Particle FiltersThe linking probability connect the MGT state space to

the MOT state spaceApproximation through the Mixed-memory Markov

Process

Linking likelihood is decomposed in three componentsAppearance similarity: distance between color histogramsDynamics consistency: same direction between group and personGroup membership: spatial proximity between person and group

10

Linking likelihood

Results

11

Compare Co-PF against MGT (without collaboration)An annotated dataset for group tracking does not

existQuantitative evaluation on a synthetic dataset

emulating real scenarios

ATA = Average Tracking AccuracyMOTA = Multiple Object Tracking Accuracy MOTP = Multiple Object Tracking PrecisionFP = False Positive MO = Multiple Objects FN = False Negative TSR = Tracking Success Rate

[Kasturi et al 2009]

[Kasturi et al 2009] R Kasturi, D Goldgof, P Soundararajan, V Manohar, J Garofolo,R Bowers, M Boonstra, V Korzhova, and J Zhang,“Framework for performance evaluation of face, text, and vehicledetection and tracking in video: Data, metrics, and protocol,”IEEE TPAMI, 31(2):319–336, 2009.

Results

12

Qualitative evaluation on publicly available dataset

PETS 2009 dataset http://www.cvg.rdg.ac.uk/PETS2009/a.html

MGT

Results

13

Qualitative evaluation on publicly available dataset

PETS 2009 dataset http://www.cvg.rdg.ac.uk/PETS2009/a.html

Co-PF

Results

14

Qualitative evaluation on publicly available dataset

PETS 2009 dataset http://www.cvg.rdg.ac.uk/PETS2009/a.html

MGT

Results

15

Qualitative evaluation on publicly available dataset

PETS 2009 dataset http://www.cvg.rdg.ac.uk/PETS2009/a.html

Co-PF

Conclusions

16

A probabilistic, collaborative framework for multi-group tracking have been proposed

Additional evidence on the individuals helps the group tracking in an effective way

The results prove that the collaboration between trackers improve the performances

Future directions:Collaboration on the other direction (MGT

MOT)Detection, split, and merge of the groups