Real-Time Tracking with Mean Shift

Presented by: Qiuhua LiuMay 6, 2005

Outline

Color model for the targetIntroduction to mean-shift Tracking algorithm with mean shiftCompassion with Particle Filter algorithm with the similar color model

Color Model for the Target

The target is represented by an ellipsoidal region in the image, normalized to a unit circle. Let be the normalized pixel locations in the region centered at 0. The probability of the feature(color) of the target was modeled by the its histogram with kernel :

niix ,...1*}{

bins ,...1,])([)||(||ˆ1

*2* muuxbxkCqn

iiiu

K

The kernel has a convex and monotonic decreasing kernel profile , assigning small weights to pixels farther away from the center.

Kk

The profile of kernel is defined as a function such that

Let be the normalized pixel locations of the target candidates, centered at y in the current frame. The target candidate is modeled as:

hniix ,...1*}{

bins ,...1,])([)()(ˆ1

*2

muuxbh

xykCyp

hn

ii

ihu

)||(||)( 2xkxK

Rk ],0[:K

Target Candidate

Similarity Function

The similarity function is defined as the metric distance between the candidate and the target model:

Choose as the Bhattacharyya coefficients (it is a divergence type measure)

Minimizing the distance is equivalent to maximizing .

]),(ˆ[1)( qypyd

m

uuu qypqyp

1

)(ˆ]),(ˆ[

Maximization with Mean Shift

Assume the target candidate histogram does not change drastically, using Taylor expansion around the values at location :

Only need to maximize the second term, which is the density estimate with kernel profile k(x) at y in the current frame, with the data being weighted by wi.

)(ˆ 0ypu 0y

)ˆ(ˆ

ˆ)(ˆ

2

1ˆ)ˆ(ˆ

2

1]),(ˆ[

0110 yp

qypqypqyp

u

um

uu

m

uuu

hn

i

ii

hm

uuu h

xykw

Cqyp

1

2

10 )(

2ˆ)ˆ(ˆ

2

1

where ])([)ˆ(ˆ

ˆ

1 0

uxbyp

qw i

m

i u

ui

Mean Shift

First Introduced by Fukunaga and Hostetler in 1975 [1], Mean shift is a non-parametric, iterative procedure to find the mode of a density function represented by a set of samples and a Kernel K :

niix ,...1}{

n

i

id h

xxK

nhxf

1

1)(ˆ d: dimension of data;

h: band width.

n

i

idk h

xxk

nhxf

1

21

)(ˆ

With the definition of the Profile of a kernel:

With mean shift method, the kernel is recursively moved from the current location to the new location until converge with:

For a kernel with a convex and monotonic decreasing kernel profile, it is guaranteed to converge (to local maxima)

0y1y

h

h

n

ii

i

n

ii

ii

hxy

gw

hxy

gwxy

1

2

0

1

2

0

1

)ˆ

(

)ˆ

(ˆ

where ).()( xkxg

Mean Shift

The Epanechnikov kernel has a profile:

Then

where cd is the volume of the unit d -dimensional sphere.

otherwise0

1 if)1)(2(2

1)(

1 xxdcxK d

E

constxkxg )()(

One Normally Used Kernel

h

h

n

i i

n

i ii

w

wxy

1

11ˆ (*)

Tracking Algorithm with Mean Shift

Very Simple: Given the target model and its location in

the previous frame. 1. Initialize the location at the current frame with

. 2. Compute the next location according to (*). 3. Iterate 1 and 2 until converge.

uq 0y

0y

1y

Tracked Result:

Mean Shift Maximization:

Summary and Comparison to Particle Filter Method

Advantage: Good color histogram model and distance

measure. Deterministic method: the mean shift usually

converged at 2 to 3 iterations –Fast. Disadvantage:

Sometimes get stuck at local minimum. Difficult to handle abrupt motion: Due to use of the kernels, the center of the target

in the current frame has to be covered by the target model in the previous frame. Otherwise, the local maximum of the Bhattacharyya coefficient would not be a reliable indicator.

Connection to Particle Filter Tracking

Adopting the same distance measure, Jaco Vermaak [4][5] proposed the following observation likelihood function for probabilistic tracking with particle filters and VB inference :

The histogram does not necessarily need a kernel.

)])(,[exp()|( 2tuutt xpqdxyp

Comparison

Top: Deterministic with Mean-shift

Bottom: Probabilistic with particle filters

References

[1] Fukunaga et al, “The Estimation of the Gradient of a Density Function, with Applications in Pattern Recognition”, IEEE Trans. on Information Theory, 1975

[2] Dorin Comaniciu et al, “Real-time Tracking of Non-Rigid Objects Using Mean Shift”, CVPR 2000.

[3] Dorin Comaniciu et al, “Kernel-Based Object Tracking”, IEEE Trans . On Pattern Analysis and Machine Learning , May 2003.

[4] Jaco Vermaak et al[5] Jaco Vermaak et al, “Variational Inference for

Visual Tracking”, CVPR, 2003