Tracking

using the

Kalman Filter

Point Tracking

Estimate the location of a given point along a sequence of images.

(x0,y0)

(xn,yn)

Object Tracking

– Generate some conclusions about the motion of the scene, objects, or the camera, given a sequence of images.

– Knowing this motion, predict where things are going to project in the next image, so that we don’t have so much work looking for them.

– For example- unstable camera + Walking man:

a. Stabilize the camera using the dominant motion ( find motion parameters ! )

b. Assume that the man translates horizontally.

Modeling “noise” or “uncertainty”

rotation

The General Model

kkkk wxAx 1

kkkk vxHz

Dynamics Process noise~N(0,Q)

ProjectionMeasurement noise

~N(0,R)

Prediction

kk xAx ˆˆ 1

QApAp Tkk 1

Estimated state

Estimated uncertainty / noise

Update

Updated state

Updated uncertainty / noise

)ˆ(ˆˆ 1111 kkkk xHzKxx

11 )( kk pKHIp

111 )(

RHPHHPK Tk

Tk

The weighting factor

kk xAx ˆˆ 1

QApAp Tkk 1

)ˆ(ˆˆ 1111 kkkk xHzKxx

11 )( kk pKHIp

111 )(

RHPHHPK Tk

Tk

Prediction Update

Summery

Gaussian: “Normal” distribution

• 1D Gaussian:

• General Gaussian:

),(~ vNx

v

xx

eexp 2

)(

2

)( 2

2

2

)(

TxVxexp )()(2/1 1

)(

),(~ VNx

Adding two information sources

• We are given to information sources: Z1 and Z2

• Both are normally distributed (v1 > v2)

• We would like to believe more to Z2, but still use the information from Z1 !

• Mathematically:

),(~)|( 111 vNZxp ),(~)|( 222 vNZxp

)|()|(),|( 2121 zxpzxpzzxp

The solution

1

21

2

)(

1)|( v

x

ezxp

2

22

2

)(

2 )|( v

x

ezxp

2

22

1

21

2

)(

2

)(

2121 )()(),( v

zx

v

zx

eexzpxzpxzzp

),(~),|( 21 vNZZxp

221

11

21

2 vv

v

vv

v

21

111

vvv

The solution (cont’)

)( 121 k

21

21

vv

vvv

21

1

vv

vK

The merging of two Gaussians

A “noisy” measure, be don’t believe it very much

A more reliable measure

The merging of two Gaussians (cont’)

The result is a new Gaussian with a smaller variance than the original ones !

Why to use the normal distribution?

• Simple to manipulate

• Minimize the squared error.

• The “big numbers” low.

• The distribution of many “natural” things.

What happens when we have a “wrong” estimation of the measurements variance ?

The correct variance (The same variance that was used to simulate the points)

The variance is too small:The estimation doesn’t converge

The variance is too large:The convergence is very slow

Tracking using the Kalman Filter Two more examples.

The General Model

kkkk wxAx 1

kkkk vxHz

Dynamics Process noise

ProjectionMeasurement noise

Example 1: Estimating a constant

kk xx 1 )0,( kk wIA

kkk vxz

Measurement noise

)( IHk

1, Rzx kk

Prediction: kk xx ˆˆ 1

kk pp 1Update

)ˆ(ˆˆ 1111 kkkk xzKxx

11 )1( kk pKp

11

1

kk

k

VP

PK

We can combine the prediction and update

)ˆ(ˆˆ 11 kkkk xzKxx

kk pKp )1(1

1

kk

k

VP

PK

Claim1:

Claim2:

Conclusion:The Kalman filter gives a weighted

mean !

k

i ik VP 0

11

i

k

i ik

k ZVP

X

0

1ˆ

i

k

i iik Z

VVX

0

1)

/1

1(ˆ

Example 2 : Shihab4

In X: constant velocityIn Y: constant

acceleration

Example2 -dynamics

kkk tvyy 1

kkk tavv 1

kk aa 1

kkk tuxx 1

kk uu 1

k

k

k

k

k

k

u

x

a

v

y

X

Example2 -measurements

• For each possible location, give a score• Normalize the sum of the scores to 1.• The result is a matrix of “probabilities” for

each location.• Fit a 2D Gaussian to this matrix, whose

center is given by:

Given an image of the missile (or other source of information):

k

kk y

xZ