7B-L2 The Kalman filter

CS4495/6495 Introduction to Computer Vision

Tracking as induction

Base case:

• Assume we have some initial prior that predicts state in the absence of any evidence: 𝑃(𝑋0)

• At the first frame, correct this, given value of 𝑌0 = 𝑦0

Tracking as induction

Given corrected estimate for frame 𝑡:

• Predict for frame 𝑡 + 1

• Correct for frame 𝑡 + 1

predict correct

Prediction:

0 1 1 1 0 1 1| , , | | , ,t t t t t t tP X y y P X X P X y y dX

Last time: Prediction and correction

dynamics model

corrected estimate

from previous step

Last time: Prediction and correction

Correction:

0 1

0 1

0 1

| | , ,| , , ,

| | , ,

t t t t

t t

t t t t t

t

P y X P X y yP X y y

P y X P X y y dXy

observation model

predicted estimate

Linear Dynamics Model Dynamics model: State undergoes linear transformation plus Gaussian noise

1~ ,tt t t dN D x x

Linear Measurement Model Observation model: Measurement is linearly transformed state plus Gaussian noise

~ ,tt t t mN M y x

Example: Constant velocity (1D) State vector is position and velocity

1 1

1

( )t t t

t t

p p t v

v v

t

t

t

px

v

1

1

1

1

0 1

t

t t t

t

ptx D x noise noise

v

Example: Constant velocity (1D) Measurement is position only

1 0t

t t

t

py Mx noise noise

v

Example: Constant acceleration (1D) State vector is position, velocity & acceleration

1 1

1 1

1

( )

( )

t t t

t t t

t t

p p t v

v v t a

a a

t

t t

t

p

x v

a

1

1 1

1

1 0

0 1

0 0 1

t

t t t t

t

t p

x D x noise t v noise

a

Example: Constant acceleration (1D) Measurement is position only

1 0 0

t

t t t

t

p

y Mx noise v noise

a

The Kalman Filter

• A method for tracking linear dynamical models in Gaussian noise

• Predicted/corrected state distributions are Gaussian • You only need to maintain the mean and covariance

• The calculations are easy (all the integrals can be done in closed form)

The Kalman Filter

The Kalman Filter: 1D state

,t t

0 1, ,t tP X y y 0 , ,t tP X y yPredict Correct

Time advances (from t–1 to t)

Mean and std. dev. of predicted state:

,t t

Mean and std. dev. of corrected state:

Make measurement

Linear dynamics model defines predicted state evolution, with noise

Want to estimate distribution for next predicted state

1D Kalman Filter: Prediction

2

1~ ,t t dX N dx

0 1 1 1 0 1 1, , | | , ,t t t t t t tP X y y P X X P X y y dX

Linear dynamics model defines predicted state evolution, with noise

The distribution for next predicted state is also a Gaussian

Update the mean:

Update the variance:

1D Kalman Filter: Prediction

1t td

2 2 2

1( ) ( )t d td

2

0 1, , , ( )t t t tP X y y N

2

1~ ,t t dX N dx

Mapping of state to measurements:

Predicted state:

Want to estimate corrected distribution:

1D Kalman Filter: Correction

2

0 1, , , ( )t t t tP X y y N

0 1

0

0 1

| | , ,, ,

| | , ,

t t t t

t t

t t t t t

P y X P X y yP X y y

P y X P X y y dX

2~ ,t t mY N mx

Kalman: With linear, Gaussian dynamics and

measurements, the corrected distribution is:

2

0 , , , ( )t t t tP X y y N

2 2

2 2 2

( )

( )

t m t tt

m t

my

m

Update the mean: 2 2

2

2 2 2

( )( )

( )

m tt

m tm

Update the variance:

1D Kalman Filter: Intuition 2 2

2 2 2

( )

( )

t m t tt

m t

my

m

From:

Dividing throughout by 𝑚2…

22

2

22

2

( )

( )

t m tt

t

mt

y

m m

m

1D Kalman Filter: Intuition

What is this?

• The weighted average of prediction and measurement based on variances!

2 2

2 2 2

( )

( )

t m t tt

m t

my

m

22

2

22

2

( )

( )

t m tt

t

mt

y

m m

m

Measurement guess of 𝑥

Variance of 𝑥 computed from the measurement

Variance of prediction

From:

Prediction of 𝑥

Prediction vs. correction

t t 2( ) 0t

2 2

2 2 2

( )

( )

t m t tt

m t

my

m

2 22

2 2 2

( )( )

( )

m tt

m tm

tt

y

m 2( ) 0t

The measurement is ignored!

The prediction is ignored!

( 0)t What if there is no prediction uncertainty?

( 0)m What if there is no measurement uncertainty?

1D Kalman Filter: Intuition 2

2

2

22

2

( )

( )

t m tt

t

mt

y

m m

m

Also:

( ) ( )t tt t t

t

y ya b a b b

m m

a b a b

1D Kalman Filter: Intuition

( ) ( )t tt t t

t

y ya b a b b

m m

a b a b

(

)

)

(

tt

t

tt

t t

yb

m

a

yk

mb

Predicted Residual Kalman Gain

State is 2d: position + velocity Measurement is 1d: position

Recall: constant velocity model example

time p

osi

tio

n

Actual state

measurements

Kalman filter processing

time

o state

x measurement

* predicted mean estimate

+ corrected mean estimate

bars: variance estimates before and after measurements

po

siti

on

time

po

siti

on

Kalman filter processing

o state

x measurement

* predicted mean estimate

+ corrected mean estimate

bars: variance estimates before and after measurements

time

po

siti

on

Kalman filter processing

o state

x measurement

* predicted mean estimate

+ corrected mean estimate

bars: variance estimates before and after measurements

time

po

siti

on

Kalman filter processing

o state

x measurement

* predicted mean estimate

+ corrected mean estimate

bars: variance estimates before and after measurements

PREDICT CORRECT

1t t tx D x

1 t

T

t t t t dD D

t t t t t t

x x K y M x

t t t t

I K M

1

t

T T

t t t t t t mK M M M

More weight on residual when measurement error covariance approaches zero.

Less weight on residual as a priori estimate error covariance approaches zero.

N-dimensional

Tracking with KFs: Gaussians

Initial (prior) estimate

prediction

X

Y

update

X

Y

measurement

X

Y

X

Y

prior

measurement evidence

posterior

( | ) ( | ) ( )p x y p y x p x

( ') ( ' | ) ( )p x p x x p x dx

Posterior 𝑡 Prior 𝑡+1

A Quiz

Does this agree with your intuition?

prior

measurement evidence

posterior?

( | ) ( | ) ( )p x y p y x p x

Kalman filter pros and cons

•Pros

• Simple updates, compact and efficient

Kalman filter pros and cons

•Cons

• Unimodal distribution, only single hypothesis

• Restricted class of motions defined by linear model

• Extensions call “Extended Kalman Filtering”

• So what might we do if not Gaussian? Or even unimodal?