using the kalmanfilter extended kalmanfilter · pdf fileusing the kalmanfilter extended...
TRANSCRIPT
SA-1
Using the Kalman filterExtended Kalman filter
Doz. G. BleserProf. Stricker
Computer Vision: Object and People Tracking
Outline
• Recap: Kalman filter algorithm• Using Kalman filters• Extended Kalman filter algorithm
2
Recap: Kalman filter
• Bayes update rule + linear Gaussian models Kalman filter
3
posterior likelihood motion model posterior at t-1
measure predictcorrect
),;( 1 tttttt RuBxAxN
),;( tttt QxCzN),;( tttxN
),;( 111 tttxN
4
Recap: Kalman filter algorithm
1. Kalman_filter( t‐1,t‐1, ut, zt):
2. Prediction:3.4.
5. Correction:6.7.8.
9. Return t,t
ttttt uBA 1
tTtttt RAA 1
1)( tTttt
Tttt QCCCK
)( tttttt CzK
tttt CKI )(
Requirements:
• Initial belief is Gaussian
• Linear Gaussian state‐space model
SA-1
Using Kalman filters
Application to a simple 2D tracking problem
Some slides based on K. Smith
5
Track an aircraft in a video sequence
6
• Assumptions:– Image processing provides
noisy 2D positions of aircraft– Motion model is const. pos.→ Linear Gaussian system
• Formulate the state‐space model:
~ 0,
~ 0,
Track an aircraft in a video sequence
7
• State: 2D image position
• Measurement: 2D image position
Track an aircraft in a video sequence
8
• Motion/dynamic model:
1 00 1
00
00
, ~ 0,
• Measurement model:
C 1 00 1
Q0
0, ~ 0, Q
9
1. Kalman_filter( t‐1,t‐1, ut, zt):
2. Prediction:3.4.
5. Correction:6.7.8.
9. Return t,t
ttttt uBA 1
tTtttt RAA 1
1)( tTttt
Tttt QCCCK
)( tttttt CzK
tttt CKI )(
1 00 1
00
00
C 1 00 1 Q
00
Track an aircraft in a video sequence
Note: ≔ |
Track an aircraft in a video sequence
• Prediction: | Σ | Σ• Predicted measurement predicted state• Measurement update:
0,
, Σ
≔ Innovation (residual)
Σ Σ |
|
Σ | Σ |
Kalman gain
Discussion
• Prediction: | Σ | Σ• Predicted measurement predicted state• Measurement update:
0,
≔ Innovation (residual)
Σ Σ |
|
Σ | Σ |
Kalman gain What happens, if:• → ∞• → ∞• →
Discussion
• What happens, if the object motion violates our model assumption• Effects?
– Big innovations, tracking lags• Possible solutions?
– Adapt noise settings (increase process noise/reduce meas. noise)– Choose a better model
We can estimate velocity!
past measurements
prediction
Simple constant velocity model
• Motion/dynamic model:
• Measurement model:
14
),0(00100001
QN
yxyx
yx
obs
obs
2
2
00
Q
),0(
10000100
010001
1| RN
yxyx
tt
x tt
2
2
00000000000000
R
Constant velocity model
Assuming that the object follows a const. velocity model: Better prediction Cont’d estimate, even if we don’t
have measurements
15
Discussion
16
• Constant position model • Constant velocity model
t
xyxy
x
t
xy
x
Discussion
17
How to track very agile motions?• Increased process noise• Constant acceleration model
More reactive tracking Instable, if no measurements
available• Alternative solutions:
– Switched model– Knowledge from airplane
available?
Kalman limitations
• Uni‐modal distributions fail for unpredicted motion• Problem, if image processing depends on prediction
(tracking vs. detection)
Prediction too far from actual location
to recover
18
Possible solutions:
• Switched model (ground contact as control input)
• Multiple hypothesis tracking
Kalman filter limitations
• Applies to linear Gaussian models• Many visual tracking problems are nonlinear, e.g. as soon as we
move to tracking in 3D
19Courtesy of G. Panin
Humanoid robot catches flying balls
• Several cameras observe flying balls• The 3D trajectory is estimated• A robot is supposed to catch the balls nonlinear estimation problem
20Courtesy of U. Frese
SA-1
Extended Kalman filter
Slides based on S. Thrun
21
22
Nonlinear models, Gaussian noise
• Motion model:
• Measurement model:
),,( 11 ttttttttt uxguBxAx
),( ttttttt xhzxCz
Nonlinear function
(most general model)
ttt uxg ),( 1
ttt xhz )(
~ 0,
~ 0,
Nonlinear function
Often used: model with
additive noise
Linear function
23),(~
),(~
TAABANY
BAXYNX
Nonlinear function
• PDF obtained from 500.000 Monte Carlo samples + histogramming• Then sample mean and covariance
24
Bayes update rule
• Problem: We do not stay in the Gaussian world, if motion and/or measurement models are nonlinear functions of the state
• No general closed‐form solution for Bayes filter• Approximate solutions:
– Keep the functions and approximate distributions– Linearize the functions and use again the Kalman filter
25
Non-GaussNonlinear+Gauss
Nonlinear+Gauss Gauss
EKF linearization
Monte Carlo transform + sample statistics blue GaussianFirst Order Taylor approximation red Gaussian
26
Mismatch=
linearization error
EKF linearization: additive noise
• Linearize and with First Order Taylor Expansion• Linearization points: best available estimate• Prediction (linearize around ):
• Correction (linearize around ̅ ):
27
)(),(),(
)(),(),(),(
1111
111
111
ttttttt
ttt
tttttt
xGugxug
xx
ugugxug
)()()(
)()()()(
ttttt
ttt
ttt
xHhxh
xx
hhxh
Jacobian matrix
Jacobian matrix
28
Extended Kalman filter: additive noise
1. Extended_Kalman_filter( t‐1,t‐1, ut, zt):
2. Prediction:3.4.
5. Correction:6.7.8.
9. Return t,t
),( 1 ttt ug
tTtttt RGG 1
1)( tTttt
Tttt QHHHK
))(( ttttt hzK
tttt HKI )(
1
1),(
t
ttt x
ugG
t
tt x
hH
)(
ttttt uBA 1
tTtttt RAA 1
1)( tTttt
Tttt QCCCK
)( tttttt CzK
tttt CKI )(
Linearization: non‐additive noise
• Linearization points: best estimate and 0 (assuming zero‐mean noise)
• Prediction (linearize around and 0):
• Correction (linearize around ̅ and 0):
29
tttttttttt
tt
ttttt
t
ttttttt
WxGugxug
ugxx
ugugxug
)(),(),,(
),,()()0,,()0,,(),,(
1111
111
1
111
tttttttt
tt
tttt
t
tttt
VxHhxh
hxx
hhxh
)()0,(),(
),()()0,()0,(),(
Jacobians
Jacobians
30
Extended Kalman filter: non‐additive noise
1. Extended_Kalman_filter( t‐1,t‐1, ut, zt):
2. Prediction:3.4.
5. Correction:6.7.8.
9. Return t,t
),( 1 ttt ug T
tttTtttt WRWGG 1
1)( Tttt
Tttt
Tttt VQVHHHK
))(( ttttt hzK
tttt HKI )(
1
1 )0,,(
t
ttt x
ugG
t
tt x
hH
)0,(
t
tt
hV
),(
t
tttt
ugW
),,( 1
Gauss approximation formula
Let:• beunknown with mean and covariance Σ• , a nonlinear function ofThe Gauss approximation for mean and covariance Σ is:
Σ
∎
31
EKF linearization: problems
• Bigger uncertainty → bigger lineariza on error32
33
EKF linearization: problems
EKF linearization: problems
• Higher local nonlinearity → bigger lineariza on error34
35
EKF linearization: problems
Extended Kalman filter: summary
• Kalman Filter was optimal for linear Gaussian models• Problem with EKF: the functions and are linearized, hence
approximated Not optimal (we keep Gaussian representations, but is not Gaussian)
• If and are highly nonlinear (difficult to quantize), the estimation may become unstable
• Good results, if functions are approx. linear around mean• Less certain estimate is more affected by linearization errors
EKF is not an optimal Bayesian tracker
• However, it is successfully used in many applications
36
(Extended) Kalman filter: limitations
• Question: is the (E)KF feasible for the above estimation problem?• Implications of using Gaussians:
– Gaussians are unimodal! They possess a single maximum!– Typical for many tracking problems: posterior focused around true
state with small margin of uncertainty– Poor match for multimodal problem (global estimation)
37
(Extended) Kalman filter: limitations
M. Breitenstein, F. Reichlin, B. Leibe, E. Koller-Meier, L. Van Gool, Robust Tracking-by-Detection using a Detector Confidence Particle Filter , International Conference on Computer Vision (ICCV), 2009
No non‐Gaussian observation models
38
More Bayes filters (for the most interested ones)
• Iterated Extended Kalman filter• Unscented Kalman filter• Information filter
• Histogram filter• Particle filter
• Interacting multiple model filter• Multiple hypothesis tracking
• Bibliography: Books by Sebastian Thrun and Y. Bar‐Shalom on http://av.dfki.de
39
Alternatives for (E)KF
Nonlinear functions, multiple modes, non‐Gaussian noise (in general less efficient)
Mixture of Gaussians, fixed number of modes, different
model assumptions
Outlook
Next lectures:• State‐space models for 3D visual tracking and applications• People tracking based on RGB‐D dataOral exam:• Suggested periods:
– 14. ‐ 17.08.2012– (27. ‐ 29.08.2012)– 10. ‐ 14.09.2012
• Please write an email to [email protected] with cc to [email protected] and ask for an appointment. The email should contain:– Full name, matriculation number, faculty, course of studies– Preferred date(s) and time
40