computer vision 2 lecture 6 · 2 content of the lecture •single-object tracking •bayesian...
TRANSCRIPT
![Page 1: Computer Vision 2 Lecture 6 · 2 Content of the Lecture •Single-Object Tracking •Bayesian Filtering Kalman Filters, EKF Particle Filters •Multi-Object Tracking •Visual Odometry](https://reader035.vdocuments.mx/reader035/viewer/2022070914/5fb54fd6bd315d6ec778a890/html5/thumbnails/1.jpg)
Computer Vision 2 – Lecture 6 Beyond Kalman Filters (09.05.2016)
Prof. Dr. Bastian Leibe, Dr. Jörg Stückler
[email protected], [email protected]
RWTH Aachen University, Computer Vision Group
http://www.vision.rwth-aachen.de
![Page 2: Computer Vision 2 Lecture 6 · 2 Content of the Lecture •Single-Object Tracking •Bayesian Filtering Kalman Filters, EKF Particle Filters •Multi-Object Tracking •Visual Odometry](https://reader035.vdocuments.mx/reader035/viewer/2022070914/5fb54fd6bd315d6ec778a890/html5/thumbnails/2.jpg)
2
Content of the Lecture
• Single-Object Tracking
• Bayesian Filtering Kalman Filters, EKF
Particle Filters
• Multi-Object Tracking
• Visual Odometry
• Visual SLAM & 3D Reconstruction
Lecture: Computer Vision 2 (SS 2016) – Beyond Kalman Filters
Prof. Dr. Bastian Leibe, Dr. Jörg Stückler
![Page 3: Computer Vision 2 Lecture 6 · 2 Content of the Lecture •Single-Object Tracking •Bayesian Filtering Kalman Filters, EKF Particle Filters •Multi-Object Tracking •Visual Odometry](https://reader035.vdocuments.mx/reader035/viewer/2022070914/5fb54fd6bd315d6ec778a890/html5/thumbnails/3.jpg)
3
Today: Beyond Gaussian Error Models
Figure from Isard & Blake
Lecture: Computer Vision 2 (SS 2016) – Beyond Kalman Filters
Prof. Dr. Bastian Leibe, Dr. Jörg Stückler
![Page 4: Computer Vision 2 Lecture 6 · 2 Content of the Lecture •Single-Object Tracking •Bayesian Filtering Kalman Filters, EKF Particle Filters •Multi-Object Tracking •Visual Odometry](https://reader035.vdocuments.mx/reader035/viewer/2022070914/5fb54fd6bd315d6ec778a890/html5/thumbnails/4.jpg)
4
Topics of This Lecture
• Recap: Kalman Filter Basic ideas
Limitations
Extensions
• Particle Filters Basic ideas
Propagation of general densities
Factored sampling
• Case study Detector Confidence Particle Filter
Role of the different elements
Lecture: Computer Vision 2 (SS 2016) – Beyond Kalman Filters
Prof. Dr. Bastian Leibe, Dr. Jörg Stückler
![Page 5: Computer Vision 2 Lecture 6 · 2 Content of the Lecture •Single-Object Tracking •Bayesian Filtering Kalman Filters, EKF Particle Filters •Multi-Object Tracking •Visual Odometry](https://reader035.vdocuments.mx/reader035/viewer/2022070914/5fb54fd6bd315d6ec778a890/html5/thumbnails/5.jpg)
5
Recap: Tracking as Inference
• Inference problem The hidden state consists of the true parameters we care about,
denoted X.
The measurement is our noisy observation that results from the underlying state, denoted Y.
At each time step, state changes (from Xt-1 to Xt) and we get a new
observation Yt.
• Our goal: recover most likely state Xt given
All observations seen so far.
Knowledge about dynamics of state transitions.
X1 X2
Y1 Y2
Xt
Yt
…
Slide credit: Kristen Grauman
Lecture: Computer Vision 2 (SS 2016) – Beyond Kalman Filters
Prof. Dr. Bastian Leibe, Dr. Jörg Stückler
![Page 6: Computer Vision 2 Lecture 6 · 2 Content of the Lecture •Single-Object Tracking •Bayesian Filtering Kalman Filters, EKF Particle Filters •Multi-Object Tracking •Visual Odometry](https://reader035.vdocuments.mx/reader035/viewer/2022070914/5fb54fd6bd315d6ec778a890/html5/thumbnails/6.jpg)
6
Recap: Tracking as Induction
• Base case: Assume we have initial prior that predicts state in absence of any
evidence: P(X0)
At the first frame, correct this given the value of Y0=y0
• Given corrected estimate for frame t:
Predict for frame t+1
Correct for frame t+1
predict correct
Slide credit: Svetlana Lazebnik
Lecture: Computer Vision 2 (SS 2016) – Beyond Kalman Filters
Prof. Dr. Bastian Leibe, Dr. Jörg Stückler
![Page 7: Computer Vision 2 Lecture 6 · 2 Content of the Lecture •Single-Object Tracking •Bayesian Filtering Kalman Filters, EKF Particle Filters •Multi-Object Tracking •Visual Odometry](https://reader035.vdocuments.mx/reader035/viewer/2022070914/5fb54fd6bd315d6ec778a890/html5/thumbnails/7.jpg)
7
Recap: Prediction and Correction
• Prediction:
• Correction:
1101110 ,,||,,| ttttttt dXyyXPXXPyyXP
Dynamics
model
Corrected estimate
from previous step
Slide credit: Svetlana Lazebnik
ttttt
tttttt
dXyyXPXyP
yyXPXyPyyXP
10
100
,,||
,,||,,|
Observation
model
Predicted
estimate
Lecture: Computer Vision 2 (SS 2016) – Beyond Kalman Filters
Prof. Dr. Bastian Leibe, Dr. Jörg Stückler
![Page 8: Computer Vision 2 Lecture 6 · 2 Content of the Lecture •Single-Object Tracking •Bayesian Filtering Kalman Filters, EKF Particle Filters •Multi-Object Tracking •Visual Odometry](https://reader035.vdocuments.mx/reader035/viewer/2022070914/5fb54fd6bd315d6ec778a890/html5/thumbnails/8.jpg)
8
Recap: Linear Dynamic Models
• Dynamics model State undergoes linear transformation Dt plus Gaussian noise
• Observation model Measurement is linearly transformed state plus Gaussian noise
1~ ,tt t t dN x D x
~ ,tt t t mN y M x
Slide credit: Svetlana Lazebnik, Kristen Grauman
Lecture: Computer Vision 2 (SS 2016) – Beyond Kalman Filters
Prof. Dr. Bastian Leibe, Dr. Jörg Stückler
![Page 9: Computer Vision 2 Lecture 6 · 2 Content of the Lecture •Single-Object Tracking •Bayesian Filtering Kalman Filters, EKF Particle Filters •Multi-Object Tracking •Visual Odometry](https://reader035.vdocuments.mx/reader035/viewer/2022070914/5fb54fd6bd315d6ec778a890/html5/thumbnails/9.jpg)
9
Recap: Constant Velocity (1D Points)
• State vector: position p and velocity v
• Measurement is position only
1
11 )(
tt
ttt
vv
vtpp
t
t
tv
px
noisev
ptnoisexDx
t
t
ttt
1
1
110
1
(greek letters
denote noise
terms)
noisev
pnoiseMxy
t
t
tt
01
Slide credit: Svetlana Lazebnik, Kristen Grauman
Lecture: Computer Vision 2 (SS 2016) – Beyond Kalman Filters
Prof. Dr. Bastian Leibe, Dr. Jörg Stückler
![Page 10: Computer Vision 2 Lecture 6 · 2 Content of the Lecture •Single-Object Tracking •Bayesian Filtering Kalman Filters, EKF Particle Filters •Multi-Object Tracking •Visual Odometry](https://reader035.vdocuments.mx/reader035/viewer/2022070914/5fb54fd6bd315d6ec778a890/html5/thumbnails/10.jpg)
10
Recap: Constant Acceleration (1D Points)
• State vector: position p, velocity v, and acceleration a.
• Measurement is position only
1
11
1
2
21
11
)(
)()(
tt
ttt
tttt
aa
atvv
atvtpp
t
t
t
t
a
v
p
x
noise
a
v
p
t
tt
noisexDx
t
t
t
ttt
1
1
1
2
21
1
100
10
1
(greek letters
denote noise
terms)
noise
a
v
p
noiseMxy
t
t
t
tt
001
Slide credit: Svetlana Lazebnik, Kristen Grauman
Lecture: Computer Vision 2 (SS 2016) – Beyond Kalman Filters
Prof. Dr. Bastian Leibe, Dr. Jörg Stückler
![Page 11: Computer Vision 2 Lecture 6 · 2 Content of the Lecture •Single-Object Tracking •Bayesian Filtering Kalman Filters, EKF Particle Filters •Multi-Object Tracking •Visual Odometry](https://reader035.vdocuments.mx/reader035/viewer/2022070914/5fb54fd6bd315d6ec778a890/html5/thumbnails/11.jpg)
11
Recap: General Motion Models
• Assuming we have differential equations for the motion E.g. for (undampened) periodic motion of a linear spring
• Substitute variables to transform this into linear system
• Then we have
2
2
d pp
dt
1p p2
dpp
dt
2
3 2
d pp
dt
1,
2,
3,
t
t t
t
p
x p
p
001
10
12
21
t
tt
Dt
1,1,3
1,31,2,2
1,3
2
21
1,21,1,1
)(
)()(
tt
ttt
tttt
pp
ptpp
ptptpp
Lecture: Computer Vision 2 (SS 2016) – Beyond Kalman Filters
Prof. Dr. Bastian Leibe, Dr. Jörg Stückler
![Page 12: Computer Vision 2 Lecture 6 · 2 Content of the Lecture •Single-Object Tracking •Bayesian Filtering Kalman Filters, EKF Particle Filters •Multi-Object Tracking •Visual Odometry](https://reader035.vdocuments.mx/reader035/viewer/2022070914/5fb54fd6bd315d6ec778a890/html5/thumbnails/12.jpg)
12
Recap: The Kalman Filter
Know corrected state from
previous time step, and all
measurements up to the
current one
Predict distribution over
next state.
Time advances: t++
Time update
(“Predict”)
Measurement update
(“Correct”)
Receive measurement
10 ,, tt yyXP
tt ,
Mean and std. dev.
of predicted state:
tt yyXP ,,0
tt ,
Mean and std. dev.
of corrected state:
Know prediction of state,
and next measurement
Update distribution over
current state.
Slide credit: Kristen Grauman
Lecture: Computer Vision 2 (SS 2016) – Beyond Kalman Filters
Prof. Dr. Bastian Leibe, Dr. Jörg Stückler
![Page 13: Computer Vision 2 Lecture 6 · 2 Content of the Lecture •Single-Object Tracking •Bayesian Filtering Kalman Filters, EKF Particle Filters •Multi-Object Tracking •Visual Odometry](https://reader035.vdocuments.mx/reader035/viewer/2022070914/5fb54fd6bd315d6ec778a890/html5/thumbnails/13.jpg)
13
Recap: General Kalman Filter (>1dim)
PREDICT CORRECT
1ttt xDx
td
T
tttt DD
1 tttttt xMyKxx
tttt MKI
1 tm
T
ttt
T
ttt MMMK
More weight on residual
when measurement error
covariance approaches 0.
Less weight on residual as a
priori estimate error
covariance approaches 0.
“residual”
for derivations,
see F&P Chapter 17.3
Slide credit: Kristen Grauman
“Kalman gain”
Lecture: Computer Vision 2 (SS 2016) – Beyond Kalman Filters
Prof. Dr. Bastian Leibe, Dr. Jörg Stückler
![Page 14: Computer Vision 2 Lecture 6 · 2 Content of the Lecture •Single-Object Tracking •Bayesian Filtering Kalman Filters, EKF Particle Filters •Multi-Object Tracking •Visual Odometry](https://reader035.vdocuments.mx/reader035/viewer/2022070914/5fb54fd6bd315d6ec778a890/html5/thumbnails/14.jpg)
14
Resources: Kalman Filter Web Site
http://www.cs.unc.edu/~welch/kalman
• Electronic and printed references Book lists and recommendations
Research papers
Links to other sites
Some software
• News
• Java-Based KF Learning Tool On-line 1D simulation
Linear and non-linear
Variable dynamics
Slide adapted from Greg Welch
Lecture: Computer Vision 2 (SS 2016) – Beyond Kalman Filters
Prof. Dr. Bastian Leibe, Dr. Jörg Stückler
![Page 15: Computer Vision 2 Lecture 6 · 2 Content of the Lecture •Single-Object Tracking •Bayesian Filtering Kalman Filters, EKF Particle Filters •Multi-Object Tracking •Visual Odometry](https://reader035.vdocuments.mx/reader035/viewer/2022070914/5fb54fd6bd315d6ec778a890/html5/thumbnails/15.jpg)
15
Remarks
• Try it! Not too hard to understand or program
• Start simple Experiment in 1D
Make your own filter in Matlab, etc.
• Note: the Kalman filter “wants to work” Debugging can be difficult
Errors can go un-noticed
Slide adapted from Greg Welch
Lecture: Computer Vision 2 (SS 2016) – Beyond Kalman Filters
Prof. Dr. Bastian Leibe, Dr. Jörg Stückler
![Page 16: Computer Vision 2 Lecture 6 · 2 Content of the Lecture •Single-Object Tracking •Bayesian Filtering Kalman Filters, EKF Particle Filters •Multi-Object Tracking •Visual Odometry](https://reader035.vdocuments.mx/reader035/viewer/2022070914/5fb54fd6bd315d6ec778a890/html5/thumbnails/16.jpg)
16
Topics of This Lecture
• Recap: Kalman Filter Basic ideas
Limitations
Extensions
• Particle Filters Basic ideas
Propagation of general densities
Factored sampling
• Case study Detector Confidence Particle Filter
Role of the different elements
Lecture: Computer Vision 2 (SS 2016) – Beyond Kalman Filters
Prof. Dr. Bastian Leibe, Dr. Jörg Stückler
![Page 17: Computer Vision 2 Lecture 6 · 2 Content of the Lecture •Single-Object Tracking •Bayesian Filtering Kalman Filters, EKF Particle Filters •Multi-Object Tracking •Visual Odometry](https://reader035.vdocuments.mx/reader035/viewer/2022070914/5fb54fd6bd315d6ec778a890/html5/thumbnails/17.jpg)
17
Extension: Extended Kalman Filter (EKF)
• Basic idea State transition and observation model don’t need to be linear functions
of the state, but just need to be differentiable.
The EKF essentially linearizes the nonlinearity around the current
estimate by a Taylor expansion.
• Properties Unlike the linear KF, the EKF is in general not an optimal estimator.
If the initial estimate is wrong, the filter may quickly diverge.
Still, it’s the de-facto standard in many applications
Including navigation systems and GPS
𝑦𝑡 = ℎ 𝑥𝑡 + 𝛿
𝑥𝑡 = 𝑔 𝑥𝑡−1, 𝑢𝑡 + 휀
Lecture: Computer Vision 2 (SS 2016) – Beyond Kalman Filters
Prof. Dr. Bastian Leibe, Dr. Jörg Stückler
![Page 18: Computer Vision 2 Lecture 6 · 2 Content of the Lecture •Single-Object Tracking •Bayesian Filtering Kalman Filters, EKF Particle Filters •Multi-Object Tracking •Visual Odometry](https://reader035.vdocuments.mx/reader035/viewer/2022070914/5fb54fd6bd315d6ec778a890/html5/thumbnails/18.jpg)
18
Recap: Kalman Filter – Detailed Algorithm
• Algorithm summary Assumption: linear model
Prediction step
Correction step
Lecture: Computer Vision 2 (SS 2016) – Beyond Kalman Filters
Prof. Dr. Bastian Leibe, Dr. Jörg Stückler
![Page 19: Computer Vision 2 Lecture 6 · 2 Content of the Lecture •Single-Object Tracking •Bayesian Filtering Kalman Filters, EKF Particle Filters •Multi-Object Tracking •Visual Odometry](https://reader035.vdocuments.mx/reader035/viewer/2022070914/5fb54fd6bd315d6ec778a890/html5/thumbnails/19.jpg)
19
Extended Kalman Filter (EKF)
• Algorithm summary Nonlinear model
Prediction step
Correction step
with the Jacobians
Lecture: Computer Vision 2 (SS 2016) – Beyond Kalman Filters
Prof. Dr. Bastian Leibe, Dr. Jörg Stückler
![Page 20: Computer Vision 2 Lecture 6 · 2 Content of the Lecture •Single-Object Tracking •Bayesian Filtering Kalman Filters, EKF Particle Filters •Multi-Object Tracking •Visual Odometry](https://reader035.vdocuments.mx/reader035/viewer/2022070914/5fb54fd6bd315d6ec778a890/html5/thumbnails/20.jpg)
20
Kalman Filter – Other Extensions
• Unscented Kalman Filter (UKF) Used for models with highly nonlinear predict and update functions.
Here, the EKF can give very poor performance, since the covariance is
propagated through linearization of the non-linear model.
Idea (UKF): Propagate just a few sample points (“sigma points”) around
the mean exactly, then recover the covariance from them.
More accurate results than the EKF’s Taylor expansion approximation.
• Ensemble Kalman Filter (EnKF) Represents the distribution of the system state using a collection (an
ensemble) of state vectors.
Replace covariance matrix by sample covariance from ensemble.
Still basic assumption that all prob. distributions involved are Gaussian.
EnKFs are especially suitable for problems with a large number of
variables.
Lecture: Computer Vision 2 (SS 2016) – Beyond Kalman Filters
Prof. Dr. Bastian Leibe, Dr. Jörg Stückler
![Page 21: Computer Vision 2 Lecture 6 · 2 Content of the Lecture •Single-Object Tracking •Bayesian Filtering Kalman Filters, EKF Particle Filters •Multi-Object Tracking •Visual Odometry](https://reader035.vdocuments.mx/reader035/viewer/2022070914/5fb54fd6bd315d6ec778a890/html5/thumbnails/21.jpg)
21
Even More Extensions
• Switching Linear Dynamic System (SLDS)
Use a set of k dynamic models A(1),...,A(k), each of which describes a
different dynamic behavior.
Hidden variable zt determines which model is active at time t.
A switching process can change zt according to distribution .
Figure source: Erik Sudderth
Lecture: Computer Vision 2 (SS 2016) – Beyond Kalman Filters
Prof. Dr. Bastian Leibe, Dr. Jörg Stückler
![Page 22: Computer Vision 2 Lecture 6 · 2 Content of the Lecture •Single-Object Tracking •Bayesian Filtering Kalman Filters, EKF Particle Filters •Multi-Object Tracking •Visual Odometry](https://reader035.vdocuments.mx/reader035/viewer/2022070914/5fb54fd6bd315d6ec778a890/html5/thumbnails/22.jpg)
22
Topics of This Lecture
• Recap: Kalman Filter Basic ideas
Limitations
Extensions
• Particle Filters Basic ideas
Propagation of general densities
Factored sampling
• Case study Detector Confidence Particle Filter
Role of the different elements
Today: only main ideas
Formal introduction
next lecture
Lecture: Computer Vision 2 (SS 2016) – Beyond Kalman Filters
Prof. Dr. Bastian Leibe, Dr. Jörg Stückler
![Page 23: Computer Vision 2 Lecture 6 · 2 Content of the Lecture •Single-Object Tracking •Bayesian Filtering Kalman Filters, EKF Particle Filters •Multi-Object Tracking •Visual Odometry](https://reader035.vdocuments.mx/reader035/viewer/2022070914/5fb54fd6bd315d6ec778a890/html5/thumbnails/23.jpg)
23
When Is A Single Hypothesis Too Limiting?
• Consider this example:
say we are tracking the
face on the right using a
skin color blob to get our
measurement.
Update Initial position
x
y
x
y
Prediction
x
y
Measurement
x
y
Video from Jojic & Frey
Figure from Thrun & Kosecka Slide credit: Kristen Grauman
Lecture: Computer Vision 2 (SS 2016) – Beyond Kalman Filters
Prof. Dr. Bastian Leibe, Dr. Jörg Stückler
![Page 24: Computer Vision 2 Lecture 6 · 2 Content of the Lecture •Single-Object Tracking •Bayesian Filtering Kalman Filters, EKF Particle Filters •Multi-Object Tracking •Visual Odometry](https://reader035.vdocuments.mx/reader035/viewer/2022070914/5fb54fd6bd315d6ec778a890/html5/thumbnails/24.jpg)
24
Propagation of General Densities
Figure from Isard & Blake
Lecture: Computer Vision 2 (SS 2016) – Beyond Kalman Filters
Prof. Dr. Bastian Leibe, Dr. Jörg Stückler
![Page 25: Computer Vision 2 Lecture 6 · 2 Content of the Lecture •Single-Object Tracking •Bayesian Filtering Kalman Filters, EKF Particle Filters •Multi-Object Tracking •Visual Odometry](https://reader035.vdocuments.mx/reader035/viewer/2022070914/5fb54fd6bd315d6ec778a890/html5/thumbnails/25.jpg)
25
Factored Sampling
• Idea: Represent state distribution non-parametrically
Prediction: Sample points from prior density for the state, P(X)
Correction: Weight the samples according to P(Y |X)
ttttt
tttttt
dXyyXPXyP
yyXPXyPyyXP
10
100
,,||
,,||,,|
Slide credit: Svetlana Lazebnik
Lecture: Computer Vision 2 (SS 2016) – Beyond Kalman Filters
Prof. Dr. Bastian Leibe, Dr. Jörg Stückler
![Page 26: Computer Vision 2 Lecture 6 · 2 Content of the Lecture •Single-Object Tracking •Bayesian Filtering Kalman Filters, EKF Particle Filters •Multi-Object Tracking •Visual Odometry](https://reader035.vdocuments.mx/reader035/viewer/2022070914/5fb54fd6bd315d6ec778a890/html5/thumbnails/26.jpg)
26
Particle Filtering
• (Also known as Sequential Monte Carlo Methods)
• Idea We want to use sampling to propagate densities over time
(i.e., across frames in a video sequence).
At each time step, represent posterior P(Xt|Yt) with weighted sample
set.
Previous time step’s sample set P(Xt-1|Yt-1) is passed to next time step
as the effective prior.
Lecture: Computer Vision 2 (SS 2016) – Beyond Kalman Filters
Prof. Dr. Bastian Leibe, Dr. Jörg Stückler
![Page 27: Computer Vision 2 Lecture 6 · 2 Content of the Lecture •Single-Object Tracking •Bayesian Filtering Kalman Filters, EKF Particle Filters •Multi-Object Tracking •Visual Odometry](https://reader035.vdocuments.mx/reader035/viewer/2022070914/5fb54fd6bd315d6ec778a890/html5/thumbnails/27.jpg)
27
Particle Filtering
• Many variations, one general concept: Represent the posterior pdf by a set of randomly chosen weighted
samples (particles)
Randomly Chosen = Monte Carlo (MC)
As the number of samples become very large – the characterization
becomes an equivalent representation of the true pdf.
Lecture: Computer Vision 2 (SS 2016) – Beyond Kalman Filters
Prof. Dr. Bastian Leibe, Dr. Jörg Stückler
Sample space
Posterior
Slide adapted from Michael Rubinstein
![Page 28: Computer Vision 2 Lecture 6 · 2 Content of the Lecture •Single-Object Tracking •Bayesian Filtering Kalman Filters, EKF Particle Filters •Multi-Object Tracking •Visual Odometry](https://reader035.vdocuments.mx/reader035/viewer/2022070914/5fb54fd6bd315d6ec778a890/html5/thumbnails/28.jpg)
28
Particle Filtering
Lecture: Computer Vision 2 (SS 2016) – Beyond Kalman Filters
Prof. Dr. Bastian Leibe, Dr. Jörg Stückler
Start with weighted
samples from previous
time step
Sample and shift
according to dynamics
model
Spread due to
randomness; this is pre-
dicted density P(Xt|Yt-1)
Weight the samples
according to observation
density
Arrive at corrected density
estimate P(Xt|Yt)
M. Isard and A. Blake, CONDENSATION -- conditional density propagation for
visual tracking, IJCV 29(1):5-28, 1998
Slide credit: Svetlana Lazebnik
![Page 29: Computer Vision 2 Lecture 6 · 2 Content of the Lecture •Single-Object Tracking •Bayesian Filtering Kalman Filters, EKF Particle Filters •Multi-Object Tracking •Visual Odometry](https://reader035.vdocuments.mx/reader035/viewer/2022070914/5fb54fd6bd315d6ec778a890/html5/thumbnails/29.jpg)
29
Particle Filtering – Visualization
Lecture: Computer Vision 2 (SS 2016) – Beyond Kalman Filters
Prof. Dr. Bastian Leibe, Dr. Jörg Stückler
Code and video available from
http://www.robots.ox.ac.uk/~misard/condensation.html
![Page 30: Computer Vision 2 Lecture 6 · 2 Content of the Lecture •Single-Object Tracking •Bayesian Filtering Kalman Filters, EKF Particle Filters •Multi-Object Tracking •Visual Odometry](https://reader035.vdocuments.mx/reader035/viewer/2022070914/5fb54fd6bd315d6ec778a890/html5/thumbnails/30.jpg)
30
Particle Filtering Results
Lecture: Computer Vision 2 (SS 2016) – Beyond Kalman Filters
Prof. Dr. Bastian Leibe, Dr. Jörg Stückler
http://www.robots.ox.ac.uk/~misard/condensation.html
![Page 31: Computer Vision 2 Lecture 6 · 2 Content of the Lecture •Single-Object Tracking •Bayesian Filtering Kalman Filters, EKF Particle Filters •Multi-Object Tracking •Visual Odometry](https://reader035.vdocuments.mx/reader035/viewer/2022070914/5fb54fd6bd315d6ec778a890/html5/thumbnails/31.jpg)
31
Particle Filtering Results
• Some more examples
Lecture: Computer Vision 2 (SS 2016) – Beyond Kalman Filters
Prof. Dr. Bastian Leibe, Dr. Jörg Stückler
http://www.robots.ox.ac.uk/~misard/condensation.html
Videos from Isard & Blake
![Page 32: Computer Vision 2 Lecture 6 · 2 Content of the Lecture •Single-Object Tracking •Bayesian Filtering Kalman Filters, EKF Particle Filters •Multi-Object Tracking •Visual Odometry](https://reader035.vdocuments.mx/reader035/viewer/2022070914/5fb54fd6bd315d6ec778a890/html5/thumbnails/32.jpg)
32
Obtaining a State Estimate
• Note that there’s no explicit state estimate maintained,
just a “cloud” of particles
• Can obtain an estimate at a particular time by querying the current
particle set
• Some approaches
“Mean” particle
Weighted sum of particles
Confidence: inverse variance
Really want a mode finder—mean of tallest peak
Lecture: Computer Vision 2 (SS 2016) – Beyond Kalman Filters
Prof. Dr. Bastian Leibe, Dr. Jörg Stückler
![Page 33: Computer Vision 2 Lecture 6 · 2 Content of the Lecture •Single-Object Tracking •Bayesian Filtering Kalman Filters, EKF Particle Filters •Multi-Object Tracking •Visual Odometry](https://reader035.vdocuments.mx/reader035/viewer/2022070914/5fb54fd6bd315d6ec778a890/html5/thumbnails/33.jpg)
33
Condensation: Estimating Target State
Lecture: Computer Vision 2 (SS 2016) – Beyond Kalman Filters
Prof. Dr. Bastian Leibe, Dr. Jörg Stückler
From Isard & Blake, 1998
State samples
(thickness proportional to weight)
Mean of weighted
state samples
Figures from Isard & Blake Slide credit: Marc Pollefeys
![Page 34: Computer Vision 2 Lecture 6 · 2 Content of the Lecture •Single-Object Tracking •Bayesian Filtering Kalman Filters, EKF Particle Filters •Multi-Object Tracking •Visual Odometry](https://reader035.vdocuments.mx/reader035/viewer/2022070914/5fb54fd6bd315d6ec778a890/html5/thumbnails/34.jpg)
34
Summary: Particle Filtering
• Pros: Able to represent arbitrary densities
Converging to true posterior even for non-Gaussian and nonlinear
system
Efficient: particles tend to focus on regions with high probability
Works with many different state spaces
E.g. articulated tracking in complicated joint angle spaces
Many extensions available
Lecture: Computer Vision 2 (SS 2016) – Beyond Kalman Filters
Prof. Dr. Bastian Leibe, Dr. Jörg Stückler
![Page 35: Computer Vision 2 Lecture 6 · 2 Content of the Lecture •Single-Object Tracking •Bayesian Filtering Kalman Filters, EKF Particle Filters •Multi-Object Tracking •Visual Odometry](https://reader035.vdocuments.mx/reader035/viewer/2022070914/5fb54fd6bd315d6ec778a890/html5/thumbnails/35.jpg)
35
Summary: Particle Filtering
• Cons / Caveats: #Particles is important performance factor
Want as few particles as possible for efficiency.
But need to cover state space sufficiently well.
Worst-case complexity grows exponentially in the dimensions
Multimodal densities possible, but still single object
Interactions between multiple objects require special treatment.
Not handled well in the particle filtering framework
(state space explosion).
Lecture: Computer Vision 2 (SS 2016) – Beyond Kalman Filters
Prof. Dr. Bastian Leibe, Dr. Jörg Stückler
![Page 36: Computer Vision 2 Lecture 6 · 2 Content of the Lecture •Single-Object Tracking •Bayesian Filtering Kalman Filters, EKF Particle Filters •Multi-Object Tracking •Visual Odometry](https://reader035.vdocuments.mx/reader035/viewer/2022070914/5fb54fd6bd315d6ec778a890/html5/thumbnails/36.jpg)
36
Topics of This Lecture
• Recap: Kalman Filter Basic ideas
Limitations
Extensions
• Particle Filters Basic ideas
Propagation of general densities
Factored sampling
• Case study Detector Confidence Particle Filter
Role of the different elements
Lecture: Computer Vision 2 (SS 2016) – Beyond Kalman Filters
Prof. Dr. Bastian Leibe, Dr. Jörg Stückler
![Page 37: Computer Vision 2 Lecture 6 · 2 Content of the Lecture •Single-Object Tracking •Bayesian Filtering Kalman Filters, EKF Particle Filters •Multi-Object Tracking •Visual Odometry](https://reader035.vdocuments.mx/reader035/viewer/2022070914/5fb54fd6bd315d6ec778a890/html5/thumbnails/37.jpg)
37
Challenge: Unreliable Object Detectors
• Example: Low-res webcam footage (320240), MPEG compressed
Lecture: Computer Vision 2 (SS 2016) – Beyond Kalman Filters
Prof. Dr. Bastian Leibe, Dr. Jörg Stückler
Detector input Tracker output
How to get from here… …to here? ?
![Page 38: Computer Vision 2 Lecture 6 · 2 Content of the Lecture •Single-Object Tracking •Bayesian Filtering Kalman Filters, EKF Particle Filters •Multi-Object Tracking •Visual Odometry](https://reader035.vdocuments.mx/reader035/viewer/2022070914/5fb54fd6bd315d6ec778a890/html5/thumbnails/38.jpg)
38
Tracking based on Detector Confidence
• Detector output is often not perfect Missing detections and false positives
But continuous confidence still contains useful cues.
• Idea pursued here: Use continuous detector confidence to track persons over time.
Lecture: Computer Vision 2 (SS 2016) – Beyond Kalman Filters
Prof. Dr. Bastian Leibe, Dr. Jörg Stückler
(using ISM detector) (using HOG detector)
![Page 39: Computer Vision 2 Lecture 6 · 2 Content of the Lecture •Single-Object Tracking •Bayesian Filtering Kalman Filters, EKF Particle Filters •Multi-Object Tracking •Visual Odometry](https://reader035.vdocuments.mx/reader035/viewer/2022070914/5fb54fd6bd315d6ec778a890/html5/thumbnails/39.jpg)
39
Main Ideas
• Detector confidence particle filter Initialize particle cloud on
strong object detections.
Propagate particles using
continuous detector confidence
as observation model.
• Disambiguate between
different persons Train a person-specific classifier
with online boosting.
Use classifier output to distinguish
between nearby persons.
Lecture: Computer Vision 2 (SS 2016) – Beyond Kalman Filters
Prof. Dr. Bastian Leibe, Dr. Jörg Stückler
t
[Breitenstein, Reichlin, Leibe et al., ICCV’09]
![Page 40: Computer Vision 2 Lecture 6 · 2 Content of the Lecture •Single-Object Tracking •Bayesian Filtering Kalman Filters, EKF Particle Filters •Multi-Object Tracking •Visual Odometry](https://reader035.vdocuments.mx/reader035/viewer/2022070914/5fb54fd6bd315d6ec778a890/html5/thumbnails/40.jpg)
40
Detector Confidence Particle Filter
• State:
• Motion model (constant velocity)
• Observation model
Lecture: Computer Vision 2 (SS 2016) – Beyond Kalman Filters
Prof. Dr. Bastian Leibe, Dr. Jörg Stückler
Discrete
detections
Detector
confidence
Classifier
confidence
t
![Page 41: Computer Vision 2 Lecture 6 · 2 Content of the Lecture •Single-Object Tracking •Bayesian Filtering Kalman Filters, EKF Particle Filters •Multi-Object Tracking •Visual Odometry](https://reader035.vdocuments.mx/reader035/viewer/2022070914/5fb54fd6bd315d6ec778a890/html5/thumbnails/41.jpg)
41
When Is Which Term Useful?
Lecture: Computer Vision 2 (SS 2016) – Beyond Kalman Filters
Prof. Dr. Bastian Leibe, Dr. Jörg Stückler
Discrete detections Detector confidence Classifier confidence
![Page 42: Computer Vision 2 Lecture 6 · 2 Content of the Lecture •Single-Object Tracking •Bayesian Filtering Kalman Filters, EKF Particle Filters •Multi-Object Tracking •Visual Odometry](https://reader035.vdocuments.mx/reader035/viewer/2022070914/5fb54fd6bd315d6ec778a890/html5/thumbnails/42.jpg)
42
Each Observation Term Increases Robustness!
Lecture: Computer Vision 2 (SS 2016) – Beyond Kalman Filters
Prof. Dr. Bastian Leibe, Dr. Jörg Stückler
CLEAR MOT scores
Detector only
![Page 43: Computer Vision 2 Lecture 6 · 2 Content of the Lecture •Single-Object Tracking •Bayesian Filtering Kalman Filters, EKF Particle Filters •Multi-Object Tracking •Visual Odometry](https://reader035.vdocuments.mx/reader035/viewer/2022070914/5fb54fd6bd315d6ec778a890/html5/thumbnails/43.jpg)
43
Each Observation Term Increases Robustness!
Lecture: Computer Vision 2 (SS 2016) – Beyond Kalman Filters
Prof. Dr. Bastian Leibe, Dr. Jörg Stückler
Detector
+ Confidence
CLEAR MOT scores
![Page 44: Computer Vision 2 Lecture 6 · 2 Content of the Lecture •Single-Object Tracking •Bayesian Filtering Kalman Filters, EKF Particle Filters •Multi-Object Tracking •Visual Odometry](https://reader035.vdocuments.mx/reader035/viewer/2022070914/5fb54fd6bd315d6ec778a890/html5/thumbnails/44.jpg)
44
Each Observation Term Increases Robustness!
Lecture: Computer Vision 2 (SS 2016) – Beyond Kalman Filters
Prof. Dr. Bastian Leibe, Dr. Jörg Stückler
Detector
+ Classifier
CLEAR MOT scores
![Page 45: Computer Vision 2 Lecture 6 · 2 Content of the Lecture •Single-Object Tracking •Bayesian Filtering Kalman Filters, EKF Particle Filters •Multi-Object Tracking •Visual Odometry](https://reader035.vdocuments.mx/reader035/viewer/2022070914/5fb54fd6bd315d6ec778a890/html5/thumbnails/45.jpg)
45
Each Observation Term Increases Robustness!
Lecture: Computer Vision 2 (SS 2016) – Beyond Kalman Filters
Prof. Dr. Bastian Leibe, Dr. Jörg Stückler
Detector
+ Confidence
+ Classifier False negatives,
false positives,
and ID switches
decrease!
CLEAR MOT scores
![Page 46: Computer Vision 2 Lecture 6 · 2 Content of the Lecture •Single-Object Tracking •Bayesian Filtering Kalman Filters, EKF Particle Filters •Multi-Object Tracking •Visual Odometry](https://reader035.vdocuments.mx/reader035/viewer/2022070914/5fb54fd6bd315d6ec778a890/html5/thumbnails/46.jpg)
46
Qualitative Results
Lecture: Computer Vision 2 (SS 2016) – Beyond Kalman Filters
Prof. Dr. Bastian Leibe, Dr. Jörg Stückler
![Page 47: Computer Vision 2 Lecture 6 · 2 Content of the Lecture •Single-Object Tracking •Bayesian Filtering Kalman Filters, EKF Particle Filters •Multi-Object Tracking •Visual Odometry](https://reader035.vdocuments.mx/reader035/viewer/2022070914/5fb54fd6bd315d6ec778a890/html5/thumbnails/47.jpg)
47
Remaining Issues
• Some false positive initializations at wrong scales… Due to limited scale range of the person detector.
Due to boundary effects of the person detector.
Lecture: Computer Vision 2 (SS 2016) – Beyond Kalman Filters
Prof. Dr. Bastian Leibe, Dr. Jörg Stückler
![Page 48: Computer Vision 2 Lecture 6 · 2 Content of the Lecture •Single-Object Tracking •Bayesian Filtering Kalman Filters, EKF Particle Filters •Multi-Object Tracking •Visual Odometry](https://reader035.vdocuments.mx/reader035/viewer/2022070914/5fb54fd6bd315d6ec778a890/html5/thumbnails/48.jpg)
48
References and Further Reading
• A good tutorial on Particle Filters M.S. Arulampalam, S. Maskell, N. Gordon, T. Clapp. A Tutorial
on Particle Filters for Online Nonlinear/Non-Gaussian Bayesian
Tracking. In IEEE Transactions on Signal Processing, Vol. 50(2), pp.
174-188, 2002.
• The CONDENSATION paper M. Isard and A. Blake, CONDENSATION - conditional density
propagation for visual tracking, IJCV 29(1):5-28, 1998
Lecture: Computer Vision 2 (SS 2016) – Beyond Kalman Filters
Prof. Dr. Bastian Leibe, Dr. Jörg Stückler