tracking - cvg @ ethz · computer vision tracking loop search region actual object position from...
TRANSCRIPT
![Page 1: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/1.jpg)
ComputerVision
TrackingTracking
Many thanks to: H. Bischof, B. Leibe, V. Ferrari, K. Graumann, Y. Ukrainitz, D. Wagner, V Lepetit, M. Breitenstein, P. Sabzmeydani, Z. Kalal from whom I borrowed many slides and videos.
![Page 2: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/2.jpg)
ComputerVision
CV
PR
’06]
We all know what tracking is, right?
[Gra
bner
et al.,
Vid
eoP
roc
CV
PR
’06]
![Page 3: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/3.jpg)
ComputerVision Tracking
actual object position
Time t+1Time t
„ FIND IT AGAIN“
![Page 4: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/4.jpg)
ComputerVision What to track?
![Page 5: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/5.jpg)
ComputerVision What to track?
centerpoint
![Page 6: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/6.jpg)
ComputerVision What to track?
multiplepoints
![Page 7: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/7.jpg)
ComputerVision What to track?
(body)parts
![Page 8: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/8.jpg)
ComputerVision What to track?
region
![Page 9: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/9.jpg)
ComputerVision What to track?
outline
![Page 10: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/10.jpg)
ComputerVision What to track?
structure
![Page 11: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/11.jpg)
ComputerVision Approaches
(i) Model-based trackingapplication-specific
human body, faces, space shuttle,…
![Page 12: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/12.jpg)
ComputerVision Approaches
(i) Model-based trackingapplication-specific
human body, faces, space shuttle,…
(ii) Feature trackingmore genericcorner tracking
blob/contour trackingintensity profile tracking
region tracking
![Page 13: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/13.jpg)
ComputerVision
Saliency
Object
Tracking Cues
Model/ Tracking History
Scene
![Page 14: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/14.jpg)
ComputerVision Applications!
• Structure-from-Motion
• Gesture/Action Recognition
• Video editing
• Augmented Reality• Augmented Reality
• Navigation
• ….
![Page 15: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/15.jpg)
ComputerVision Applications: Game Interface
![Page 16: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/16.jpg)
ComputerVision Applications: SfM
• Tracked Points gives correspondences
![Page 17: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/17.jpg)
ComputerVision Applications: SfM
et
al.
IJC
V 2
00
4]
[Po
llefe
yes
et
al.
IJC
V 2
00
4]
![Page 18: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/18.jpg)
ComputerVision
Applications: Analysis of Motion Pattern
Single-Agent Level
Multi-Agent Level Scene Level
Detail Level
![Page 19: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/19.jpg)
ComputerVision
[Ess
et
al.
CV
PR
’08
]
![Page 20: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/20.jpg)
ComputerVision Outline
• Point Tracking• Template Tracking• Region Tracking• Model-based Tracking• Foreground vs. Background• Foreground vs. Background• Tracking-by-Detection
– Object classes– specific object
• Combining Tracking and Detection• Context in Tracking
![Page 21: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/21.jpg)
ComputerVision
x
y
Vision = Inverse Graphics
x
y
z
![Page 22: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/22.jpg)
ComputerVision Motion
![Page 23: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/23.jpg)
ComputerVision Motion
![Page 24: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/24.jpg)
ComputerVision Motion
![Page 25: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/25.jpg)
ComputerVision
x
y
Vision = Inverse Graphics
x
y
z
Tracking = Inverse Animation
![Page 26: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/26.jpg)
ComputerVision Steps of Tracking
predictpredict correctcorrect
• Recap: Particle filtering– Tracking can be seen as the process of
propagating the posterior distribution of state given measurements across time.
![Page 27: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/27.jpg)
ComputerVision )|,( 111 −−− ttt zppp &
)|,( 1−ttt zppp &
prediction CONDEN
Particle Filter
)|( tt pzpweighing with
)|,( ttt zppp &
update
NSATION
![Page 28: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/28.jpg)
ComputerVision General Tracking Loop
predict to t+1
time t
measure at t+1
update locationupdate model
![Page 29: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/29.jpg)
ComputerVision
Point TrackingPoint Tracking
![Page 30: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/30.jpg)
ComputerVision Estimate Optimal
Transformation
![Page 31: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/31.jpg)
ComputerVision Simple 1D Problem
I0(x)
I (x+h)I1(x+h)
![Page 32: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/32.jpg)
ComputerVision Sum of Squared Differences
I0(x)
I (x+h)
h
I1(x+h)
E(h) = [I0(x) – I1(x+h) ]2
![Page 33: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/33.jpg)
ComputerVision Calculation of Displacement
E(h) [ I0 (x) – I1(x) – hI1’(x) ]2≈
E(h) = [I0(x) – I1(x+h) ]2
-2[I1’(x)(I0(x) – I1(x) ) – hI1’(x)2] h
E
∂∂ ≈
I0(x) – I1(x)h I1’(x)≈
![Page 34: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/34.jpg)
ComputerVision Interpretation
h I0(x)
I (x+h)
I0(x) – I1(x)
I1’(x)I1(x+h)
I0(x) – I1(x)h I1’(x)≈
I1’(x)
![Page 35: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/35.jpg)
ComputerVision Problem A: Local Minima
(a) (b)
![Page 36: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/36.jpg)
ComputerVision Problem A: Local Minima
![Page 37: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/37.jpg)
ComputerVision Problem B: Zero Gradient
I(x) - I0(x)h ≈
I0’(x)
?
![Page 38: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/38.jpg)
ComputerVision Problem B: Aperture problem
No gradient along one direction:
![Page 39: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/39.jpg)
ComputerVision Problem B: Aperture problem
No gradient along one direction:
![Page 40: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/40.jpg)
ComputerVision “Solving” the Aperture Problem
• How to get more equations for a pixel?
• Spatial coherence constraint: Pixel’s neighbors have the same movementneighbors have the same movement
I0(x)
I1(x+h)
![Page 41: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/41.jpg)
ComputerVision
, I
I x ∂∂= ,
∂∂=y
II y
t
II t ∂
∂=
Recall: Optical Flow
0 =++ tyx IvIuI
1 equation in 2 unknowns
, =dt
dxu
dt
dyv =
, x
I x ∂= ,
∂=
yI y
tI t ∂
=
![Page 42: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/42.jpg)
ComputerVision Least Squares Problem
Pseudo Inverse
Over determined System of Equations
![Page 43: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/43.jpg)
ComputerVision Eigenvectors of ATA
• Haven’t we seen an equation like this • Haven’t we seen an equation like this before?
• Recall the Harris corner detector!
• “Good Features to Track”
![Page 44: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/44.jpg)
ComputerVision Interpreting the Eigenvalues
λ2
“Corner”λλλλ1 ~ λλλλ2 and large
“Edge” λλλλ2 >> λλλλ1
λ1
“Edge” λλλλ1 >> λλλλ2
“Flat” region
![Page 45: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/45.jpg)
ComputerVision
Samples: Edge / Low Texture / High Texture
![Page 46: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/46.jpg)
ComputerVision Example
![Page 47: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/47.jpg)
ComputerVision
Template TrackingTemplate Tracking
![Page 48: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/48.jpg)
ComputerVision
Lucas-KanadeTemplate Tracker
• From Points to templates
• Estimate „optimal“ warp W
![Page 49: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/49.jpg)
ComputerVision
Ba
ker
& M
att
he
ws,
IJC
V’0
4,
Lu
cas
-Ka
na
de
20
Ye
ars
On
: A
Un
ifyin
g F
ram
ew
ork
][
Ba
ker
& M
att
he
ws,
IJC
V’0
4,
Lu
cas
20
Ye
ars
On
: A
Un
ifyin
g F
ram
ew
ork
![Page 50: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/50.jpg)
ComputerVision Example
![Page 51: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/51.jpg)
ComputerVision
Tracking by Detection of Local
Image FeaturesImage Features(specific target)
![Page 52: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/52.jpg)
ComputerVision
3D Object Detection
Reference image(s) of the object to detect
Test image
![Page 53: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/53.jpg)
ComputerVision Standard Approach
• Step 1: Keypoint detection– invariant to scale, rotation, or perspective
![Page 54: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/54.jpg)
ComputerVision Standard Approach
• Step 2: Patch rectification
![Page 55: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/55.jpg)
ComputerVision Standard Approach
• Step 3: Build descriptor vector
![Page 56: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/56.jpg)
ComputerVision Standard Approach
• Step 4: Match descriptor vectors
Query
Database
![Page 57: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/57.jpg)
ComputerVision Summary
Search in the Database
Search in the Database
Keypoint Detection
Keypoint Recognition
DatabaseDatabase
Pre-processingMake the actual classification easier
Robust 3D Pose Calculation
(RANSAC)
Robust 3D Pose Calculation
(RANSAC)
Geometric verification
![Page 58: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/58.jpg)
ComputerVision
[Wa
gn
er
et
al.
ISM
AR
’08
]
![Page 59: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/59.jpg)
ComputerVision
[Wa
gn
er
et
al.
‘09
]
![Page 60: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/60.jpg)
ComputerVision
Region TrackingRegion Tracking
![Page 61: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/61.jpg)
ComputerVision Background Modeling
Input Background Model
Moving Foreground
Blobs (Objects)
-
![Page 62: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/62.jpg)
ComputerVision Mean Shift Tracking
• The mean shift tracker tracks a region, with a prescribed (color) distribution
• The similarity between the tracked region and the target region is
an
d M
ee
r, I
CC
V’9
9] region and the target region is
maximized, through evolution towards higher density in a parameter space
• Typically this search only takes a few iterations
[Co
ma
nic
iua
nd
Me
er,
IC
CV
’99
]
![Page 63: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/63.jpg)
ComputerVision Meanshift Tracking
Region ofinterest (Kernel) Center of
mass
Mean Shiftvector Measurements
![Page 64: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/64.jpg)
ComputerVision Intuitive Description
![Page 65: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/65.jpg)
ComputerVision Intuitive Description
![Page 66: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/66.jpg)
ComputerVision Intuitive Description
![Page 67: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/67.jpg)
ComputerVision Intuitive Description
![Page 68: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/68.jpg)
ComputerVision Intuitive Description
![Page 69: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/69.jpg)
ComputerVision Intuitive Description
![Page 70: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/70.jpg)
ComputerVision Intuitive Desciption
![Page 71: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/71.jpg)
ComputerVision Example
![Page 72: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/72.jpg)
ComputerVision Elderly People Monitoring
![Page 73: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/73.jpg)
ComputerVision
Model based Model based Tracking
![Page 74: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/74.jpg)
ComputerVision
Articulated Tracking with Part-Based Model
• part appearance + relative geometry.
![Page 75: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/75.jpg)
ComputerVision Using Models
• Goal– Recover a person’s body articulation
– Detailed parameterization in terms of joint locations or joint angles
• Two basic classes of approaches– Articulated tracking as high-
dimensional inference
– Part-based models
![Page 76: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/76.jpg)
ComputerVision
[Ra
ma
na
ne
t a
l. C
VP
R’0
5]
![Page 77: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/77.jpg)
ComputerVision
Tracking as On-line Foreground vs.
Background Background Classification
![Page 78: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/78.jpg)
ComputerVision Tracking as Classification
• Learning current object appearance vs. local background.
currentbackgroundbackground
[Gra
bn
er
et
al.
CV
PR
’06
]
currentobject
appearance
![Page 79: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/79.jpg)
ComputerVision Tracking as Classification
object
backgroundvs.
![Page 80: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/80.jpg)
ComputerVision Tracking as Classification
object
backgroundvs.
![Page 81: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/81.jpg)
ComputerVision Tracking Loop
search Region
actual object position
from time t to t+1 evaluate classifier on sub-patches
-
+
- -
-
create confidence mapanalyze map and set new
object position update classifier (tracker)
![Page 82: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/82.jpg)
ComputerVision
![Page 83: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/83.jpg)
ComputerVision “Simple tracking”
![Page 84: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/84.jpg)
ComputerVision “Tracking the Invisible”
![Page 85: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/85.jpg)
ComputerVision When does it fail…
![Page 86: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/86.jpg)
ComputerVision
search Region
actual object position
from time t to t+1
evaluate classifier on sub-patches
-
+
- -
-
create confidence mapanalyze map andset new object
position
update classifier(tracker)
Self-learning!
![Page 87: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/87.jpg)
ComputerVision Drifting
Tracked Patches Confidence
![Page 88: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/88.jpg)
ComputerVision Drifting
![Page 89: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/89.jpg)
ComputerVision
Tracking by DetectionDetection(object class)
![Page 90: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/90.jpg)
ComputerVision Traditional Tracking
t=1
initialization
t=2position in prev. frame
candidate new positions(e.g., dynamics)
best new position(e.g., max color similarity)
![Page 91: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/91.jpg)
ComputerVision Tracking-by-Detection
…
detect object(s) independently in each frame
associate detections over time into tracks
![Page 92: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/92.jpg)
ComputerVision Multiple Objects
Frame 5Frame 1 Frame 9
![Page 93: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/93.jpg)
ComputerVision
Example: Multiple Object Tracking
![Page 94: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/94.jpg)
ComputerVision How to get the detections?
Persons Background
Supervised Learning
![Page 95: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/95.jpg)
ComputerVision Using the classifier
![Page 96: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/96.jpg)
ComputerVision How to link them?
• Space-Time Analysis:(a) collect detections
Detections
Space Time Volume
[Le
ibe
et
al.
CV
PR
’07
]
![Page 97: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/97.jpg)
ComputerVision Trajectory Estimation
(a) collect detections(b) trajectory growing and selection
t
x
t
z
Space Time Volume
![Page 98: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/98.jpg)
ComputerVision Trajectory Estimation
(a) collect detections(b) trajectory growing and selection
t
x
t
z
H1H2
Space Time Volume
![Page 99: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/99.jpg)
ComputerVision Result
Input (Object Detections) “Tracking” Result
![Page 100: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/100.jpg)
ComputerVision
![Page 101: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/101.jpg)
ComputerVision More information helps…
• Articulated tracking– “walking” people
• 3D Information
Ground Plane Depth verification
![Page 102: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/102.jpg)
ComputerVision
[Gammeter et al. ECCV’08]
![Page 103: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/103.jpg)
ComputerVision Towards Scene Interpretation
![Page 104: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/104.jpg)
ComputerVision
Combining Tracking and Tracking and
Detection
![Page 105: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/105.jpg)
ComputerVision Refining an object model
• Limit drift
Current Model
Fix (initial) Model
[Gra
bn
er
et
al.
EC
CV
’08
]
![Page 106: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/106.jpg)
ComputerVision Recover from Drift
![Page 107: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/107.jpg)
ComputerVision Drifting
CLICK HERE TO START
![Page 108: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/108.jpg)
ComputerVision Combination: KLT & TbD
• Use a KLT Tracker to explore
• Learn an object detector on the fly.
[Ka
lal
et
al.
CV
PR
’10
]
![Page 109: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/109.jpg)
ComputerVision
![Page 110: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/110.jpg)
ComputerVision
![Page 111: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/111.jpg)
ComputerVision
![Page 112: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/112.jpg)
ComputerVision
Contextin Trackingin Tracking
![Page 113: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/113.jpg)
ComputerVision I’m Carl – Track me…
[Gra
bn
er
et
al.
CV
PR
’10
]
![Page 114: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/114.jpg)
ComputerVision Tracking Carl
![Page 115: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/115.jpg)
ComputerVision SUPPORTERS…
• … came with different strength.
• … change over time.
![Page 116: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/116.jpg)
ComputerVision SUPPORTERS…
• … came with different strength.
• … change over time.
![Page 117: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/117.jpg)
ComputerVision SUPPORTERShelp Tracking of…
• … objects which change there appearance very quickly.
• … occluded objects or object outside the image.
• … small and/or low textured objects or even “virtual points”.
![Page 118: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/118.jpg)
ComputerVision ETH-Cup Sequenze
![Page 119: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/119.jpg)
ComputerVision ETH-Cup: Humans
![Page 120: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/120.jpg)
ComputerVision Of the Web Tracker
![Page 121: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/121.jpg)
ComputerVision ETH-Cup: Supporters
![Page 122: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/122.jpg)
ComputerVision Beyond the Image
Supporters
![Page 123: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/123.jpg)
ComputerVision Coupled Motion
Supporters
![Page 124: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/124.jpg)
ComputerVision Changing Supporters
Supporters
![Page 125: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/125.jpg)
ComputerVision
Obviously, there are failure cases……. and magician knows that.
Supporters
![Page 126: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/126.jpg)
ComputerVision
Tracking IssuesTracking Issues
![Page 127: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/127.jpg)
ComputerVision Tracking Requirements
• Strongly depends on the application!
Robust, Accurate, Fast,…
• Constrain the tracking task!
Information about the object, dynamics, environment,…
![Page 128: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/128.jpg)
ComputerVision Tracking Issues
• Initialization
object position
Time t = 0
object position
![Page 129: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/129.jpg)
ComputerVision Tracking Issues
• Prediction vs. Correction– If the dynamics model is too strong, will end up
ignoring the data
– If the observation model is too strong, tracking is reduced to repeated detectionis reduced to repeated detection
http://www.ethlife.ethz.ch/archive_articles/091008_kalman_per/index
![Page 130: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/130.jpg)
ComputerVision Tracking Issues
• Obtaining observation…– Generative: “render” the state on top of
the image and compare
– Discriminative: classifier or detector scorescore
• …and dynamics model– specify using domain knowledge
– learn (very difficult)
![Page 131: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/131.jpg)
ComputerVision Tracking Issues
• Nonlinear dynamics– Sometimes needed
to keep multiple trackers in paralleltrackers in parallel
– E.g., for abrupt direction changes („Persons“)
Wrong prediction
Correctprediction
![Page 132: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/132.jpg)
ComputerVision Tracking Issues
• Data Association - Multiple Object Tracking– What if we don’t know which
measurements to associate with which tracks?tracks?
![Page 133: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/133.jpg)
ComputerVision Tracking Issues
• Data Association – Fast Motion
![Page 134: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/134.jpg)
ComputerVision Tracking Issues
• Data Association – Background / Appearance Change– Cluttered Background
– Changes in shape, orientation, color,…– Changes in shape, orientation, color,…
![Page 135: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/135.jpg)
ComputerVision Tracking Issues
• Data Association – Occlusions / Self Occlusions
![Page 136: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/136.jpg)
ComputerVision Tracking Issues
• Model- vs. Model-free-Tracking
![Page 137: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/137.jpg)
ComputerVision Tracking Issues
• Drift– Errors caused by dynamical model,
observation model, and data association tend to accumulate over time
![Page 138: Tracking - CVG @ ETHZ · Computer Vision Tracking Loop search Region actual object position from time t to t+1 evaluate classifier on sub-patches-+ - --create confidence map analyze](https://reader033.vdocuments.mx/reader033/viewer/2022060510/5f276c0b20ba1323f12bc0e3/html5/thumbnails/138.jpg)
ComputerVision End.