golf
DESCRIPTION
TRANSCRIPT
![Page 1: golf](https://reader033.vdocuments.mx/reader033/viewer/2022051412/54bec47a4a79598c3d8b462c/html5/thumbnails/1.jpg)
Robust Visual Golf Club Tracking
Vincent Lepetit, Nicolas Gehring, Pascal FuaCVlab - EPFL
![Page 2: golf](https://reader033.vdocuments.mx/reader033/viewer/2022051412/54bec47a4a79598c3d8b462c/html5/thumbnails/2.jpg)
2
Aim
• Completely automatic video based system.– No user intervention.– Use of a single video (PAL) camera.– No external expensive devices.– No specifically instrumented golf clubs or clothing.
• Usable in natural environments– Cluttered (fixed) background
1.5 sec
![Page 3: golf](https://reader033.vdocuments.mx/reader033/viewer/2022051412/54bec47a4a79598c3d8b462c/html5/thumbnails/3.jpg)
3
Club: thin, specular reflexion, moves very fast (up to 170km/h).
sec25
1+tt
![Page 4: golf](https://reader033.vdocuments.mx/reader033/viewer/2022051412/54bec47a4a79598c3d8b462c/html5/thumbnails/4.jpg)
4
• Club extraction• Tracking algorithm with
– local motion model– global motion model
![Page 5: golf](https://reader033.vdocuments.mx/reader033/viewer/2022051412/54bec47a4a79598c3d8b462c/html5/thumbnails/5.jpg)
5
Dealing with interlaced images• For each frame:
50 « half-images » per second
2 « half-images » from an interlaced image
![Page 6: golf](https://reader033.vdocuments.mx/reader033/viewer/2022051412/54bec47a4a79598c3d8b462c/html5/thumbnails/6.jpg)
6
Detection of moving objects
Threshold + Closing
Threshold + Closing
-
-AND
Mask of themoving objects in frame n
Frame n
Frame n-1
Frame n+1
![Page 7: golf](https://reader033.vdocuments.mx/reader033/viewer/2022051412/54bec47a4a79598c3d8b462c/html5/thumbnails/7.jpg)
7
Hypothesis generationDetection of adjacent parallel segments under the moving-object mask1. Edges detection
2. Segment detection (contour extraction, polygonal approximation)
3. Parallel segment detection and fusion
![Page 8: golf](https://reader033.vdocuments.mx/reader033/viewer/2022051412/54bec47a4a79598c3d8b462c/html5/thumbnails/8.jpg)
8
Some results of the parallel segments detection
Hypothesis generation
![Page 9: golf](https://reader033.vdocuments.mx/reader033/viewer/2022051412/54bec47a4a79598c3d8b462c/html5/thumbnails/9.jpg)
9
Hypothesis generation• The resulting segment is only a part of the shaft
Search for the shaft end-points
• Looking for the club head in the moving-object mask (as the last white point)
• Looking for the “hands” in the color image
![Page 10: golf](https://reader033.vdocuments.mx/reader033/viewer/2022051412/54bec47a4a79598c3d8b462c/html5/thumbnails/10.jpg)
10
• Results– In this example, two hypothesis:
– Others results:
Hypothesis generation
![Page 11: golf](https://reader033.vdocuments.mx/reader033/viewer/2022051412/54bec47a4a79598c3d8b462c/html5/thumbnails/11.jpg)
11
• Heuristics for removing some hypotheses– Given a 2D point somewhere between the golfer shoulders,we can remove some physically impossible hypotheses
Hypothesis generation
ImpossiblePossible
Shoulders Shoulders
• No accurate position for this point needed• Can be easily provided by the user
![Page 12: golf](https://reader033.vdocuments.mx/reader033/viewer/2022051412/54bec47a4a79598c3d8b462c/html5/thumbnails/12.jpg)
12
• Some wrong hypotheses can not be removed:
Why we still need a tracking algorithm ?
?
?
??
![Page 13: golf](https://reader033.vdocuments.mx/reader033/viewer/2022051412/54bec47a4a79598c3d8b462c/html5/thumbnails/13.jpg)
13
Tracking
• Many visual tracking techniques have been proposed in the computer vision literature.– Data Association approaches (MHT)– ConDensation
– Based on recursive motion models: Xt+1 = f(Xt)– Difficult to consider a specific motion such as a golf
swing.– Suffer from a lack of robustness for practical
applications when:• Frequent mis-detections• Large acceleration • Abrupt motion changes
![Page 14: golf](https://reader033.vdocuments.mx/reader033/viewer/2022051412/54bec47a4a79598c3d8b462c/html5/thumbnails/14.jpg)
14
New tracking algorithm
• Idea:– Take into account previous frames + next
frames– Consider the detections in these frames to
locally estimate the club shaft motion
![Page 15: golf](https://reader033.vdocuments.mx/reader033/viewer/2022051412/54bec47a4a79598c3d8b462c/html5/thumbnails/15.jpg)
15
New algorithmMLESAC applied to tracking:• Choose randomly 3 frames (in the previous and next frames)• Choose randomly one detection in these frames• Compute the shaft motion assuming a locally constant
acceleration• Estimate the shaft position in the previous and next frames
Several examples:
• Compute the support of the predicted motion i.e. the number offrames where there is a detection near the predicted position
• Repeat and keep the shaft motion with the maximum likelihood
Deals easily with mis-detections and false-alarmsRobust motion estimation
![Page 16: golf](https://reader033.vdocuments.mx/reader033/viewer/2022051412/54bec47a4a79598c3d8b462c/html5/thumbnails/16.jpg)
16
• Parametrisation of the shaft (double-pendulum model):
• Estimation– From the three randomly selected shafts si, sj, sk,– assuming a constant acceleration for all the parameters,
• we can predict the position, velocity and acceleration of the shaft
s 0 = [Shoulders 0, L 0, R 0, Ψ 0, ϕ 0] in the current frame:
• we can also predict the position in the other frames:
Motion estimation
s = [Shoulders, L, R, Ψ, ϕ] Lϕϕϕϕ
ψψψψ
R
������
�
�
������
�
�
−
−
−
=
2
)1(.1
2
)1(.1
2
)1(.1
kkk
jjj
iii
A
���
�
�
���
�
�
ΨΨΨ
=���
�
�
���
�
�
ΨΨΨ
−
k
j
i
A 1
0
0
0
��
�
…000 2
)1( Ψ−+Ψ+Ψ=Ψ ���nn
nn
![Page 17: golf](https://reader033.vdocuments.mx/reader033/viewer/2022051412/54bec47a4a79598c3d8b462c/html5/thumbnails/17.jpg)
17
Maximum Likelihood Estimation
• Mt motion at time t• Zt ={zt-nB … zt+nA} detection sets for frames t-nB to t+nA
• Random sampling:is an initial estimate of Mt, and refined using all the correct
detections
)()|(maxarg MMZpM tM
t δ=
)()|(maxarg~
SStM
MMZpMS
δ=
∏+
=++=
A
B
n
niSititSt ypMZp )|()|( ,z
Classical observation model
![Page 18: golf](https://reader033.vdocuments.mx/reader033/viewer/2022051412/54bec47a4a79598c3d8b462c/html5/thumbnails/18.jpg)
18
• The shaft position can be estimated when it is notdetected, with very good accuracy:
• Using the next frames makes the tracker:– More robust– More accurate– Almost Automatic!
Advantages
![Page 19: golf](https://reader033.vdocuments.mx/reader033/viewer/2022051412/54bec47a4a79598c3d8b462c/html5/thumbnails/19.jpg)
19
Other resultssame parameters
![Page 20: golf](https://reader033.vdocuments.mx/reader033/viewer/2022051412/54bec47a4a79598c3d8b462c/html5/thumbnails/20.jpg)
20
Making the tracker more robust
• Using a global motion model
![Page 21: golf](https://reader033.vdocuments.mx/reader033/viewer/2022051412/54bec47a4a79598c3d8b462c/html5/thumbnails/21.jpg)
21
upswing
downswing
Temporal Segmentation of the Sequence
![Page 22: golf](https://reader033.vdocuments.mx/reader033/viewer/2022051412/54bec47a4a79598c3d8b462c/html5/thumbnails/22.jpg)
22
Trajectory EstimationSeparated global motions models for– upswing– downswing
expressed in polar coordinates system
π
7π/3 7π/3
−π/2
![Page 23: golf](https://reader033.vdocuments.mx/reader033/viewer/2022051412/54bec47a4a79598c3d8b462c/html5/thumbnails/23.jpg)
23
Simple polynomial functions of rather small degrees
Upswingdeg. 4 polynomial
Downswingdeg. 6 polynomial
![Page 24: golf](https://reader033.vdocuments.mx/reader033/viewer/2022051412/54bec47a4a79598c3d8b462c/html5/thumbnails/24.jpg)
24
Trajectory Estimation
Same algorithm than before, with a global motion model•Robust estimation (b)•Refinement using the complete support (c)
![Page 25: golf](https://reader033.vdocuments.mx/reader033/viewer/2022051412/54bec47a4a79598c3d8b462c/html5/thumbnails/25.jpg)
25
Experimental results
![Page 26: golf](https://reader033.vdocuments.mx/reader033/viewer/2022051412/54bec47a4a79598c3d8b462c/html5/thumbnails/26.jpg)
26
Further Research
• Define a better global motion model (PCA ?)
• Analysis of the motion parameters
• Tracking of the golfer body