1 hierarchical part-based human body pose estimation * ramanan navaratnam * arasanathan thayananthan...
TRANSCRIPT
1
Hierarchical Part-Based Hierarchical Part-Based Human Body Pose EstimationHuman Body Pose Estimation
* Ramanan Navaratnam
* Arasanathan Thayananthan
† Prof. Phil Torr
* Prof. Roberto Cipolla
* University Of Cambridge † Oxford Brookes University
2
IntroductionIntroduction
Input
3
IntroductionIntroduction
Input Output
4
OverviewOverview
1. Motivation
2. Hierarchical parts
3. Template search
4. Pose estimation in a single frame
5. Temporal smoothing
6. Summary & Future work
5
OverviewOverview
1. Problem motivation ???
2. Hierarchical parts
3. Template search
4. Pose estimation in a single frame
5. Temporal smoothing
6. Summary & Future work
6
OverviewOverview
1. Problem motivation ???
2. Hierarchical parts
3. Template search
4. Pose estimation in a single frame
5. Temporal smoothing
6. Summary & Future work
7
OverviewOverview
1. Problem motivation ???
2. Hierarchical parts
3. Template search
4. Pose estimation in a single frame
5. Temporal smoothing
6. Summary & Future work
8
MotivationMotivation ‘Real-time Object Detection for Smart Vehicles’
– D. M. Gavrila & V. Philomin (ICCV 1999)
‘Filtering using a tree-based estimator’ – Stenger et.al. (ICCV 2003)
9
MotivationMotivation
Exponential increase of templates with dimensions
‘Real-time Object Detection for Smart Vehicles’ – D. M. Gavrila & V. Philomin (ICCV 1999)
‘Filtering using a tree-based estimator’ – Stenger et.al. (ICCV 2003)
10
MotivationMotivation ‘Pictorial Structures for Object Recognition’
– P. Felzenszwalb & D. Huttenlocher (IJCV 2005)
‘Human upper body pose estimation in static images’ – M.W. Lee & I. Cohen (ECCV 2004)
11
MotivationMotivation
Part based approach Assembling parts together is complex
‘Pictorial Structures for Object Recognition’ – P. Felzenszwalb & D. Huttenlocher (IJCV 2005)
‘Human upper body pose estimation in static images’ – M.W. Lee & I. Cohen (ECCV 2004)
12
MotivationMotivation ‘Automatic Annotation of Everyday Movements’
– D. Ramanan & D. A. Forsyth (NIPS 2003)
‘3-D model-based tracking of humans in action:a multi-view approach’ – D. M. Gavrila & L. S. Davis (CVPR 1996)
13
MotivationMotivation ‘Automatic Annotation of Everyday Movements’
– D. Ramanan & D. A. Forsyth (NIPS 2003)
‘3-D model-based tracking of humans in action:a multi-view approach’ – D. M. Gavrila & L. S. Davis (CVPR 1996)
‘State space decomposition’
14
Hierarchical PartsHierarchical Parts
15
Hierarchical PartsHierarchical Parts
16
Hierarchical PartsHierarchical Parts
17
Hierarchical PartsHierarchical Parts
18
Hierarchical PartsHierarchical PartsConditional prior
p(x i
/xpar ent(i))
Spatial dimensions (translation)Joint Angles
19
Hierarchical PartsHierarchical Parts
Head and torsoUpper armLower Arm
False Positive
Tru
e Po
siti
ve
20
Hierarchical PartsHierarchical PartsDetection Threshold = 0.81
Detections
Head and torso
6156
Part
21
Hierarchical PartsHierarchical PartsDetection Threshold = 0.81
Detections
Head and torso
6156
13 199 44 993
Part
Lower arm
22
Template SearchTemplate Search
23
Template SearchTemplate Search
24
Template SearchTemplate Search
25
Template SearchTemplate Search
Features Chamfer distance Appearance
26
Template SearchTemplate Search
Features Chamfer distance Appearance
27
Template SearchTemplate Search
Features Chamfer distance Appearance
28
Template SearchTemplate Search
Features Chamfer distance Appearance
29
Template SearchTemplate Search
Features Chamfer distance Appearance
30
Template SearchTemplate Search
Features Chamfer distance Appearance
31
Template SearchTemplate Search
Features Chamfer distance Appearance
32
Template SearchTemplate Search
Features Chamfer distance Appearance
33
Template SearchTemplate Search
Features Chamfer distance Appearance
34
Template SearchTemplate Search
Learning Appearance Match ‘T’ pose based on edge likelihood only in initial
frames
Update 3D histograms in RGB space that approximates P(RGB/part) and P(RGB)
35
Pose Estimation in a Single FramePose Estimation in a Single Frame
36
Pose Estimation in a Single FramePose Estimation in a Single Frame
37
Pose Estimation in a Single FramePose Estimation in a Single Frame
38
Temporal SmoothingTemporal Smoothing
HMM
39
Temporal SmoothingTemporal Smoothing
HMMT = t
40
Temporal SmoothingTemporal Smoothing
HMM
Viterbi back tracking
41
Temporal SmoothingTemporal Smoothing
Viterbi back tracking
42
Temporal SmoothingTemporal Smoothing
43
Summary & Future workSummary & Future work
Summary
Realtime process (unoptimized code at 1Hz, 2.4 Ghz IG RAM)
3D pose
Automatic initialisation and recovery from failure
44
Summary & Future workSummary & Future work
Summary
Realtime process (unoptimized code at 1Hz, 2.4 Ghz IG RAM)
3D pose
Automatic initialisation and recovery from failure
Future workExtend robustness to illumination changes
Non-fronto-parallel poses
Poses when arms are inside the body silhouette
Simple gesture recognition by assigning semantics to regions of articulation space