Download - 3D Hand Tracking
![Page 1: 3D Hand Tracking](https://reader035.vdocuments.mx/reader035/viewer/2022062300/557e788dd8b42a7b5c8b54e9/html5/thumbnails/1.jpg)
04/13/2023 1
A Unified Framework for 3D Hand Tracking
Rudra PoudelJose A S FonsecaJian J ZhangHammadi Nait-Charif
ISVC 2013
![Page 2: 3D Hand Tracking](https://reader035.vdocuments.mx/reader035/viewer/2022062300/557e788dd8b42a7b5c8b54e9/html5/thumbnails/2.jpg)
04/13/2023Next Introduction 2
Overview1. Introduction2. Applications3. Related Work4. Skin Colour Detection5. Hand Region Segmentation6. Hand Pose Estimation7. 3D Hand Tracking8. Conclusions
![Page 3: 3D Hand Tracking](https://reader035.vdocuments.mx/reader035/viewer/2022062300/557e788dd8b42a7b5c8b54e9/html5/thumbnails/3.jpg)
04/13/2023Continue … 3
1. Introduction• Aim: detect 3D
position of hand joints• Task in details:
o Hand detection (skin colour, hand shape)
o Joints detectiono Estimate 3D positions of the
joints
• Problems: appearance similarity, occlusion, random motions, 27 degree-of-freedom/dimensions
![Page 4: 3D Hand Tracking](https://reader035.vdocuments.mx/reader035/viewer/2022062300/557e788dd8b42a7b5c8b54e9/html5/thumbnails/4.jpg)
04/13/2023Next Applications 4
1. Introduction
Overview of the system
![Page 5: 3D Hand Tracking](https://reader035.vdocuments.mx/reader035/viewer/2022062300/557e788dd8b42a7b5c8b54e9/html5/thumbnails/5.jpg)
04/13/2023 5
2. Applications
Next Related Work
• Human machine/computer interaction- TV remote controller
• Sign language recognition• Gamming• Health and Medicine• Many more …
![Page 6: 3D Hand Tracking](https://reader035.vdocuments.mx/reader035/viewer/2022062300/557e788dd8b42a7b5c8b54e9/html5/thumbnails/6.jpg)
04/13/2023Continue … 6
3. Related Work• Computer vision is
holy grail of Artificial Intelligent/Robotics
• 3D Hand tracking deals with three major problems- detection, recognition, tracking
• Research started two decades back
Starner et al. (PAMI, 1998)
![Page 7: 3D Hand Tracking](https://reader035.vdocuments.mx/reader035/viewer/2022062300/557e788dd8b42a7b5c8b54e9/html5/thumbnails/7.jpg)
04/13/2023Next Skin Color Detection 7
3. Related Work• Oikonomidis, I., Kyriazis, N., Argyros, A.A.:
Efficient model-based 3d tracking of hand articulations using kinect. In: BMVC. (2011): template matching so computationally very expensive
• Keskin, C., Kirac, F., Kara, Y., Akarun, L.: Real time hand pose estimation using depth sensors. In: ICCV Workshops. (2011) 1228–1234: needed extra occlusion handling technique, hasn’t utilizes temporal information
![Page 8: 3D Hand Tracking](https://reader035.vdocuments.mx/reader035/viewer/2022062300/557e788dd8b42a7b5c8b54e9/html5/thumbnails/8.jpg)
04/13/2023 8
4. Skin Colour Detection
Next Hand Region Segmentation
• Learned two skin and non-skin histograms using- 4,700 skin and 9,000 non-skin images
• Manually labelled pixels is nearly 1 billion
• Includes all lighting conditions and races
![Page 9: 3D Hand Tracking](https://reader035.vdocuments.mx/reader035/viewer/2022062300/557e788dd8b42a7b5c8b54e9/html5/thumbnails/9.jpg)
04/13/2023 9
5. Hand Region Segmentation
Next Hand Pose Estimation
• Kinect depth image • Segmented hand using skin colour, depth and hand at previous frame
![Page 10: 3D Hand Tracking](https://reader035.vdocuments.mx/reader035/viewer/2022062300/557e788dd8b42a7b5c8b54e9/html5/thumbnails/10.jpg)
04/13/2023Continue … 10
6. Hand Pose Estimation
Learned hand parts classifier using 450 thousands artificial images
Example of hand parts detection
![Page 11: 3D Hand Tracking](https://reader035.vdocuments.mx/reader035/viewer/2022062300/557e788dd8b42a7b5c8b54e9/html5/thumbnails/11.jpg)
04/13/2023Continue … 11
6. Hand Pose Estimation
• Regression of joint positions using hough voting technique
![Page 12: 3D Hand Tracking](https://reader035.vdocuments.mx/reader035/viewer/2022062300/557e788dd8b42a7b5c8b54e9/html5/thumbnails/12.jpg)
04/13/2023Continue … 12
6. Hand Pose Estimation
Comparison: our vs. state-of-the-art Keskin (2011)
![Page 13: 3D Hand Tracking](https://reader035.vdocuments.mx/reader035/viewer/2022062300/557e788dd8b42a7b5c8b54e9/html5/thumbnails/13.jpg)
04/13/2023 13
6. Hand Pose Estimation
Example of hand pose estimation
Next 3D Hand Tracking
![Page 14: 3D Hand Tracking](https://reader035.vdocuments.mx/reader035/viewer/2022062300/557e788dd8b42a7b5c8b54e9/html5/thumbnails/14.jpg)
04/13/2023 14
7. 3D Hand Tracking
Next Conclusion
• Tracking using- hand joints estimation at current frame (t), temporal information and kinematic constraints
• 3D hand tracking demo: http://www.youtube.com/watch?v=xqyfWWlAnVI
![Page 15: 3D Hand Tracking](https://reader035.vdocuments.mx/reader035/viewer/2022062300/557e788dd8b42a7b5c8b54e9/html5/thumbnails/15.jpg)
04/13/2023 15
8. Conclusion• Depth sensor Kinect: RGB and depth images• Skin colour detection• Hand segmentation using skin colour, depth and
hand position at previous frame• Hand joints regression• 3D hand tracking using joints detection, temporal
information and kinematic constraints
• Used techniques- Naive Bayes for skin detection, Random Forest for joints detection, Mean-shift for joint vote aggression, Markov Random Fields for tracking
Next Questions
![Page 16: 3D Hand Tracking](https://reader035.vdocuments.mx/reader035/viewer/2022062300/557e788dd8b42a7b5c8b54e9/html5/thumbnails/16.jpg)
04/13/2023 16
Questions ?
Thank you !!!
9th International Symposium on Visual Computing