interactive sand art drawing using rgb-d sensor international journal of software engineering and...

Interactive Sand Art Drawing Using RGB-D Sensor

International Journal of Software Engineering and Knowledge EngineeringSai-Keung Wong, Kai-Min Chen, Ting Yu Chen

National Chiao Tung University

• Introduction

• Related Works

• Proposed System and Method

• Experimental Result

• Conclusion

2

Outline

3

Introduction

4

Related Work

5

System Overview

6

Background Subtraction

• Since our skin color detection method is based on HS (hue saturation color space), the problem induced by illumination change has little effect to our system.

• Employ the running average method by Piccardi [26] to remove the stationary objects in the scene.

[26] M. Piccardi. Background subtraction techniques: a review. In IEEE International Conference on Systems, man and cybernetics, volume 4, pages 3099-3104, 2004.

7

Depth Segmentation

• Not only uses skin color but also depth information to extract the regions of hands.

• Apply a depth threshold to discard the parts that are farther away from the camera and keep the parts close to the camera.

8

Skin color detection

• Apply pixel-based skin color detection method [30] in hue saturation color space to extract the skin part from the image.

• Even though the illumination changes in our testing environment, we still obtain acceptable segmentation results.

• After extracting the skin part, we use morphology closing operation and Gaussian filter to smoothen the skin segmentation result.

[30] V. Vezhnevets, V. Sazonov, and A. Andreeva. A survey on pixel-based skin color detection techniques. In Proceedings of Graphicon, volume 3, pages 85-92, 2003.

9

Hand feature extraction

• The goal is to analyze the image and extract the information that is useful for the application of sand art drawing.• palm position• The fingertips• the hand gesture

10

Hand feature extraction

• We extract the contours in the image and these contours represent the shape of hands.

• After the contours are extracted, we perform a blob pruning method.• remove contours with small areas as these small areas do not probably belong to hands.

• Then, we proceed to perform the hand feature extraction algorithm which has three steps.

11

Step One: distance transform.

12

Step Two: finger extraction based on the time series curve.

13

Step Three: computation of the finger position.

14

Motion/gesture management

15

Experimental Results