estimation of skin color range using achromatic features

Wen-Hung Liao

Department of Computer ScienceNational Chengchi University

November 27, 2008

Estimation of Skin Color Range Using Achromatic

Features

Outline

Motivation and Related WorkColor SpacesFixed vs. Dynamic Range ApproachExperimental Results

Skin color segmentationHand & finger detection

Conclusion

Background

Previous claims: skin color is restricted to a “fixed” range in certain color coordinates: Sobottka & Pitas: Hue:[0,50º],

Saturation:[0.23,0.68] Chai & Ngan: Cb:[77,127], Cr[137,177] Kawato & Ohya: Decision boundary in

normalized RGB space

Decision Boundary in Normalized RGB Space

Sobottka & Pitas: Fixed Hue + Saturation

Chai & Ngan: Fixed Cb,Cr

Kawato & Ohya

Comparative Analysis

From: Phung et al, Skin segmentation using color pixel classification: analysis and comparison, IEEE Transactions on PAMI, 2005.

Observation

It is true that the skin color lies in a small range, yet this range tends to shift under different lighting conditions.

Question: Is it possible to dynamically adjust the range of skin color to enhance the robustness of color-based segmentation?

The Proposed Solution

Use achromatic information (face detection) to help determine the range.

Limitation: Face must be present and

detected. Suitable for vision-based human

computer interface.

Five Classes of Color Space

Color space Representative color space

Basic color spaces RGB 、 normalized RGB

Perceptual color spaces HSV 、 HIS

Orthogonal color spaces YCbCr 、 YUV

Perceptually uniform color spaces

CIELab 、 CIELuv

Other color spaces Mixture

Color Spaces Investigated

color space domains

RGB Red 、 Green 、 Blue

HSV Hue 、 Saturation 、 Value

CIELab L、 a、 b

YCbCr Y、 Cb 、 Cr

CIELuv L、 u、 v

* Dynamically set the threshold in Hue domain

Determining the Threshold (I)

Step 1: detecting and locating the face Step 2: mark the cheek area X = X0 +(W0 /5)

Y = Y0 +(H0 /2) width = W0 /5 height = H0 /5

Step 3: obtain the hue distribution of the marked area.

(X(X00, Y, Y00))WW00

HH00

Determining the Threshold (II)

Step 4: assume that the histogram is peaked at A: search to the left and right of A

untilLocal minimum <A/10 is

uncoveredA non-zero global minimum is found

0 255

Face Detection using DSE

Directional Sobel Edges

Experiment: Skin Color Segmentation

Compare the performance of 5 different methods: Dynamic threshold Fixed threshold – fixed Hue Kawato & Ohya – fixed Normalized RGB Sobottka & Pitas – fixed Hue & Saturation Chai & Ngan – fixed Cb & Cr

Material Images captured by a low-cost webcam

under different lighting conditions. A total of 400 images (taken indoor) are

manually segmented and labeled.

Skin Color Segmentation: Experimental Results

false positive

false negative

true negative

true positive

Dynamic Threshold

0.0736 0.1706 0.9264 0.8294

fixed Hue 0.2125 0.3361 0.7875 0.6639

fixed Normalized RGB

0.0504 0.5303 0.9496 0.4697

fixed Hue & Sat

0.0588 0.5747 0.9412 0.4253

fixed Cr & Cb 0.0857 0.2996 0.9143 0.7004

Best and Worst Case Performance

best TP worst TP

Dynamic Threshold

0.9947 0.3494

fixed Hue 0.9977 0.0733

fixed Normalized RGB

0.9055 0.0002

fixed Hue & Sat 0.8891 0.0005

fixed Cr & Cb 0.9447 0.2234

Recall and Precision

00.10.20.30.40.50.60.70.80.9

1

adaptive fixed Hue fixed RGB fixed Hue& Sat

fixed Cr &Cb

Recall Precision

Recall = TP/(TP+FP)Precision =

TP/(TP+FN)

Speed-up the Process1. Detecting Face

2. Record color distribution of cheek area

3. Tracking face 4. Local search

5. Update color distribution

(After K frames)

Performance Improvement

0

5

10

15

20

25

30

0 10 20 30 40

K

FPS

Experiment: Hand Detection

Color-based hand segmentation No post-processing Does not involve statistical modeling

and classifier

Plamar vs. Dorsal Side

Hue histogram

Hue histogram

Hand Detection: Experimental Results

Hand detection

Dorsal sideDorsal side

(fingers)Plamar side

Plamar side (fingers)

Accuracy 92.65% 94.26% 90.78% 95.01%

Fingertip Detection

150 images# of

fingers detected

Dynamic threshold Fixed Threshold

5 108 72% 17 11%

4 21 14% 22 15%

3 10 7% 23 15%

2 5 3% 20 13%

1 1 1% 20 13%

0 5 3% 48 33%

Conclusion

Perform comparative evaluation of several color-based segmentation methods.

Propose and implement a dynamic range estimation algorithm using achromatic features.

Superior performance in terms of skin-color segmentation, hand and finger detection.

Suitable for vision-based HCI.

Q & A

Thank you

Experimental Result

Dynamic Threshold worst TP

Experimental Result

Fixed Hue worst TP

Experimental Result

Fixed Normalized RGB worst TP

Experiment Result

Fixed Hue & Saturation worst TP

Experiment Result

Fixed Cb & Cr worst TP

Recall = TP/(TP+FP)Precision =

TP/(TP+FN)