Triangle-based approach to the detection of human face

March 2001 PATTERN RECOGNITION

Speaker Jing. AIP Lab

Outline Introduction Segmentation of potential face regions Face verification Experimental results and discussion

Introduction 1/3

Given a still or video image, detect and localize an unknown number of faces

– Security mechanism (replace key, card,passwd)– Criminology (find out possible criminals)– Content-based image retrieval – video coding – video conferencing – Crowd( 大眾 ) surveillance and intelligent human-comput

er interfaces.

Applications

Problem

Introduction 2/3

Requirement

* achieve the task regardless of

- illumination, orientation, and camera distance

Why difficult ?

Human face is a dynamic objectHigh degree of variability in appearance ( 面孔的多變性 )

* Speedy and correct detection rate

Introduction 3/3

Drawbacks of the papers until now– Free of background– Cannot detect a small face ( < 50 *

50)– Cannot detect multiple face ( >3)– Cannot handle the defocus and noise– Cannot conquer the partial occlusion

of mouth or wear sunglasses– Cannot detect a face of side view

A classified algorithms

Begin the method

Overview of the system1. Form 4-connected components2. Find the center for each one

1. Search any 3 center that form an isosceles or right triangle

1. Normalize the size of potential face regions

1. Calculate the weight by mask function

Segmentation 4 step for segmenting the potential

face– Convert the input image to a binary image– Find the blocks using 4-connected

component– Search the triangle– Clip the satisfy triangle region

Step1: Convert the image RGB Color Image

– Eliminating the hue and saturation – Gray-level binary image

– Remove noise using opening operation– Eliminate holes by the closing operation

Gray-level < T are labelled as blackGray-level > T are white

Step 2:Form the blocks & Searching triangle Form the blocks by using 4-connected

components algorithm

Locate the center of each block

Searching the triangle– Frontal view (isosceles triangle)

– Side view (right triangle)

Step 3: Frontal view (isosceles triangle) Isosceles triangle: D(ij)=D(jk)

Matching rule:

i k

j

),max(25.0|| cbcb

),max(25.0|| cbab Eye to mouth

mouth to mouth

a

b c

Clipping the region 2/4

X1=X4=Xi – 1/3 dX2=X3=Xk + 1/3 dY1=Y2=Yi + 1/3 dY3=Y4=Yj – 1/3 d

Xi,Yi d Xk,Yk

Xj,Yj

x1 x2

Side view (right triangle) 3/4 Right triangle

Matching Rules: (25% derivation)1. 0.4 a < | a-c | < 0.6 a2. 0.13 a < | a-b | < 0.19 a3. 0.29 a < | b-c | < 0.44 a

i j

k

3

2 1a

b

c

Clipping the region 4/4

i

j

k

d

1.2d

d/4

d

d/6

X1=X4=Xi-d/6X2=X3=Xi+1.2dY1=Y2=Yi+d/4Y3=Y4=Yi-d

Speedup of searching

How many triangles ?

If the mouth & right eye are already known, => the left eye should be located in the near

area.

nC3

i

j

k

Face verification

3 steps in verificationStep1: Normalization the potential facial areas

– 60 * 60 pixels

Step 2: Calculating the weight by masking function

Step 3:Verification by thresholding the weight

Question 1 . How to generate the face mask ?

Question 2 . How to calculate the weight ?

Question 1 . How to generate the face mask ?

Read the 10 binary training masks Add the corresponding entries Binarized the added mask

Ex:

Question 2 . How to calculate the weight

Eye and mouth are labeled as black, others as white– If the pixels in the P is equal to T

• Both Black: Weight + 6• Both White : Weight + 2

– White in P and black in T• Weight –2

– White in T and black in P• Weight - 4

P: potential facial regionT: Training mask

Verification For each potential facial regions

– Threshold value is given for decision making• Front view => 4000 < threhold < 5500• Side view => 2300 < threhold < 2600

Finally, eliminate the regions that– Overlap with the chosen facial region

Result—frontal view

Original Binary Isosceles triangle

clipping Normalized

Result – Side View

Original Binary Isosceles triangle

clipping Normalized

Experimental results

500 test images– included 450 different persons– 600 faces that are used

11 faces cannot be found correctly98% success rate

Experiment result

Scaling: 5*5 to 640*480

Light condition

Experiment Result

Distinct position

Defocus face

Experiment Result

Changed expressions

Experiment Result

Noise Occlusion Sunglasses

cartoon Chinese doll

Experiment Result

2.5 sec 28 sec

Target machine: PII 233 PC

Experiment ResultMulti-faces and video stream

Experiment Result

False cases

Too Dark Right eye being occluded

Conclusion

Manage different sizes, changed light conditions, varying pose and expression

Cope with partial occlusion problem Detect a side-view face In the future, using this algorithm

for solving face recognition problem

My opinions The processing time depend on the

complexity of the image. Real-time requirement was

unachievable. (some images need 28 sec to process)