FACE IMAGE QUALITY INSPECTION SYSTEM ACCORDING TO ISO

STANDARD

ONG PAIK WEN

UNIVERSITI TEKNOLOGI MALAYSIA

FACE IMAGE QUALITY INSPECTION SYSTEM ACCORDING TO ISO

STANDARD

ONG PAIK WEN

A project report submitted in partial fulfilment of the

requirements for the award of the degree of

Master of Engineering (Electrical - Computer and Microelectronic System)

Faculty of Electrical Engineering

Universiti Teknologi Malaysia

JUNE 2012

iii

Specially dedicated to my beloved parents, brother and sister for their continuous

love, encouragement, guidance, motivation, support and inspiration throughout my

journey of education....

iv

ACKNOWLEDGEMENT

First and foremost, I would like to grab this opportunity to express my sincere

gratitude to my supervisor, Assoc. Prof. Dr. Syed Abdul Rahman for the guidance,

motivation, inspiration and inputs provided throughout the duration of completing

this project report. Without his never ending support and guidance, this project report

would not have been the same as presented here.

My sincere appreciation also extends to all my friends and colleagues who

have rendered their assistance at various occasions.

My fellow postgraduate course mates should also be recognized for sharing a

lot of technical knowledge with me and provided lots of encouragement throughout

the journey of completing this project report.

Last but not least, to my beloved family who have always been there to

encourage, comfort and give their fullest support when I most needed them.

v

ABSTRACT

Poor quality of face images is one of the reasons leading towards face

recognition performance degradation. Therefore, this gives the motivation to have an

International Standard (ISO/IEC 19794-5 Document) to provide guidelines for the

usage of proper facial photographs for applications such as the E-Passport so that the

face recognition process can be carried out more effectively. Nevertheless, despite

the emergence of the ISO/IEC 19794-5 Standard, traditional passport photographs

quality acceptance presently used is based on human visual perceptions and is very

subjective. This is because there is no standardized checking system utilized which is

able to give the same result on the quality acceptance of a photo regardless where or

when it is being evaluated. As a result, there is a need for a system that is able to

perform automatic checking on a passport size image to be developed. This project

develops a system to perform automatic checking on a passport size image to ensure

that it satisfies the image quality requirements according to the ISO/IEC 19794-5

Standard. The criteria considered in this project are image resolution, image aspect

ratio, image brightness, image background colour, image eye distance, image head

height and head width as well as image head rotation. The system developed

managed to achieve at least 90% accuracy on all the attributes evaluated.

vi

ABSTRAK

Gambar pasport dengan kualiti yang kurang memuaskan adalah salah satu

sebab utama yang akan menjejaskan proses pengenalpastian identiti individu

berdasarkan proses pengenalan muka. Dengan ini telah tercetusnya motivasi untuk

pembentukan satu Standard Antarabangsa (Dokumen ISO/IEC 19794-5) bagi

menyediakan garis panduan untuk penggunaan gambar-gambar yang sesuai dan

berpadanan dalam dokumen pengenalan diri seperti E-Pasport supaya proses

pengenalan muka dapat dijalankan dengan lebih lancar dan berkesan. Namun begitu,

walaupun Dokumen ISO/IEC 19794-5 telah sedia ada untuk rujukan, tetapi penilaian

kualiti gambar pasport yang digunakan pada masa kini masih berdasarkan persepsi

visual manusia. Oleh itu, penerimaan atau penolakan kualiti gambar pasport masih

merupakan sesuatu proses yang sangat subjektif. Ini kerana tidak adanya satu sistem

penilaian yang selaras dan seragam yang dapat digunakan untuk memberi keputusan

yang seragam kepada penerimaan kualiti gambar pasport tidak kira bila atau di mana

gambar tersebut dinilai. Dengan ini, wujudnya keperluan untuk membangunkan satu

sistem yang mampu menilai kualiti gambar bersaiz pasport secara automatik. Projek

ini bertujuan untuk melahirkan satu sistem untuk melaksanakan penyemakan

automatik kualiti gambar bersaiz pasport untuk memastikan bahawa ia memenuhi

kualiti seperti mana yang telah ditetapkan dalam Standard ISO/IEC 19794-5. Kriteria

yang dipertimbangkan dalam projek ini adalah resolusi imej, nisbah aspek imej,

kecerahan imej, warna latar belakang imej, jarak mata imej, ketinggian dan lebar

kepala imej serta sudut putaran kepala imej. Sistem yang dibina berjaya mencapai

ketepatan sekurang-kurangnya 90% ke atas semua kriteria yang dinilai.

vii

TABLE OF CONTENTS

CHAPTER

1

2

TITLE

DECLARATION

DEDICATION

ACKNOWLEDGEMENT

ABSTRACT

ABSTRAK

TABLE OF CONTENTS

LIST OF TABLES

LIST OF FIGURES

LIST OF ABBREVIATIONS

INTRODUCTION

1.1 Project Background

1.2 Problem Statement

1.3 Objective

1.4 Scope of Work

1.5 Project Report Outline

LITERATURE REVIEW

2.1 Researches on Face Image Quality Evaluation

2.2 Researches on Face Region Detection

2.3 Researches on Eye Detection

PAGE

ii

iii

iv

v

vi

vii

x

xi

xvi

1

1

3

4

4

6

7

8

10

15

viii

3

4

METHODOLOGY

3.1 Software Selection for Project Implementation

3.2 Colour Space Selection for Image Processing

3.3 Stages for Implementation

3.3.1 Pre-Processing Stage

3.3.2 Detection Stage

3.3.2.2 Image Width & Image Height

3.3.2.2 Image Aspect Ratio

3.3.2.3 Image Brightness

3.3.2.4 Background Colour

3.3.2.5 Eye Position

3.3.2.6 Head Width & Head Height

3.3.2.7 Head Rotation

3.3.3 Recognition Stage

3.3.3.1 Image Width & Image Height

3.3.3.2 Image Aspect Ratio

3.3.3.3 Image Brightness

3.3.3.4 Background Colour

3.3.3.5 Eye Position

3.3.3.6 Head Width & Head Height

3.3.3.7 Head Rotation

3.3.3.8 Overall Quality Check Result

3.3.4 Display Stage

3.4 Project Milestone

EXPERIMENTS, RESULTS & ANALYSIS

4.1 Pre-Processing Stage

4.2 Detection & Recognition Stage

4.2.1 Image Width, Image Height & Image

Aspect Ratio

4.2.2 Image Brightness

4.2.3 Background Colour

4.2.4 Warning Box

20

20

21

21

22

23

23

23

23

25

28

29

31

31

32

33

33

35

36

38

40

41

41

42

44

44

46

46

48

52

56

ix

5

4.2.5 Eye Distance

4.2.6 Head Width & Head Height

4.2.7 Head Rotation

4.2.8 Overall Quality Check Result

4.3 Display Stage

4.4 Performance Comparison

CONCLUSION & FUTURE WORK

5.1 Conclusion

5.2 Future Work and Improvements

57

61

64

66

68

70

72

72

73

REFERENCES 74

x

LIST OF TABLES

TABLE NO. TITLE PAGE

1.1 General face image requirements in the ISO/IEC

19794-5 Standard 2

1.2 Criteria from the ISO/IEC 19794-5 Standard that will

be evaluated by the Face Image Quality Inspection

System 5

4.1 Measured mean values for image brightness assessment 49

4.2 Recognition results for the Face Image Quality

Inspection System 71

xi

LIST OF FIGURES

FIGURE NO. TITLE PAGE

1.1 Generic face image recommended in the ISO/IEC 19794-5

Standard 3

2.1 Colour Triangle Model by Jun Zhang et al. 11

2.2 The CCS Model used by Jun Zhang et al. 11

2.3 Modified Golden Ratio approximation used by Y.H. Chan

et al. 13

2.4 Skin detection algorithm by V. A. Oliveira et al. 14

2.5 The proposed eye detection method by Qiong Wang et al. 16

2.6 Proposed method for eye detection by Kun Peng et.al. 19

3.1 High level block diagram of the overall implementation 22

3.2 Mean computation region for the colour elements 24

3.3 Brightness alteration of the test images by applying

different gamma values 25

3.4 Cropped regions for background colour check 26

3.5 HSV colour palette used for background colour

computation 27

3.6 Detection of eyes in a passport size image 28

3.7 Face region segmentation and extraction 29

xii

3.8 Diagram showing the detected eye locations and the

corresponding head width and height computation 30

3.9 Head rotation computation 31

3.10 Overall flow chart for the computation and evaluation of the

image width, image height and image aspect ratio 32

3.11 Overall flow chart for the computation and assessment of

the image brightness for an input image 34

3.12 Overall flow chart for the computation of the image

background colour 36

3.13 Overall flow chart for the computation of the image eye

distance 37

3.14 Overall flow chart for the computation of the image head

width and head height 39

3.15 Overall flow chart for the computation of the image head

rotation 40

3.16 GUI developed for the Face Image Quality Inspection

System 42

3.17 Project Gantt chart 43

4.1 Conversion of RGB image to Gray image 45

4.2 Conversion of RGB image to YCbCr image and separation

to their individual components 45

4.3 Conversion of RGB image to HSV image and separation to

their individual components 46

4.4 An example where the computation of the image width,

image height and image aspect ratio is meeting the ISO

Standard 47

xiii

4.5 An example where the computation of the image width,

image height and image aspect ratio is not meeting the ISO

Standard 48

4.6 Example of input image with appropriate brightness 50

4.7 Example of input image which is considered as bright

because there are some faded shades in the face region 50

4.8 Example of input image which is considered as dark 51

4.9 Instance where the brightness of the input image is

misclassified due to the colour of the clothing worn is

almost similar to the image background 52

4.10 Samples where the background colour of the input image is

meeting the ISO Standard because the background is either

white or blue 53

4.11 Instances where the background colour of the input image is

not meeting the ISO Standard because it is not in light blue,

white or grey in colour 54

4.12 A case where the background is detected as non-uniform as

there is an outlier spot in the top right corner of the H

Element image 55

4.13 Another sample where the background is detected as non-

uniform as there is some non-uniform spots in the H

Element image 55

4.14 Warning Box to prompt user for the continuation of the

image quality assessment of the input image 56

4.15 Condition where the user selected "No" at the warning box

and the computation for the subsequent categories under the

biometric attributes will be skipped 57

xiv

4.16 Example where the eyes are successfully detected and

measured on the input image. The eyes are marked with red

markers 59

4.17 Example where the input image is a spectacles wearer and

the eyes are not detected if the "Person is wearing

spectacles" checkbox is not checked 59

4.18 Example of the successful eye detection if the "Person is

wearing spectacles" checkbox is checked. The eyes are

marked with red markers 60

4.19 Demonstration of a case where the eye distance

measurement is slightly inaccurate due to the reflection on

the spectacles of the input image 60

4.20 Example where the head height and head width are

accurately measured for an adult. The measurement points

for the head width and head height are marked with greeen

markers and yellow markers respectively 62

4.21 Illustration where the head height and head width are

accurately measured for a child below 11 years old. The

measurement points for the head width and head height are

marked with green markers and yellow markers respectively 63

4.22 Example where the head height measurement is not accurate

due to the shadow induced by the head scarf 63

4.23 Example where the head height and head width

measurement is not computed because the face region of the

input image cannot be deteced. This is because the image

does not contain the required colour information for the face

region detection 64

4.24 Illustration where the head rotation of the input image is

meeting the ISO / IEC 19794-5 Standard recommendation 65

xv

4.25 Instance where the head rotation of the input image is not

meeting the ISO Standard recommendation because the

rotation goes above 5 degrees 65

4.26 Another case where the head rotation of the input image is

not meeting the ISO Standard recommendation because the

rotation goes above 5 degrees 66

4.27 An example where the input image is meeting the ISO

Standard as all of the attributes are within the desired range

and dimensions 67

4.28 An example where the input image is not meeting the ISO

Standard because the head height is smaller than the

required dimension 68

4.29 The final GUI which is deployed to the eventual Face Image

Quality Inspection System 69

4.30 Face Image Quality Inspection System after completing the

evaluation of the quality of an input image 70

xvi

LIST OF ABBREVIATIONS

ACC - Adaptive Cross-Correletion Algorithm

E-Passport - Electronic Passport

GUI - Graphic User Interface

HSV - Colour space based on the Hue, Saturation and Value elements

ISO - International Organization for Standardization

RGB - An additive colour model in which the Red, Green and Blue

light is added together in various ways to reproduce a broad

array of colours

YCbCr - A family of colour spaces used as a part of colour image

pipeline in video and digital photography systems. Y is the

luma component while Cb and Cr are the blue-difference and

red-difference chroma components

1

CHAPTER 1

INTRODUCTION

1.1 Project Background

Nowadays, many countries including Malaysia have started to implement the

usage of biometric passport as the identification document for travelers. This

approach is to limit the usage of falsified documents by irresponsible individuals.

Over the years, fingerprints have been employed as the key differentiator to

distinguish between different individuals. Nevertheless, to further enhance the

security measures, facial image has also been added as one of the mandatory

biometric identifier to be used in the biometric passport. Unfortunately, face image

of bad quality is one of the reasons leading towards face recognition performance

degradation. Poor illumination, tilting of the facial pose and bad focus are among the

fundamental reasons that create degraded quality photograph which eventually may

lead to disqualification of the facial image to be used in identification documents.

Therefore, this creates the motivation to have an International Standard (ISO/IEC

19794-5 Document) to provide guidelines for the usage of proper facial photograph

for applications such as the E-Passport so that the face recognition process can be

carry out more effectively in order to achieve a tighter security control.

The ISO/IEC 19794-5 Standard proposes the defined thresholds and

allowable ranges for the biometric parameters of a face image. It also specifies the

recommended size for the photograph to be used in the E-Passport. The standard also

includes instructions for proper lighting, facial pose and focus distance when the

2

photograph is taken. Generally, the ISO/IEC 19794-5 Standard categorized the face

image qualities into three main aspects. The aspects are scene requirements,

photographic requirements and digital requirements which are summarized in Table

1.1 below. In addition, Figure 1.1 shows the generic face image recommended in the

ISO/IEC 19794-5 Standard.

Table 1.1: General face image requirements in the ISO/IEC 19794-5 Standard [1]

Clause Attribute Constraint

Scene

Posture Control on deviation from frontal

Illumination Uniformly illuminated with no

shadow

Background Plain light coloured

Eyes Open and clearly visible

Mouth Close and clearly visible

Photographic

Head Position Placed in the center

Distance to Camera Moderate head size

Colour Neutral colour

Exposure Appropriate brightness

Digital Focus Not out of focus

Resolution Width constraint of the head

3

Figure 1.1: Generic face image recommended in the ISO/IEC 19794-5 Standard [1]

1.2 Problem Statement

Traditional passport photographs quality acceptance presently used are based

on human visual perceptions. Therefore, the pass or fail criteria is very subjective as

it really depends on the leniency of the evaluator to judge the quality of the image

provided. An image maybe considered acceptable by evaluator A but it may not be

the same verdict when it is being examined by evaluator B. Hence, the gap that exists

today is that there is no standardized checking system utilized which is able to give

the same result on the quality acceptance of a photo regardless where or when it is

being evaluated.

As a result, there is a need for a system that is able to perform automatic

checking on a passport size image to be developed. This system will be utilized to

4

ensure that the passport photograph used in the identification document satisfies the

image quality requirements according to the ISO/IEC 19794-5 Standard.

Furthermore, this system can then be employed across all the immigration offices or

at the centers where the passport photos are captured so that a standardized checking

process can be realized.

1.3 Objective

The purpose of this project is to develop a software system that is able to

evaluate the quality of a digital passport size photograph to ensure that the image

complies with the ISO/IEC 19794-5 Standard. In addition, a recommendation will be

provided at the end of the test to confirm if the image presented is suitable for use in

identification documents such as the E- Passport. The developed system is called the

Face Image Quality Inspection System and this naming convention will be used in

the subsequent sections of this project report.

1.4 Scope of Work

The Face Image Quality Inspection System involves a series of research work

on digital image processing to develop a system that is capable to automatically

validate the face images provided to check if it satisfies the ISO/IEC 19794-5

Standard. The system is developed based on the following assumptions:

The Face Image Quality Inspection System is a purely software based system

developed with the Matlab software.

The input to the system is a passport size photograph.

The input image provided is guaranteed to contain a human face.

The input image is assumed to be containing only one face and is captured

under controlled environment.

5

Images wearing head gear and sunglasses are not covered in the context of

this project.

There are actually a lot of criterions listed in the ISO/IEC 19794-5 Standard

to specify the requirements of a recommended facial image. However, only a subset

of the criterions as listed in Table 1.2 will be covered and evaluated by the Face

Image Quality Inspection System.

Table 1.2: Criteria from the ISO / IEC 19794-5 Standard that will be evaluated by the

Face Image Quality Inspection System

Criteria

ISO / IEC 19794-5 Recommendation

(Full Frontal Image)

Image

General

Attributes

Image Width Minimum 420 pixels

Image Height Minimum 525 pixels

Image Aspect Ratio

(Width : Height)

Between 1:1.25 to 1:1.34

Image Brightness Appropriate brightness level

Image Background

Colour

Plain and light coloured background

(grey, light blue or white)

Image

Biometric

Attributes

Image Eye Position Inter eye distance between 20% to 30%

of the Image Width

Image Head Width Occupy 50% to 70% of the Image

Width

Image Head Height Occupy 70% to 80% of the Image

Height

Image Head Rotation Less than +/- 5 degrees

The results of the image quality evaluation from the Face Image Quality

Inspection System will be displayed onto a GUI developed using Matlab.

Recommendation on whether the image is appropriate to be used in identification

documents will be provided based on these results. If image resolution is the only

condition that is failing on the input image while the other criterions fulfill the

6

required specifications, the Face Image Quality Inspection System is capable to help

resize the image to the minimum recommended size and certify the image for use in

the identification document. Otherwise, if any one of the other image attributes fail,

then the image will not be approved for use in identification document.

1.5 Project Report Outline

The organization of this project report is as follows. Chapter 1 provides some

insights on the project background and problem statement that exists today which

leads to the motivation of developing the Face Image Quality Inspection System as

the research work for this project report. In addition, the project objectives and scope

of work is also discussed in Chapter 1. Meanwhile, literature reviews covering the

previous work on image quality evaluation, face detection as well as eye detection

are discussed in Chapter 2. The methodological approach to realize the Face Image

Quality Inspection System is depicted in Chapter 3. Implementation results with the

Matlab software applying all the algorithms discussed in the methodology section are

presented in Chapter 4. Finally, Chapter 5 covers the conclusion along with the

recommendations for future research work and possible improvements to further

enhance the Face Image Quality Inspection System.

74

REFERENCES

[1] ISO/IEC JTC 1/SC 37 N 506. Biometric Data Interchange Formats Part 5:

Face Image Data, 22 March 2004

[2] ISO/IEC FCD 19794-5 Conformance Test Methodology Document

[3] M. Subasic, S. Loncaric, T. Petkovic, H. Bogunovic, Face Image Validation

System, Proceedings of the 4th International Symposium on Image and Signal

Processing and Analysis, 2005

[4] Jitao Sang, Zhen Lei, Stan Z. Li, Face Image Quality Evaluation for ISO/IEC

Standards 19794-5 and 29794-5, Center for Biometrics and Security

Research, Beijing, China, 2009

[5] Y.H. Chan, S.A.R. Abu-Bakar, Face Detection System Based on Feature-

Based Chrominance Colour Information, Proceedings of the International

Conference on Computer Graphics, Imaging and Visualization (CGIV’04),

2004

[6] Jun Zhang, Qieshi Zhang, Jinglu Hu, RGB Color Centroids Segmentation

(CCS) for Face Detection, ICGST-GVIP Journal, ISSN 1687-398X,

Volume(9), Issue(II), April 2009

[7] V. A. Oliveira, A. Conci, Skin Detection using HSV Color Space,

Computation Institute, Universidade Federal Fluminense, Brazil, 2009

[8] Tanmay Rajpathaka, Ratnesh Kumarb, Eric Schwartzb, Eye Detection Using

Morphological and Color Image Processing, 2009 Florida Conference on

Recent Advances in Robotics, FCRAR 2009

[9] Qiong Wang, Jingyu Yang, Eye Detection in Facial Images with

Unconstrained Background, Journal of Pattern Recognition Research 1

(2006) 55-62, 2006

[10] Kun Peng, Liming Chen, Su Ruan, Georgy Kukharev, Robust Algorithm for

Eye Detection on Gray Intensity Face without Spectacles, JCS&T Vol. 5 No.

3, October 2005