Face Recognition Based on 4 Dimensions LocalBinary Patterns

Chao Wang, Jucheng Yang*, Yarui Chen, Chao Wu,Song Dong, Xiaoyuan Zhang

College of Computer Science and InformationEngineering, Tianjin University of Science and

TechnologyTianjin, China.

E-mail: jcyang@tust.edu.cn

Abstract—In face recognition, LBP (Local Binary Patterns) is avery popular method, which can solve the defects of thetraditional local feature extraction methods with fixed scale andsmall extraction scale. However, the LBP operator only describesthe relationship between the center pixel and its neighborhoodpixels, it ignores the relationship among the operators. 3DLBP (3Dimensions Local Binary Patterns) operator embodies thisrelationship to get a better local description. However, both ofthem neglect the center pixel value, which also reflects someproperties of the image points. In this paper, we propose a novelface recognition method based on 4DLBP (4 Dimensions LocalBinary Patterns), which adds the pixel value of the center pointinto the 3DLBP features for face recognition. We firstly partitionthe face image into small blocks, and then we extract the 4DLBPfeatures of the blocked images, and combine the features toobtain the final facial features. Finally, we use the extremelearning machine (ELM) as classifier to train and classify the faceimages. The experimental results show the proposed method hasbetter performances than the traditional methods.

Keywords-LBP; 3DLBP; 4DLBP; face recognition; ELM

I. INTRODUCTIONBiometrics is a kind of science of using individual

personal characteristics to verify the personal identity. It isinherently more reliable than traditional methods such aspasswords and PIN numbers. The core of biometric recognitionis how to access these biological characteristics, and how to usea variety of image processing and pattern recognitionalgorithms to verify its identity. Usually, biometrics isclassified into two classes: biological and behavior biometrics.Biological biometrics mainly include fingerprint, palm vein,retina, iris, face, body odor, even blood vessels, DNA, skeletaletc,while behavior biometrics mainly include signature, speech,gait, etc..

At present, face recognition[1][2][3][4] becomes theresearch focus, which has wide application for the advantages,such as direct, friendly, convenient, non-invasive and so on.Usually, human can easily identify face and facial expression,but it is a challenge for machine to do it. There are manyreasons. First of all, the face image is a 3D non-rigid surfacewith irregular features. Secondly, the face images may changewith the age, health and the expression. Thirdly, differentilluminations and angles will influent the accuracy of facerecognition. Besides, the mechanism of human brain for face

recognition is unknown, which involves the computer vision,pattern recognition, physiology and psychology etc. In recentlyyears, many specialists have made deep researches on facerecognition [5].The current face recognition methods aredivided into two categories: the model-based and the subspace-based. The model-based methods [6][7] use the spatialgeometry information and the relationship between the featurepoints to achieve face recognition. The subspace-basedmethods [8][9][10][11] project the feature vector to thesubspace for further recognition.

Ojala[12] in 1999 first proposed the LBP (Local BinaryPatterns) operator,which is defined as a gray-scale invarianttexture measure, and it has been widely used in facerecognition. The LBP has the following advantages: 1) TheLBP has high discriminant ability, and it is insensitive toillumination conditions. So the LBP can describe the localcharacteristic effectively. 2) The LBP is simple and fast.However, the traditional LBP operator does not consider thecenter pixel point, and it loses some local structure informationin a particular case. In 2002, Ojala [13]proposed an extendedmodel of LBP operator, which are uniform pattern LBP androtation invariant LBP operator. Huang [14] proposed a facerecognition method based on 3DLBP, which improved thelocal characteristics of LBP, and achieved better effects. In2009, Huang[15] proposed a face description and recognitionalgorithm based on LBP Pyramid feature. Face image Pyramidis constructed by multiscale analysis, and the LBP operator isused to extract the LBP features for each layer. The LBPpyramid of the image has been established. At the end, thestatistics histogram sequence of each layer need to beconnected as the identification feature for face recognition.Xiao [16] proposed a FWTLBP(Fast Wavelet Transform LocalBinary Patterns) algorithm, which was a combination of fastwavelet decomposition algorithm and LBP. This methodrealized multiresolution analysis of image texture and hadeffect on image de-noising. In addition, LBP has a gooddescription for the image texture, and it can promoterecognition rate by merging with other features. Yuan [17]proposed a fusion of LBP and Local Non-negative MatrixFactorization (LNMF) method for face recognition. Theauthors use the LBP operator to extract the LBP histogramsequence (LBPHS) for face image, and then get the histogramsequence weight (Weight LBPHS)to extract the nonnegative

subspace and its coefficient matrix by LNMF method. Theprinciple of the nearest neighbor is used for face classification.

Usually, the LBP operator only describes the relationshipbetween the center pixel and its neighborhood pixels, but itignores the relationship between the operators. 3DLBPoperator embodies this relationship, and it can get a better localdescription. However, the common defect of two methodsneglects the center pixel value,and the center pixel valuereflects some properties of the image points. In this paper, wepropose a novel 4DLBP operator for face recognition, whichadds the pixel value of the center point to the 3DLBP operator.The 4DLBP has strong discriminant ability through describingsome subtle features in detail for the image, such as: bright,dark, the edge and so on. For the face recognition with 4DLBP,we first segment the face into blocks. Then we extract the4DLBP features in each block. Thirdly, we obtain the facefeatures by combining the block features. Finally, we use theELM classifier for face recognition. The experimental resultsverify the effectiveness and correctness of the method.

The paper is organized as follows: section II introducesthe theories of LBP, 3DLBP, 4DLBP and the ELM classifier;section III describes the face recognition method based on4DLBP; Section IV gives the results of the experiment, and thelast section is the conclusion.

II. RELATED KNOWLEDGE

A. LBP(Local Binary Patterns)The core idea of LBP is similar to the traditional local

feature extraction algorithm. It extracts a micro structure in theimage, where these micro structure models construct theimage features [18]. The LBP makes the local binary codeaccording to the difference between the center pixel and theround pixels, which is shown in Fig. 1.

Fig.1 Principle of LBP codeThe computational formula of LBP is [9]:

1

,0

( , ) 2 ( )P

iP R c c i c

i

LBP x y S g g (1)

1, 0( )

0, 0i c

i ci c

g gS g g

g g(2)

Where ( , )c cx y denotes the center pixel c, i denotes theneighbor node of c, cg denotes the image pixel value of c,and ig denotes the image pixel value of i.

The LBP feature extraction method is to make histogramstatistics for LBP coding mode in the image, the featuredimension histogram is determined by the number of modelLBP. In general, the distribution of LBP pixels around thepixel center for circular neighborhood, as shown in Fig. 2, theLBP coding has two important parameters of P and R: P is thenumber of neighbor pixels, R is the radius of the pixels aroundthe neighborhoods. One of the most commonly used (P, R)parameters is (8, 1), and it is widely used in face recognition.

Different P and R represent the image texture features ondifferent scales.

Fig.2 Central pixel and surrounding regionThere are 2P combinations in the LBP (P, R) of 0 and 1

for the LBP algorithm, but the dimension in the image textureand texture classification is high. T. Ojala [19] proposed theconcept of the uniform model, which showed that there is 90%uniform mode in all modes for the natural images with visibleform. Commonly, 2U

PRLBP denotes the uniform LBP operator,and U2 denotes the uniform mode.

B. 3DLBPThe LBP operator describes relationship of the center pixel

and its neighborhood pixels. The LBP operator is shown in theFig. 1. The neighborhood pixel values (f=l, 2...8) are comparedwith the center pixel value, and they are processed with thethreshold based methods. We get a 8 bit binary number bi(i=l,2,..., 8) in a clockwise direction, and then convert it into adecimal number, which is the result of the LBP operatorproduce for the center pixel .

Fig. 3 Schematic diagram of 3DLBP feature acquisition

The normalized face image is normalized to 0 ~ 255, andthe gray scale difference of the adjacent two points which isless than 7 is 93%, so we need three bit binary numbers tocode the gray difference of each point and its neighborhoodpoints. The three bit binary number (i2 i3 i4) correspond to theGray Difference (GD),whose absolute value is 0 ~ 7. When|GD|≥ 7, GD is assigned as number 7. The binary i1 isregarded as the sign of the gray level difference. In this way,GD can be expressed as:

0001

1 GDGD

i（3）

22 3 42 2G D i i i （4）

Four binary numbers can be divided into 4 layers, thebinary units of each layer are clockwise arrangement. Finally,we get 4 groups, and each group have 8 bit binary numbers.Four decimal numbers, which are the 4 groups turn to, areregarded as the pixel representation. So it calls the 3DLBP,and the diagram is shown in Fig.3.

C. Proposed Methods: 4DLBPThe 3DLBP only considers the local structure, which

denotes the relationship between the center pixel and itsneighborhood pixels, it can describe local image texturefeature. However, 3DLBP ignores the pixel value of the centerpoint for local features. We propose the 4DLBP feature, whichadds the pixel value of the center point to the 3DLBP features.The schematic diagram of the 4DLBP features is shown inFig.4.

Fig. 4 Schematic diagram of 4DLBP feature extraction

D. ELM classifierIn recent years, Huang et. al. [20] propose the ELM

learning algorithm for classifier, which is based on a SingleLayer Forward Network (SLFN). For nsamples { , }i iX T , 1 2[ , ,..., ]T m

i i i inX x x x R , 1 2[ , ,..., ]T mi i i imT t t t R ,

a hidden layer has N~ units and excitation function ( )f x inthe ELM. The unified model for SLFN is defined as follow：

NjtbXafXf jij

N

iiiiji

N

ii ,...,2,1,)*()(

~

1

~

1

(5)

Where 1 2[ , ,..., ]Ti i i ina a a a is the input weight with the ithunit of the hidden layer and ib is the deviation. The weightbetween the ith unit of the hidden layer and the output layeris 1 2[ , ,..., ]Ti i i im . ( )f x can be the any of excitation. Theabove equation can be defined as follow:

H T (6)

Because of the learning ability of the standard SLFN, theactual output can be nearly equal to the ideal output, as in thefollowing formula:

10

N

j jj

t y (7)

Therefore, the above formula can be expressed as follow:

H Y (8)

Huang et. al. has proved [16] that the connection weightsand the thresholds will appear as a fixed value when thetraining is stable once the excitation function is an infinitelydifferentiable function. The least squares solution of linearequations as follow:

H Y (9)

Where H the Moore-Penrose is generalized inversematrix of the output layer matrix H.

III. FACE RECOGNITION METHOD BASED ON4DLBP

The general steps of face recognition based on 4DLBP areas follows: First, we preprocess the input images, and wedivide the face image into 16 blocks. Secondly, we extract the4DLBP features for each block, and combine the series offeatures to obtain the feature of the whole face image. Finally,we use the ELM classifier for face image classification.Similarly, the testing images extract the 4DLBP features, andthen these features are fed into the already trained ELM modelfor classification. The specific steps are shown in Fig.5.

Fig. 5 The flowchart of the face recognition based on 4DLBP features.

A. PreprocessingIn order to eliminate the effect of illumination variation to

the face image, the mean and variance normalization methodis used in this paper to preprocess the facial image. Thisprocess consists of two steps which are gray leveltransformation and gray stretching.

(1)Gray level transformation: The f(x, y) is pixel value ofthe image, the average pixel value of the image is:

,

1 ( , )x y

aver f x yN

(10)

Where the total number of pixels are N.

Standard deviation is :

yx

averyxfN ,

2)),((1

(11)

So that gray level transformation of pixel f(x,y) can beexpressed as:

)),((),( averyxfyxf(12)

(2)Gray stretching: when we do gray level transformationfor all pixels in the image, the pixel gray level range havechanged, gray stretching is to ensure the gray level valuesinside 0~255. After the gray-scale transformation, themaximum of all pixel value marked as max, and the minimumvalue marked as min. The gray stretching formula can beexpressed as:

min)(max255*min)),((),( yxfyxf

(13)

Fig.6 shows that the effect of the image is preprocessed,compared with the original image, the image values of thepreprocessed image become uniform (Fig.6).

Fig. 6 The original image and the pre-processed image

B. 4DLBP Feature ExtractionIn the LBP operator for face recognition, each pixel of the

face image is used the LBP operator to get a LBP value, whichresponse image. The response histogram of the image iscalculated as the feature extraction of face images. The faceimage features in the response histogram of the image isextracted by LBP.

Histogram is the first-order of the statistical characters ofthe image, and it can describe the various micro patterns in animage, such as bright spots, dark spots, smooth area, edgefrequency, but it cannot describe the structure information ofthe image. While the difference of local feature image eacharea is larger, if the whole image only generates a LBPhistogram, it will loss local difference information. In order tosolve this problem, the original face image can be divided intoblocks, then calculate the histogram of each LBP operator.

Similarly, the 4DLBP feature extraction method is toperform the 4DLBP operator on each block of uniform size ofthe face image to obtain sixteen 4DLBP feature vectors, thenthe 4DLBP feature vectors are in series to get the 4DLBPfeatures of the face image(Fig.7).

Fig.7 The 4DLBP histogram to extract 4DLBP features

C. ELM ClassificationAfter adopting the 4DLBP feature extraction from the face

images, we use the ELM classifier to identify the face images.The entire process of ELM classification includes two phases:the training phase and the testing phase.

The main steps of the ELM training phase are as follows:

Step 1: 1 2[ , ,..., ]_ face face faceNData Train M M M , N is thenumber of all the training samples.

Step 2: Set the random parameters, the input weightvector 1 2{ , ,..., }nA a a a , and the bias vector 1 2{ , ,..., }nB b b b .

Step 3: 1 21

( ) { , ,..., }.L

j i i j i ki

t f a M b T t t t

Step 4: 1 2, { , ,..., }LH T .

Step5: Calculate },...,,{, 21 nyyyYYH , andY is face image identification.

Step 6: Calculate T , compare T andY .

Step 7: Calculate the recognition rate of the trainingidentification.

The three steps in the testing stage are probably similar tothe training process, we test the model parameters obtainedfrom the training model.

The main steps of ELM testing:

Step 1: Obtain the optimal parameters 'A and 'B , 'B(training phase) and N. N is the number of the hidden layernodes.

Step 2: The test set: 1 2[ , ,.._ ., ]face face facemM MData Tes Mt , Mis the number of the all test samples.

Step3: 1 21

' ' ( ' ' ') ' { ', ',..., '}.L

j i i j i ki

t f a M b T t t t

Step 4: Calculate 'T , compare 'T and 'Y .

Step 5: Calculate the recognition rate of identification test.

IV. EXPERIMENT AND ANALYSIS

A. Face DatabaseThis experiments are adopted on the ORL and Yale face

databases. The ORL face database [21] contains 40 individualswith 10 images per person, which included changes in

expression, pose and scale, and the image size is 92×112. TheYale face database [22] contains 165 pieces of gray images of15 individuals, and each person has 11 images, which havemultiple gestures under various illuminations. The image sizeis 100×100.

B. Experiments parameters settingFirstly, we carry out several experiments on the ORL and

Yale database. For the same individual image in the database,we select N images as the training set, and the rest of theimage from the same person is used as the test set. ThePrincipal Component Analysis (PCA) is used to reducedimension, and PCA coefficient is set with 95% energy.Besides, the feature vector values are normalized to -1 and 1.

C. Discussion of different image blocksWe use 4DLBP to extract the face image features, and the

face image is divided into several small blocks. These blockscontain characteristic information and keep the localcharacteristics. The block size may affect the recognition rate.Because the pixel size of the ORL face image database is 112×92, we use the following 4 different block schemes of 2×2,2×4, 4×2, 4×4 to analyze the impact of block on the finalrecognition accuracy. Fig.8 shows the block size of 4×4 canachieve the optimal performance with the accuracy rate of90%. In the following experiments, we choose the block sizeof 4×4.

Fig.8 Different block sizes and their recognition rate on the same database.

D. Time-consuming of different methodsFor each feature extraction method, we calculate their cost

time in the feature extraction. In this experiments, we have astatistics test about that the average time consuming by theLBP, 3DLBP and 4DLBP on a block image for featureextraction.

We find that LBP spend 0.475 milliseconds for featureextraction, while the average time consuming by 3DLBP is 1.9milliseconds, and that of 4DLBP is 2.2 milliseconds. Thedifference of the average time consuming by 3DLBP and4DLBP is not obvious (Fig.9).

Fig.9 Time-consuming of different methods

E. The Experiment Results on Yale and ORL databasesBased on a 4 × 4 block processing, we compare the

characteristics of different feature extraction methods, whichinclude LBP, 3DLBP and 4DLBP on the ORL database. Inorder to test the feature generalization ability, N=2, 3, 4, 5, 6and 7 images of each person are separately taken as thetraining samples, and the rest images are taken as the testsamples in ORL face database. The experiment results areshown in Table 1.

Table 1 The experimental results of face recognition based on LBP,3DLBP,4DLBP on the ORL database

The number oftraining face N N=2 N=3 N=4 N=5 N=6 N=7

Theaverage

recognition rate

LBP 0.846 0.902 0.941 0.951 0.963 0.975 0.913

3DLBP 0.904 0.931 0.965 0.979 0.991 0.995 0.942

4DLBP 0.918 0.935 0.980 0.987 0.998 1.000 0.958

From the table, we can see that the average recognition rateof LBP is 91.38%, that of 3DLBP is 94.23%, and that of the4DLBP method is 95.84%. These results show that the 4DLBPoutperforms better than LBP and 3DLBP for face recognition.

The Yale face database contains 165 pieces of gray images,which are multiple gestures under various illuminations. Theface image is also divided into 4×4 blocks, and we use LBP,3DLBP, and 4DLBP to conduct the experiments. Theexperimental results are shown in Table 2.

Table 2 The experimental results of face recognition based on LBP,3DLBP,4DLBP on the Yale database

Trainface(N)

N=2 N=3 N=4 N=5 N=6 N=7 N=8

LBP 0.893 0.927 0.950 0.955 0.955 0.986 0.987

3DLBP 0.899 0.935 0.970 0.975 0.962 0.998 0.995

4DLBP 0.914 0.941 0.977 0.973 0.978 1.000 1.000

On the Yale database, it can be seen from the table that therecognition rates based on 4DLBP and 3DLBP are better thanLBP, while that of the 4DLBP is better than 3DLBP, too.

These results indicate that the 4DLBP is superior to LBP and3DLBP for face recognition.

V. CONCLUSIONWe propose a novel method by using the 4DLBP feature

extraction for face recognition to overcome the shortcomingsof the traditional methods such as LBP and 3DLBP. Theprocessing is as below: firstly, we divide the face image tosmall blocks. Then we extract the 4DLBP features for eachblock, and combine the series of features to obtain the featureof the whole face image. Finally, we use the ELM classifierfor face image classification. The 4DLBP has strongdiscriminant ability through describing some subtle features indetail for the image, such as: bright, dark, the edge and so on.The experimental results show that 4DLBP has a higherrecognition rate than LBP and 3DLBP, though its timeconsuming is a little higher. The results verify theeffectiveness and correctness of the method.

Acknowledgements

This work was supported by the National Natural ScienceFoundation of China (Grant No.61063035), Open Fund ofGuangdong Provincial Key Laboratory of PetrochemicalEquipment Fault Diagnosis No.GDUPTKLAB201334, and theyoung academic team construction projects of the 'twelve five'integrated investment planning in Tianjin University ofScience and Technology.

References

[1] Y. Lu, J.C. Yang, S.Q. Wu, Z. Fang. Normalization of Infrared FacialImages under Variant Ambient Temperatures, book chapter, AdvancedBiometrics Technologies, Intech, Austria， August, 2011,ISBN 978-953-307-487-0.

[2] J.C. Yang, Y.B. Jiao, N. Xiong, D.S. Park. Fast Face GenderRecognition by Using Local Ternary Pattern and Extreme LearningMachine, KSII Transactions on Internet and Information Systems, vol.7,no. 7, Jul. 2013.

[3] J.C. Yang, M. Luo, Y. Jiao. Face Recognition Based on Image LatentSemantic Analysis Model and SVM, International Journal of SignalProcessing, Image Processing and Pattern Recognition, Vol. 6, No. 3,June, 2013..

[4] Y. Lu, Z. Xie, Z. Fang, J.C. Yang, S.Q. Wu, F. Li. Time-lapse DataOriented Infrared Face Recognition Method using Block-PCA, TheInternational Conference on Multimedia Technology (ICMT) held fromOct.29-31, 2010 in Ningbo, China.）

[5] H. Han, S. Shan, X. Chen, W. Gao. A comparative study on illuminationpreprocessing in face recognition [J]. Pattern Recognition, 2013,46(6):1691-1699

[6] C.L.Huang, C,W.Chen. human facial feature detection for faceinterpretation and recognition.Pattern Recognition,1992,25(2):1435 一1444.

[7] J. Seo, H. Park. Robust recognition of face with partial variations usinglocal features and statistical learning Original Research Article.Neurocomputing, Volume 129, 10 April 2014, 41-48

[8] L.S Mrby. Application of Karhunen-Loeve Procedure for thecharacterization of Human faces.IEEE Trans.on PAMI,1990,3(l):71 一79

[9] H. Ren, H. Ji, Optik. Nonparametric subspace analysis fused to 2DPCAfor face recognition. International Journal for Light and Electron Optics,Volume 125, Issue 8, April 2014, Pages 1922-1925

[10] X. Luan,B. Fang, L. Liu, W. Yang, J. Qian. Extracting sparse error ofrobust PCA for face recognition in the presence of varying illuminationand occlusion. Pattern Recognition, Volume 47, Issue 2, February 2014,Pages 495-508

[11] V. H. Gaidhane, Y. V. Hote, V. Singh. An efficient approach for facerecognition based on common eigenvalues Original Research Article.Pattern Recognition, Volume 47, Issue 5, May 2014, Pages 1869-1879

[12] T. Ojala, M. Pietikinen, D. Harwood. A comparative study of texturemeasures with classification based on feature distributions [J]. PatternRecognition, 1996, 29(1): 51-59

[13] T. Ojala, M. Pietikainen, T. Maenpaa. Multiresolution gray scale androtation invariant texture classification with local binary patterns[J].IEEE Trans. on Pattern Analysis and Machine Intelligence, 2002,24(7):971-987.

[14] Y. Huang, Y. Wang, T. Tan. Combining statistics of geometrical andcorrelative feature for 3d face recognition[c]//bmvc.edinburgh,2006:879-888

[15] W. W. Huang, F. Li, J. F. H. Liang. Face Description and Recognitionby LBP Pyramid. journal of computer-aided design & computer graphicsvol.21, No.1 Jan., 2009

[16] X. Xiao. The Research and Application of Image Feature ExtractionBased LBP Methodology. Jilin University 2011

[17] B. Yuan, H. Wang, R. M. Wu. Fusing local binary pattern and LNMF offace recognition. Computer Engineering and Applications, 2013, 49（5）：166-169.

[18] Z. Xie, G. Liu. Infrared face recognition based on intensity of localmicropattern–weighted local binary pattern [J]. Optical Engineering,2011, 50(7): 077201-1-6.

[19] T. Ahonen, A. Hadid, M. Pietikainen. Face recognition with local binarypatterns, ECCV 2004 Proceedings on Computer Vision, Lecture Notesin Computer Sciences 3021 (2004) 469–481.

[20] G. B. Huang, Q. Y. Zhu, C.K. Siew. Extreme Learning machine: Theoryand Applications. Neurocomputing. Vol 70(1-3), pp: 489-501. 2006.

[21] The ORL Database of Faces, AT & T (Olivetti) Research Laboratories,Cambridge,U.K.,[Online].Available:(http://www.cl.cam.ac.uk/Research/DTG/attarchive:pub/data/attfaces). tar.z.

[22] The Yale Face Database B, [Online]. Available(http://cvc.yallefacesB.html).