Expert Systems with Applications 41 (2014) 8016–8026
Contents lists available at ScienceDirect
Expert Systems with Applications
journal homepage: www.elsevier .com/locate /eswa
Expert image retrieval system using directional local motif XoR patterns
http://dx.doi.org/10.1016/j.eswa.2014.07.0010957-4174/� 2014 Elsevier Ltd. All rights reserved.
⇑ Corresponding author at: Malaviya National Institute of Technology, Jaipur302017, Rajasthan, India.
E-mail addresses: [email protected] (S.K. Vipparthi), [email protected] (S.K. Nagar).
Santosh Kumar Vipparthi a,b,⇑, S.K. Nagar a
a Indian Institute of Technology Banaras Hindu University, Varanasi 221005, Uttar Pradesh, Indiab Malaviya National Institute of Technology, Jaipur 302017, Rajasthan, India
a r t i c l e i n f o
Article history:Available online 10 July 2014
Keywords:Image retrieval (IR)Local binary patterns (LBP)XoR patternsMotif matrixDatabase
a b s t r a c t
This paper presents a new image feature descriptor, namely directional local motif XoR patterns(DLMXoRPs) for image retrieval application. The proposed motif representation is entirely different fromexisting motif. The DLMXoRP presents a novel technique for the calculation of motif using 1 � 3 grids. Theproposed motif (1 � 3) representation is having a flexible structure; hence it can able to extract all direc-tional information. This flexibility is not present in the existing (2 � 2) motif. Further, the XoR operationis performed on the transformed new motif images which are not present in the literature (local binarypatterns (LBP) and motif co-occurrence matrix (MCM)). To elevate the benefits of DLMXoRP, we compareit with the Motif XoR pattern (MXoR) which is calculated by applying the XoR operation on existingtransformed motif image. The performance of the proposed method is tested by conducting three exper-iments on Corel-5000, Corel-10000 and MIT-VisTex databases. The results after investigation show a sig-nificant improvement in terms of average retrieval precision (ARP) and average retrieval rate (ARR) ascompared to the state-of-the-art techniques for image retrieval.
� 2014 Elsevier Ltd. All rights reserved.
1. Introduction
Content based image retrieval (CBIR) also known as contentbased visual information retrieval (CBVIR) has become a dynamicresearch area since last decade and has been paid ancillary atten-tion in recent years as a consequence of the vivid and rapid risein the volume of digital media. Digital media has become mostimportant tool for human communication and it provides con-tented information to the world. So digital data has gained widerange of applications in the field of medical, scientific, educational,entertainment, and many other which widened the scope of CBIR.The image retrieval system is an efficient system for searching andretrieving images from a large volume of database. CBIR utilizesthe content (color, texture, shape, etc.) of an image rather thanmetadata such as tags and keywords for retrieving images fromimmense database. Most of the web based search engines merelyimpart on metadata and this causes a lot of junk in the result. Thereis no single best representation of an image for all perceptionalsubjectivity, because the user may take photos in different condi-tions. Widespread literature survey on CBIR is accessible inChuen-Horng, Chun-Chieh, Hsin-Lun, and Jan-Ray (2014),
Fatahiyah, Rossitza, and Yu-kun (2013), Ja-Hwung, Yu-Ting, Hsin-Ho, and Vincent (2010) and Pei-Cheng, Been-Chian, Hao-Ren, andWei-Pang (2008).
The color concept of an image is one of the most significant fea-ture in the field of CBIR, if it is maintained semantically intact. Inaddition, color structure in visual scenery changes in size, resolu-tion and orientation. Many approaches are accessible to extractsubstantial features such as, texture, color, shape and integrationof two or more such features. Swain and Ballard (1990) proposedthe concept of color histogram and also introduced the histogramintersection distance metric to measure the distance between thehistograms of images. Pass, Zabih, and Miller (1997) have intro-duced the concept of color coherence vector (CCV). The CCV parti-tions the each histogram bin into two types i.e. coherent if itbelongs to large consistently colored region or incoherent, if it doesnot. Stricker and Orengo (1995) have used the first three centralmoments called mean, standard deviation and skewness of eachcolor space for image retrieval. Huang, Kumar, and Mitra (1997)have proposed the color correlogram which characterizes not onlythe color distributions of pixels, but also the spatial correlationbetween pair of colors. Lu and Burkhardt (2005) proposed the colorfeature based on vector quantized (VQ) index histograms in thediscrete cosine transform (DCT) domain.
Texture is another significant characteristic of an image. Textureanalysis has been extensively used in medicine, industrial, amuse-ment, computer vision, gesture recognition and other image
S.K. Vipparthi, S.K. Nagar / Expert Systems with Applications 41 (2014) 8016–8026 8017
processing applications due to its potential value. Various algo-rithms has been developed for texture analysis, such as the meanand variance of the wavelet coefficients used as texture featureswhich is developed by Smith and Chang (1996). Turgay and Tardi(2011) have proposed the multi scale texture classification usingthe magnitude and phase features for complex wavelet responses.Subrahmanyam, Maheshwari, and Balasubramanian (2012a) haveintroduced the wavelet and color vocabulary trees for textureretrieval.
In addition to color and texture, the shape features are also usedin image retrieval field. Usually the shape features are collectedafter the images have been segmented into regions or objects.Persoon and Fu (1977) have proposed the shape description ofimages using Fourier descriptors. Rui, She, and Huang (1996) pro-posed the modified Fourier descriptors to make it robust to thenoise and invariant to the geometric transformations.
In recent years, local image feature extraction has gained moreattention in the field of CBIR. The local feature descriptor uses thevisual features of regions or objects to describe the image. A visualcontent descriptor can either be local or global. A global descriptoruses the visual features of the whole image, while a local descrip-tor uses the visual features of regions or objects to describe theimage. Many local descriptors are reported in the literature suchas; Ojala, Matti, and Harwood (1996) have proposed local binarypatterns which can show a better performance as well as less com-putational complexity for texture classification. LBP is succeededin terms of performance and speed, this is reported in manyresearch areas like texture classification (Ojala et al., 1996; Guo,Zhang, & Zhang, 2010), face recognition (Ahonen, Hadid, &Pietikainen, 2006), image retrieval (Cheng-Hao & Shu-Yuan,2003; Subrahmanyam, Maheshwari, & Balasubramanian, 2012b,2012c, 2012d, 2012e; Valtteri, Timo, & Matti, 2005; Vipparthi &Nagar, in press, 2014). Likewise, Vipparthi and Nagar (in press)have proposed the directional local ternary pattern based on direc-tional edges for image retrieval. Further, Vipparthi and Nagar(2014) have proposed multi-joint histogram based modelling forimage retrieval application. The different LBP variant features forimage retrieval, texture retrieval and object tracking applicationshave been proposed in Subrahmanyam et al. (2012b, 2012c,2012d, 2012e). Valtteri et al. (2005) have proposed the block-based LBP feature for CBIR. Cheng-Hao and Shu-Yuan (2003)derived two types of local edge pattern (LEP) histograms, one islocal edge patterns for segmentation (LEPSEG), and the other, localedge patterns with invariance (LEPINV) for image retrieval. Marko,Matti, and Cordelia (2009) proposed the center-symmetric localbinary pattern (CS-LBP) which integrates the concept of LBP andscale invariant feature transform (SIFT) for description of interestregion.
Shufu, Shiguang, Xilin, and Jie (2010) have proposed the localGabor XoR patterns (LGXP) operator for face recognition.Jhanwar, Chaudhuri, Seetharamanc, and Zavidovique (2004) pro-posed the motif co-occurrence matrix (MCM) for CBIR. The color-MCM features also collected by applying the MCM on individualred (R), green (G), and blue (B) color channels. Lin, Chen, andChan (2009) proposed the integration of three features for CBIR.Further, it is mentioned that these three features are MCM, differ-ence between pixels of scan pattern (DBPSP) and color histogramfor K-mean (CHKM). In this paper, we propose a new featuredescriptor, directional local motif XoR pattern (DLMXoRP) which uti-lizes the concepts of LGXP (Shufu et al., 2010) and MCM (Jhanwaret al., 2004) for image retrieval. The main contributions and high-lights of this paper are given below.
1. The existing motif operation in Jhanwar et al. (2004) has usedfour pixels as one grid (i.e. 2 � 2 grid). Whereas our proposedmotif uses only three pixels as one grid (i.e. 1 � 3 grid).
2. The existing motif demands six comparisons to encode the2 � 2 grid whereas our method requires only three comparisonsfor 1 � 3 grid.
3. The existing motif is not able to provide the specific directionalinformation due to its grid structure whereas our proposedmotif is able to collect four directional information.
4. We analyze the performance of individual and combined fourdirectional features of our proposed motif.
The organization of the paper is as follows: in Section 1, a briefreview of CBIR and related work are given. Section 2 presents aconcise review of feature extraction strategies. The proposed imageretrieval system framework is given in Section 3. Experimentalresults of various methods are analyzed in Section 4 and finallyin Section 5, we conclude the summary of work.
2. Feature extraction methods
2.1. Local binary patterns (LBP)
LBP (Ojala et al., 1996) was derived from the general definitionof texture in a local neighborhood. This method shows success interms of performance and speed, and is reported in many researchareas such as texture classification, face recognition, image retrie-val, etc.
Given a center pixel in the 3 � 3 pattern, LBP value is computedby comparing its gray scale value with its surrounding neighbors asfollows:
LBPN;R ¼XN�1
n¼0
2n � f1ðgn � gcÞ ð1Þ
f1ðxÞ ¼1 x P 00 x < 0
�ð2Þ
where N is the number of neighbors, R is the radius of the neighbors,gc denotes the gray value of the center pixel and gn is the gray valueof its neighbors.
2.2. Motif co-occurrence matrix
Jhanwar et al. (2004) have proposed the motif co-occurrencematrix (MCM) for CBIR. The MCM is designed from the transformedmotif image which is computed by segmenting the whole imageinto non-overlapping 2 � 2 grid (pixel patterns). In our work, weused seven scan motif patterns with a size of 2 � 2. Each patternis replaced by a scan motif number as shown in Fig. 1. Further, thismethod has been converted into translation invariant by applyingmotif operation on overlapped 2 � 2 pixel patterns. Finally, thetransformed motif image Mi(x,y) formed and it minimizes the localgradient. More details on MCM can be found in the literature(Jhanwar et al., 2004; Lin et al., 2009).
3. Proposed image retrieval system
3.1. Motif XoR patterns (MXoR)
In order to analyze the performance of proposed method, wealso incorporated XoR operation on the existing motif as follows.The MXoR is derived by applying the XoR operator on a trans-formed motif image. For a given 3 � 3 pattern, MXoR value is com-puted by comparing the center motif value with its neighboringmotif values as follows:
MXoRN;R ¼XN�1
i¼0
MImðpi;pcÞ � 2i ð3Þ
P Q
R S
P PP P
Fig. 1. Different types of scanned motif using 2 � 2 grids.
Fig. 2. Construction of 1 � 3 sub-grids from a given 3 � 3 grid, (a) horizontal direction, (b) vertical direction and (c) and (d) diagonal directions.
Fig. 3. Proposed motif calculation in 0� direction for a given 1 � 3 sub-grid.
8018 S.K. Vipparthi, S.K. Nagar / Expert Systems with Applications 41 (2014) 8016–8026
MImðpi;pcÞ ¼1 pi – pc
0 pi ¼ pc
�ð4Þ
where N is the number of neighbors (N = 8) and R is the radius of theneighbor (R = 1), pc denotes motif value of a center pixel in a givenmotif image; pi is the motif value of neighbor.
After computing the motif XoR pattern (MXoR) for each pixel(i, j), the whole image is represented by building a histogram usingEq. (5).
HMXoRPðlÞ ¼XN1
i¼1
XN2
j¼1
f2ðMXoRði; jÞ; lÞ; l 2 ½0; ð2N � 1Þ� ð5Þ
f2ðx; yÞ ¼1 x ¼ y
0 else
�ð6Þ
where N1 � N2 represents the size of input image and N is numberof neighbors.
3.2. Proposed directional local motifs
The image is divided into an overlapped 3 � 3 grid. From eachgrid, four 1 � 3 sub-grids are retrieved in 0�, 45�, 90� and 135�directions as shown in Fig. 2. All four sub-grids are constructed
based on the center pixel of 3 � 3 grid (see in Fig. 2). Further, thedirectional motifs are derived from four sub-grids as shown inFig. 3. The detailed explanation of the 0� direction motif calculationis given in Section 3.3.
3.3. Analysis
Here our main aim is to determine a new transformed motifnumber based on the pixel behavior. The detailed explanation for0� direction motif calculation is given as follows.
Let a, b and c are the gray values of 1 � 3 sub-grid in 0� direction(see in Fig. 3).
1. For motif ‘‘1’’: when a = 1, b = 2 and C = 3. Here, ‘b’ is the centerpixel value in 1 � 3 pattern. If it satisfies the condition of ‘b > a’and ‘b < C’ then the 1 � 3 pattern is replaced by a new motifvalue ‘1’. Similarly, motif ‘‘2’’ is coded when ‘A > b’ and ‘b > c’.
2. For motif ‘‘3’’: when ‘A’ = 4, ‘b’ = 3 and ‘C’ = 5. Center pixel valueis smaller than the neighbours and ‘A < C’. Then, the new motifvalue with ‘‘3’’ is coded. Similarly, motif ‘‘4’’ is coded when‘A > C’.
3. For motif ‘‘5’’: as if ‘a’ = 3, ‘b’ = 5 and ‘c’ = 4. Center pixel value islarger than the neighbours and ‘a < c’. Then, new motif valuewith ‘‘5’’ is coded. Similarly, motif ‘‘6’’ is coded when ‘a > c’.
S.K. Vipparthi, S.K. Nagar / Expert Systems with Applications 41 (2014) 8016–8026 8019
4. For motif ‘‘7’’: if all three pixels bearing same gray scale values,then, new motif value with ‘‘7’’ is coded.
In this paper following notations are used for better under-standing. Smaller and larger value is defined by using ‘<’ and ‘>’symbols. Whereas ‘�’ and ‘�’ symbols are referred to smaller ofsmallest and larger of largest values respectively among ‘a’ and‘c’ pixels. If ‘a’ value is larger of largest value then superscript of‘A’ and ‘�’ symbol are used. Similarly for smaller of smallest value‘�’ and subscript of ‘c’ is used and vice versa in Fig. 3.
ThImðf ; a; bÞ ¼
1; ðða P f Þ & ða 6 bÞÞ2; ðða 6 f Þ & ða P bÞÞ3; ðða 6 f Þ & ða 6 bÞ & ðf 6 bÞÞ4; ðða 6 f Þ & ða 6 bÞ & ðb 6 f ÞÞ5; ðða P f Þ & ða P bÞ & ðf 6 bÞÞ6; ðða P f Þ & ða P bÞ & ðb 6 f ÞÞ7; ðða ¼ f Þ&ða ¼ bÞ&ðb ¼ f ÞÞ
8>>>>>>>>>>><>>>>>>>>>>>:
�����������������jh¼0�
ð7Þ
ThImðc; a; gÞ ¼
1; ðða P cÞ & ða 6 gÞÞ2; ðða 6 cÞ & ða P gÞÞ3; ðða 6 cÞ & ða 6 gÞ & ðc 6 gÞÞ4; ðða 6 cÞ & ða 6 gÞ & ðg 6 cÞÞ5; ðða P cÞ & ða P gÞ & ðc 6 gÞÞ6; ðða P cÞ & ða P gÞ & ðg 6 cÞÞ7; ðða ¼ cÞ&ða ¼ gÞ&ðg ¼ cÞÞ
8>>>>>>>>>>><>>>>>>>>>>>:
�����������������jh¼45�
ð8Þ
ThImðd; a;hÞ ¼
1; ðða P dÞ & ða 6 hÞÞ2; ðða 6 dÞ & ða P hÞÞ3; ðða 6 dÞ & ða 6 hÞ & ðd 6 hÞÞ4; ðða 6 dÞ & ða 6 hÞ & ðh 6 dÞÞ5; ðða P dÞ & ða P hÞ & ðd 6 hÞÞ6; ðða P dÞ & ða P hÞ & ðh 6 dÞÞ7; ðða ¼ dÞ & ða ¼ hÞ & ðh ¼ dÞÞ
8>>>>>>>>>>><>>>>>>>>>>>:
�����������������jh¼90�
ð9Þ
ThImðe; a; iÞ ¼
1; ðða P eÞ & ða 6 iÞÞ2; ðða 6 eÞ & ða P iÞÞ3; ðða 6 eÞ & ða 6 iÞ & ðe 6 iÞÞ4; ðða 6 eÞ & ða 6 iÞ & ði 6 eÞÞ5; ðða P eÞ & ða P iÞ & ðe 6 iÞÞ6; ðða P eÞ & ða P iÞ & ði 6 eÞÞ7; ðða ¼ eÞ&ða ¼ iÞ&ði ¼ eÞÞ
8>>>>>>>>>>><>>>>>>>>>>>:
�����������������jh¼135�
ð10Þ
Finally, the whole image is converted into four directional trans-formed motif images.
3.4. Directional local motif XoR patterns (DLMXoRP)
Local motif XoR operator is derived by applying XoR operationon individual directional transformed motif images. For, a given3 � 3 pattern, DLMXoRP value is computed by comparing the cen-ter motif value with its surrounding neighbor motif values asfollows:
DLMXoRPhN;R ¼
XN�1
i¼0
ThImðpi;pcÞ � 2i
�����h¼0� ;45� ;90� ;135�
ð11Þ
ThImðpi; pcÞ ¼
1 pi – pc
0 pi ¼ pc
� ����h¼0� ;45� ;90� ;135�
ð12Þ
where N is the number of neighbors, R is the radius of the neighbors,pc denotes center motif value, pi is the motif value of neighbor and hdenotes direction of motif image.
After computing the DLMXoRP, the whole image is representedby building a histogram as follows:
HhDLMXoRPðlÞ ¼
XN1
i¼1
XN2
j¼1
f2ðDLMXoRPhði; jÞ; lÞ; l 2 ½0; ð2N � 1Þ������h¼0� ;45� ;90� ;135�
ð13Þ
where N1 � N2 is the size of input image.Finally, the feature vector for a given image is constructed by
concatenating four directional histograms as given in Eq. (14).
HDLMXoRP ¼ H0�DLMXoRP; H45�
DLMXoRP; H90�DLMXoRP; H135�
DLMXoRP
h ið14Þ
3.5. Proposed system framework
Fig. 4 illustrates the flowchart of proposed feature extractionmethod and algorithm for the same is given bellow:
Algorithm
Input: Image; Output: Retrieval results1. Load the image and convert into a gray scale (if it is RGB).2. Divide the image into 3 � 3 grids.3. Collect four 1 � 3 sub-grids from each 3 � 3 grid.4. Calculate the motif values (Fig. 4(a)).5. Collect the four motif images in 0�, 45�, 90� and 135�directions.6. Apply XoR operation on four motif images to formDLMXoRPs (Fig. 4(b)).7. Construct the histograms for 0�, 45�, 90� and 135�directional DLMXoRPs.8. Construct a feature vector by concatenating thehistograms (Fig. 4(c)).9. Compare the query feature with features in the databaseusing Eq. (14) (Fig. 4 (d)).10. Retrieve the images based on the best matches(Fig. 4(e)).
3.6. Similarity measures
In this paper, four types of similarity distance measures are usedas given below:
Manhattan or L1 or city� block Distance : DsðQ m; TmÞ
¼XLg
i¼1
j f Tm ;i � fQm ;ij ð15Þ
Euclidean or L2 Distance : DsðQm; TmÞ ¼XLg
i¼1
jfTm ;i � fQm ;ij2
!1=2
ð16Þ
d1 Distance : DsðQ m; TmÞ ¼XLg
i¼1
fTm ;i � fQm ;i
1þ fTm ;i þ fQm ;i
�������� ð17Þ
Canberra Distance : DsðQ m; TmÞ ¼XLg
i¼1
jfTm ;i � fQm ;ijjfTm ;i þ fQm ;ij
ð18Þ
where Qm is query image, Lg is feature vector length, Tm is image indatabase; fTm ;i is ith feature of image Tm in the database, fQm ;i is ithfeature of query image Qm.
DLMXoRP Feature Database
Similarity Measurement
5 7 6 4 2
2 1 3 6 5
5 3 1 2 7
3 5 2 1 5
3 5 1 1 2
Motif calculation using 1x3sub-grid in 0 degree direction
127 223 248
222 120 224
184 254 152
XOR patterns calculations
Feature Vector
Result
DLMXoRP
Database Images
Query Image
a
b
c
c
d
e
Fig. 4. Proposed image retrieval system framework.
8020 S.K. Vipparthi, S.K. Nagar / Expert Systems with Applications 41 (2014) 8016–8026
4. Experimental results and discussions
In this paper, the proposed method is evaluated on three bench-mark databases followed by brief description about experimentalconditions. Corel- database (http://wang.ist.psu.edu/docs/rela-ted.shtml) comprises thousands of images of various contents rang-ing from animals, and outdoor sports to natural images. By domainprofessionals this database is pre-classified into different categoriesof size 100. Because of its size and heterogeneous content Corel-database meets all the requirements to evaluate an image retrievalsystem. In this paper, we use the Corel-5000 and Corel-10000 dat-abases which consist of 50 and 100 different categories respectively.Each category has NG (=100) images with resolution of either126 � 187 or 187 � 126.
The retrieval performance of the proposed method is measuredin terms of precision (P), average retrieval precision (ARP), recall(R) and average retrieval rate (ARR) are shown below.
For the query image Iq the precision (P) and Recall (R) aredefined as follows.
Precision : PðIqÞ ¼Number of Relevant Images Retrieved
Total Number of Images Retrievedð19Þ
ARP ¼ 1DB
XDB
i¼1
PðIiÞ�����n610
ð20Þ
Recall : RðIqÞ ¼Number of Relevant Images Retrieved
Total Number of Relevant Images in the Databaseð21Þ
ARR ¼ 1DB
XDB
i¼1
RðIiÞ�����nP10
ð22Þ
where DB is the total number of images in the database.
Table 1Performance of various methods in terms of ARP and ARR on Corel-5000 and Corel-10000
Database Performance (%) Methods
CS-LBP BLK-LBP
Corel-5000 ARP 32.96 45.75ARR 13.99 20.29
Corel-10000 ARP 26.43 38.13ARR 10.15 15.34
4.1. Experiment 1
In this experiment, the performance of the proposed method(DLMXoRP) is tested on Corel-5000 database in terms of ARP andARR. Table 1 exemplifies the retrieval performance of DLMXoRPand other existing techniques (LBP, CS-LBP, BLK-LBP, and MXoR)on Corel-5000 and Corel-10000 databases in terms of ARP andARR. Fig. 5(a) and (b) illustrate the category wise performance ofvarious methods in terms of precision and recall. Fig. 5(c) and (d)illustrate the performance comparison between the proposedmethod and other existing methods in terms of ARP and ARRrespectively on Corel-5000 database. We also analyze the individ-ual performance of directional features of DLMXoRP as shown inTable 2. From Table 2, it is clear that the DLMXoRP with all direc-tional information outperforms the individual directional featuresof DLMXoRP and MXoR in terms of ARP and ARR on Corel-5000database. Table 3 illustrates the performance of DLMXoRP usingvarious distance measures in terms of ARP and ARR on Corel-5000 database. From Table 3, it is observed that d1 distance mea-sure shows a better performance as compared to other existing dis-tance measures. From Tables 1–3 and Fig. 5, it is clear that theproposed method outperforms the other existing methods onCorel-5000 database. The retrieval performance of the proposedmethod and other existing techniques are verified with precisionversus recall curve as shown in Fig. 6. From the Fig. 6 it is clear that,the proposed method show a significant improvement as com-pared to other techniques on Corel-5000 database. The queryresults of the proposed method on Corel-5000 database is (top leftimage is query image) shown in Fig. 7.
4.2. Experiment 2
In this experiment, Corel-10000 database is used to evaluateour proposed method. Table 4 illustrates the retrieval performance
databases.
LEPINV LBP MXoR DLMXoRP
35.19 43.62 42.40 49.4714.84 19.22 18.97 21.76
28.93 37.62 34.954 40.93 34.95 40.9311.22 14.97 14.12 16.27
0 10 20 30 40 500
102030405060708090
100
Prec
isio
n (%
)
Category
CS_LBP LEPINV BLK_LBP LBP MXOR DLMXoRP
0 10 20 30 40 5005
10152025303540455055
Rec
all (
%)
Category
CS_LBP LEPINV BLK_LBP LBP MXOR DLMXoRP
0 10 20 30 40 50 60 70 80 90 10010
15
20
25
30
35
40
45
50
AR
P (%
)
No of Top Matching Images
CS_LBP LEPINV BLK_LBP LBP MXOR DLMXoRP
0 10 20 30 40 50 60 70 80 90 1002468
10121416182022
AR
R (%
)
No of Top Matching Images
CS_LBP LEPINV BLK_LBP LBP MXOR DLMXoRP
Fig. 5. Comparison of proposed method with other existing methods in terms of: (a) category wise precision, (b) category wise recall, (c) ARP and (d) ARR on Corel-5000database.
Table 2Results of proposed method with individual and combined directional features in terms of ARP and ARR on Corel-5000 database.
Database Performance (%) Methods
MXoR DLMXoRPs
0� 45� 90� 135� PM
Corel-5000 ARP 42.40 41.91 40.65 42.13 42.93 49.47ARR 18.98 18.71 18.14 18.69 18.94 21.76
PM: proposed method with all directional motif information.
S.K. Vipparthi, S.K. Nagar / Expert Systems with Applications 41 (2014) 8016–8026 8021
of MXoR and DLMXoRP with individual and combined directionalfeatures on Core-10000 database. From Table 4, it is observed thatthe DLMXoRP with all directional information shows around 6%improvement in terms of ARP as compared to MXoR and DLMXoRPwith individual directional information. Table 5 illustrates the per-formance of the proposed method with different distance mea-sures on Corel-10000 database. From Table 5, it is observed that
the distance measure (d1) shows better performance as comparedto the other existing distance measures. Fig. 8(a) and (b) illustratesthe category wise performance of the proposed method in terms ofprecision and recall. Fig. 8(c) and (d) exemplifies the performanceof various methods on Core-10000 database in terms of ARP andARR respectively. From the Tables 1, 4, 5 and Fig. 8, it is clear thatthe proposed method outperforms the other existing methods on
Table 3The performance of DLMXoRP with various distance measures in terms of ARP andARR on Corel-5000 database.
Performance Distance measure
L1 Canberra L2 d1
ARP (%) 45.092 49.416 38.544 49.468ARR (%) 19.627 21.768 16.6134 21.76
Fig. 6. Precision Vs Recall curve for the proposed method and other existingtechniques on Corel-5000 database.
Table 4Results of proposed method with individual and combined directional features interms of ARP and ARR on Corel-10000 database.
Database Performance(%)
Methods
MXoR DLMXoRPs
0� 45� 90� 135� PM
Corel-10000
ARP 34.95 33.75 33.28 34.08 34.59 40.93ARR 14.12 13.56 13.41 13.69 13.61 16.27
PM: Proposed method with all directional motif information.
Table 5The performance of DLMXoRP with various distance measures in terms of ARP andARR on Corel-10000 database.
Performance Distance measure
L1 Canberra L2 d1
ARP (%) 37.15 40.69 31.28 40.93ARR (%) 14.52 16.25 11.96 16.27
8022 S.K. Vipparthi, S.K. Nagar / Expert Systems with Applications 41 (2014) 8016–8026
Corel-10000 database in terms of their evaluation measures. FromFig. 5, it is observed that the proposed method shows less retrievalperformance on categories 2, 11, 14, 17, 20, 27, 37, 39, 46 and 50 ascompared to the other existing methods. Similarly, From Fig. 8, it isobserved that, 53, 58, 60, 63, 89 and 100 as compared to the otherexisting methods. The reason behind this is, these categories hav-ing less discriminative information. However, the overall (average)
Fig. 7. Two query results of the proposed method (top lef
performance of the proposed method shows a significant improve-ment as compared to the existing methods in terms of precision,recall, average and average retrieval rate on Corel-5000 andCorel-10000 databases. The retrieval performance of the proposedmethod and other existing techniques are verified with precisionversus recall curve as shown in Fig. 9. From Fig. 9 it is clear thatthe proposed method show a significant improvement as com-pared to other techniques on Corel-10000 database. Fig. 10 showsthe query results of the proposed method on Corel-10000database.
t image is the query image) on Corel-5000 database.
0 10 20 30 40 50 60 70 80 90 100
102030405060708090
Prec
isio
n (%
)
Category
CS_LBP LEPINV BLK_LBP LBP MXOR DLMXoRP
0 10 20 30 40 50 60 70 80 90 1000
10
20
30
40
50
60
70
Rec
all (
%)
Category
CS_LBP LEPINV BLK_LBP LBP MXOR DLMXoRP
0 10 20 30 40 50 60 70 80 90 1008
1012141618202224262830323436384042
AR
P (%
)
No of Top Matching Images
CS_LBP LEPINV BLK_LBP LBP MXOR DLMXoRP
0 10 20 30 40 50 60 70 80 90 10023456789
1011121314151617
AR
R (%
)
No of Top Matching Images
CS_LBP LEPINV BLK_LBP LBP MXOR DLMXoRP
Fig. 8. Comparison of proposed method with other existing methods in terms of: (a) category wise precision, (b) category wise recall, (c) ARP and (d) ARR on Corel-10000database.
Fig. 9. Precision Vs recall curve for proposed method and other existing techniqueson Corel-10000 database.
S.K. Vipparthi, S.K. Nagar / Expert Systems with Applications 41 (2014) 8016–8026 8023
4.3. Experiment 3
In this experiment, we use the MIT-VisTex database (http://vis-mod.media.mit.edu/pub/) which consists of 40 different textureswith size 512 � 512. These textures are divided into sixteen128 � 128 non-overlapping sub-images, thus creating a databaseof 640 (40 � 16) images. The performance of the proposed methodis measured in terms of average retrieval rate (ARR). Fig. 11 illus-trates the retrieval results of the proposed method and variousexisting methods on MIT-VisTex database in terms of ARR. FromFig. 11, it is observed that the proposed DLMXoRP shows betterperformance than the state-of-the-art features for image retrievalon MIT-VisTex database. Further, the performance of the proposedmethod is analyzed with different directional information asshown in Fig. 12. From Fig. 12, it observed that the 0� + 45� +90� + 135� directional information shows better performance than
Fig. 10. Two query results of the proposed method (top left image is the query image) on Corel-10000 database.
8 16 24 32 40 48 56 64 72 80 88 96 104 11270
75
80
85
90
95
100
AR
R (%
)
No of Top Matching Images
CS-LBP LEPINV BLK-LBP LBP MXOR PM
Fig. 11. Comparison of proposed method with other existing methods in terms ofARR MIT-VisTex database.
83
85
87
89
91
93
95
97
99
16 32 48 6
AR
R (%
)
Number of
0 De90 D0+4590+10+130+450+900+45
Fig. 12. Retrieval performance of the DLMXoRP with the po
8024 S.K. Vipparthi, S.K. Nagar / Expert Systems with Applications 41 (2014) 8016–8026
other directional information. Fig. 13 shows the query results ofthe proposed method on MIT-VisTex database.
5. Conclusions
In image retrieval, the feature extraction and similarity mea-surement are two major factors which determining the accuracyrate of the retrieval system. In this paper, we mainly focused todesign/ develop an expert feature extraction approach using direc-tional local motif XoR patterns (DLMXoRP).
The proposed scanned motif using 1 � 3 grids and its study iscompletely different from existing motif using 2 � 2 grids. In imageanalysis, extracting specific directional information is an importanttask. Thus, the existing motif representation fails to extract specificdirectional information from an image due to its grid alignmentwhereas; our proposed motif is able to collect four directional
4 80 96 112 top matches
gree 45 Degreeegree 135 Degree Degrees 45+90 Degrees35 Degrees 0+90 Degrees5 Degrees 45+135 Degrees+90 Degrees 0+45+135 Degrees+135 Degrees 45+90+135 Degrees+90+135 Degrees
ssible directional information on MIT-VisTex database.
Fig. 13. Two query results of the proposed method (top left image is the query image) on MIT-VisTex.
S.K. Vipparthi, S.K. Nagar / Expert Systems with Applications 41 (2014) 8016–8026 8025
information. The existing motif demands six comparisons toencode the 2 � 2 grid whereas our proposed method requires onlythree comparisons for 1 � 3 grid. Further, the XoR operation wasperformed on transformed motif images.
The retrieval performance of the proposed descriptor is testedon Corel-5000, Corel-10000 and MIT-VisTex databases. The retrie-val performance of DLMXoRP shows 7% and 6% improvement interms of ARP as compared to MXoR on Corel-5000 and Corel-10000 databases respectively. An improvement of 9.16% in termsof ARR as compared to MXoR on MIT-VisTex database wasachieved.
References
Ahonen, T., Hadid, A., & Pietikainen, M. (2006). Face description with local binarypatterns: Applications to face recognition. IEEE Transactions on Pattern Analysis,28(12), 2037–2041.
Cheng-Hao, Y., & Shu-Yuan, C. (2003). Retrieval of translated, rotated and scaledcolor textures. Pattern Recognition, 36, 913–929.
Chuen-Horng, L., Chun-Chieh, C., Hsin-Lun & Jan-Ray, L. (2014). Fast K-meansalgorithm based on a level histogram for image retrieval. Expert Systems withApplications, 41, 3276–3283.
Fatahiyah, M. A., Rossitza, S., & Yu-kun, L. (2013). Trademark image retrieval usingan integrated shape descriptor. Expert Systems with Applications, 40, 105–121.
Guo, Z., Zhang, L., & Zhang, D. (2010). Rotation invariant texture classification usingLBP variance with global matching. Pattern Recognition, 43(3), 706–719.
Huang, J., Kumar, S. R., & Mitra, M. (1997). Combining supervised learning withcolor correlograms for content based image retrieval. In Fifth ACM multimediaconf. (pp. 325–334).
Ja-Hwung, S., Yu-Ting, H., Hsin-Ho, Y., & Vincent, S. T. (2010). Effective content-based video retrieval using pattern-indexing and matching technique. ExpertSystems with Applications, 37, 5068–5085.
Jhanwar, N., Chaudhuri, S., Seetharamanc, G., & Zavidovique, B. (2004). Contentbased image retrieval using motif co-occurrence matrix. Image and VisionComputing, 22, 1211–1220.
Lin, C. H., Chen, R. T., & Chan, Y. K. A. (2009). Smart content-based image retrievalsystem based on color and texture feature. Image and Vision Computing, 27,658–665.
Lu, Z. M., & Burkhardt, H. (2005). Color image retrieval based on DCT-domain vectorquantization index histograms. IEEE Electronics Letters, 41(17), 956–957.
Marko, H., Matti, P., & Cordelia, S. (2009). Description of interest regions with localbinary patterns. Pattern Recognition, 42, 425–436.
Ojala, T., Matti, P., & Harwood, D. (1996). A comparative study of texture measureswith classification based on feature distributions. Pattern Recognition, 29(1),51–59.
Pass, G., Zabih, R., & Miller, J. (1997). Comparing images using color coherencevectors. In ACM multimedia conf., Boston, USA (pp. 65–73).
Pei-Cheng, C., Been-Chian, C., Hao-Ren, K., & Wei-Pang, Y. (2008). A two-levelrelevance feedback mechanism for image retrieval. Expert Systems withApplications, 34, 2193–2200.
Persoon, E., & Fu, K. S. (1977). Shape discrimination using Fourier descriptors. IEEETransactions on Systems, Man and Cybernetics, 7(3), 170–179.
Rui, Y., She, A. C., & Huang, T. S. (1996). Modified Fourier descriptors for shaperepresentation – a practical approach. In First international workshop on imagedatabases and multimedia search.
Shufu, X., Shiguang, S., Xilin, C., & Jie, C. (2010). Fusing local patterns of gabormagnitude and phase for face recognition. Image Processing, IEEE Transactions,19(5), 1349–1361.
Smith, J. R., & Chang, S. F. (1996). Automated binary texture feature sets for imageretrieval. IEEE int. conf. acoustics, speech and signal processing, Columbia Univ.,New York (pp. 2239–2242).
Stricker, M., & Orengo, M., Similarity of color images. In SPIE, storage and retrieval forimage and video database, San Jose, USA (pp. 381–392).
8026 S.K. Vipparthi, S.K. Nagar / Expert Systems with Applications 41 (2014) 8016–8026
Subrahmanyam, M., Maheshwari, R. P., & Balasubramanian, R. (2012a). Expertsystem design using wavelet and color vocabulary trees for image retrieval.International Journal of Expert systems with applications, 39, 5104–5114.
Subrahmanyam, M., Maheshwari, R. P., & Balasubramanian, R. (2012b). Directionallocal extrema patterns: A new descriptor for content based image retrieval.International Journal of Multimedia Information Retrieval, 1(3), 191–203.
Subrahmanyam, M., Maheshwari, R. P., & Balasubramanian, R. (2012c). Local tetrapatterns: A new feature descriptor for content based image retrieval. IEEETransactions on Image Processing, 21(5), 2874–2886.
Subrahmanyam, M., Maheshwari, R. P., & Balasubramanian, R. (2012d). Localmaximum edge binary patterns: A new descriptor for image retrieval and objecttracking. Signal Processing, 92, 1467–1479.
Subrahmanyam, M., Maheshwari, R. P., & Balasubramanian, R. (2012e). Directionalbinary wavelet patterns for biomedical image indexing and retrieval. Journal ofMedical Systems, 36(5), 2865–2879.
Swain, M. J., & Ballard, D. H. (1990). Indexing via color histograms. In Thirdinternational conference on computer vision (pp. 390–393).
Turgay, Ç., & Tardi, T. (2011). Multiscale texture classification and retrieval based onmagnitude and phase features of complex wavelet sub bands. Computers andElectrical Engineering, 37, 729–743.
Valtteri, T., Timo, A., & Matti, P. (2005). Block-based methods for image retrievalusing local binary patterns. In SCIA 2005. LNCS (Vol. 3450, pp. 882–891). .
Vipparthi, S., & Nagar, S. (in press). Directional local ternary patterns for multimediaimage indexing and retrieval. International Journal of Signal and Imaging SystemsEngineering.
Vipparthi, S., & Nagar, S. (2014). Multi-joint histogram based modeling for imageindexing and retrieval. Computers and Electrical Engineering. http://dx.doi.org/10.1016/j.compeleceng.2014.04.018.
