Expert image retrieval system using directional local motif XoR patterns

Download Expert image retrieval system using directional local motif XoR patterns

Post on 01-Feb-2017

216 views

Category:

Documents

4 download

TRANSCRIPT

  • Expert Systems with Applications 41 (2014) 80168026Contents lists available at ScienceDirect

    Expert Systems with Applications

    journal homepage: www.elsevier .com/locate /eswaExpert image retrieval system using directional local motif XoR patternshttp://dx.doi.org/10.1016/j.eswa.2014.07.0010957-4174/ 2014 Elsevier Ltd. All rights reserved.

    Corresponding author at: Malaviya National Institute of Technology, Jaipur302017, Rajasthan, India.

    E-mail addresses: santu155@gmail.com (S.K. Vipparthi), sknagar.eee@iitbhu.ac.in (S.K. Nagar).Santosh Kumar Vipparthi a,b,, S.K. Nagar aa Indian Institute of Technology Banaras Hindu University, Varanasi 221005, Uttar Pradesh, IndiabMalaviya National Institute of Technology, Jaipur 302017, Rajasthan, India

    a r t i c l e i n f oArticle history:Available online 10 July 2014

    Keywords:Image retrieval (IR)Local binary patterns (LBP)XoR patternsMotif matrixDatabasea b s t r a c t

    This paper presents a new image feature descriptor, namely directional local motif XoR patterns(DLMXoRPs) for image retrieval application. The proposed motif representation is entirely different fromexisting motif. The DLMXoRP presents a novel technique for the calculation of motif using 1 3 grids. Theproposed motif (1 3) representation is having a flexible structure; hence it can able to extract all direc-tional information. This flexibility is not present in the existing (2 2) motif. Further, the XoR operationis performed on the transformed new motif images which are not present in the literature (local binarypatterns (LBP) and motif co-occurrence matrix (MCM)). To elevate the benefits of DLMXoRP, we compareit with the Motif XoR pattern (MXoR) which is calculated by applying the XoR operation on existingtransformed motif image. The performance of the proposed method is tested by conducting three exper-iments on Corel-5000, Corel-10000 and MIT-VisTex databases. The results after investigation show a sig-nificant improvement in terms of average retrieval precision (ARP) and average retrieval rate (ARR) ascompared to the state-of-the-art techniques for image retrieval.

    2014 Elsevier Ltd. All rights reserved.1. Introduction

    Content based image retrieval (CBIR) also known as contentbased visual information retrieval (CBVIR) has become a dynamicresearch area since last decade and has been paid ancillary atten-tion in recent years as a consequence of the vivid and rapid risein the volume of digital media. Digital media has become mostimportant tool for human communication and it provides con-tented information to the world. So digital data has gained widerange of applications in the field of medical, scientific, educational,entertainment, and many other which widened the scope of CBIR.The image retrieval system is an efficient system for searching andretrieving images from a large volume of database. CBIR utilizesthe content (color, texture, shape, etc.) of an image rather thanmetadata such as tags and keywords for retrieving images fromimmense database. Most of the web based search engines merelyimpart on metadata and this causes a lot of junk in the result. Thereis no single best representation of an image for all perceptionalsubjectivity, because the user may take photos in different condi-tions. Widespread literature survey on CBIR is accessible inChuen-Horng, Chun-Chieh, Hsin-Lun, and Jan-Ray (2014),Fatahiyah, Rossitza, and Yu-kun (2013), Ja-Hwung, Yu-Ting, Hsin-Ho, and Vincent (2010) and Pei-Cheng, Been-Chian, Hao-Ren, andWei-Pang (2008).

    The color concept of an image is one of the most significant fea-ture in the field of CBIR, if it is maintained semantically intact. Inaddition, color structure in visual scenery changes in size, resolu-tion and orientation. Many approaches are accessible to extractsubstantial features such as, texture, color, shape and integrationof two or more such features. Swain and Ballard (1990) proposedthe concept of color histogram and also introduced the histogramintersection distance metric to measure the distance between thehistograms of images. Pass, Zabih, and Miller (1997) have intro-duced the concept of color coherence vector (CCV). The CCV parti-tions the each histogram bin into two types i.e. coherent if itbelongs to large consistently colored region or incoherent, if it doesnot. Stricker and Orengo (1995) have used the first three centralmoments called mean, standard deviation and skewness of eachcolor space for image retrieval. Huang, Kumar, and Mitra (1997)have proposed the color correlogram which characterizes not onlythe color distributions of pixels, but also the spatial correlationbetween pair of colors. Lu and Burkhardt (2005) proposed the colorfeature based on vector quantized (VQ) index histograms in thediscrete cosine transform (DCT) domain.

    Texture is another significant characteristic of an image. Textureanalysis has been extensively used in medicine, industrial, amuse-ment, computer vision, gesture recognition and other image

    http://crossmark.crossref.org/dialog/?doi=10.1016/j.eswa.2014.07.001&domain=pdfhttp://dx.doi.org/10.1016/j.eswa.2014.07.001mailto:santu155@gmail.commailto:sknagar.eee@iitbhu.ac.inmailto:sknagar.eee@iitbhu.ac.inhttp://dx.doi.org/10.1016/j.eswa.2014.07.001http://www.sciencedirect.com/science/journal/09574174http://www.elsevier.com/locate/eswa

  • S.K. Vipparthi, S.K. Nagar / Expert Systems with Applications 41 (2014) 80168026 8017processing applications due to its potential value. Various algo-rithms has been developed for texture analysis, such as the meanand variance of the wavelet coefficients used as texture featureswhich is developed by Smith and Chang (1996). Turgay and Tardi(2011) have proposed the multi scale texture classification usingthe magnitude and phase features for complex wavelet responses.Subrahmanyam, Maheshwari, and Balasubramanian (2012a) haveintroduced the wavelet and color vocabulary trees for textureretrieval.

    In addition to color and texture, the shape features are also usedin image retrieval field. Usually the shape features are collectedafter the images have been segmented into regions or objects.Persoon and Fu (1977) have proposed the shape description ofimages using Fourier descriptors. Rui, She, and Huang (1996) pro-posed the modified Fourier descriptors to make it robust to thenoise and invariant to the geometric transformations.

    In recent years, local image feature extraction has gained moreattention in the field of CBIR. The local feature descriptor uses thevisual features of regions or objects to describe the image. A visualcontent descriptor can either be local or global. A global descriptoruses the visual features of the whole image, while a local descrip-tor uses the visual features of regions or objects to describe theimage. Many local descriptors are reported in the literature suchas; Ojala, Matti, and Harwood (1996) have proposed local binarypatterns which can show a better performance as well as less com-putational complexity for texture classification. LBP is succeededin terms of performance and speed, this is reported in manyresearch areas like texture classification (Ojala et al., 1996; Guo,Zhang, & Zhang, 2010), face recognition (Ahonen, Hadid, &Pietikainen, 2006), image retrieval (Cheng-Hao & Shu-Yuan,2003; Subrahmanyam, Maheshwari, & Balasubramanian, 2012b,2012c, 2012d, 2012e; Valtteri, Timo, & Matti, 2005; Vipparthi &Nagar, in press, 2014). Likewise, Vipparthi and Nagar (in press)have proposed the directional local ternary pattern based on direc-tional edges for image retrieval. Further, Vipparthi and Nagar(2014) have proposed multi-joint histogram based modelling forimage retrieval application. The different LBP variant features forimage retrieval, texture retrieval and object tracking applicationshave been proposed in Subrahmanyam et al. (2012b, 2012c,2012d, 2012e). Valtteri et al. (2005) have proposed the block-based LBP feature for CBIR. Cheng-Hao and Shu-Yuan (2003)derived two types of local edge pattern (LEP) histograms, one islocal edge patterns for segmentation (LEPSEG), and the other, localedge patterns with invariance (LEPINV) for image retrieval. Marko,Matti, and Cordelia (2009) proposed the center-symmetric localbinary pattern (CS-LBP) which integrates the concept of LBP andscale invariant feature transform (SIFT) for description of interestregion.

    Shufu, Shiguang, Xilin, and Jie (2010) have proposed the localGabor XoR patterns (LGXP) operator for face recognition.Jhanwar, Chaudhuri, Seetharamanc, and Zavidovique (2004) pro-posed the motif co-occurrence matrix (MCM) for CBIR. The color-MCM features also collected by applying the MCM on individualred (R), green (G), and blue (B) color channels. Lin, Chen, andChan (2009) proposed the integration of three features for CBIR.Further, it is mentioned that these three features are MCM, differ-ence between pixels of scan pattern (DBPSP) and color histogramfor K-mean (CHKM). In this paper, we propose a new featuredescriptor, directional local motif XoR pattern (DLMXoRP) which uti-lizes the concepts of LGXP (Shufu et al., 2010) and MCM (Jhanwaret al., 2004) for image retrieval. The main contributions and high-lights of this paper are given below.

    1. The existing motif operation in Jhanwar et al. (2004) has usedfour pixels as one grid (i.e. 2 2 grid). Whereas our proposedmotif uses only three pixels as one grid (i.e. 1 3 grid).2. The existing motif demands six comparisons to encode the2 2 grid whereas our method requires only three comparisonsfor 1 3 grid.

    3. The existing motif is not able to provide the specific directionalinformation due to its grid structure whereas our proposedmotif is able to collect four directional information.

    4. We analyze the performance of individual and combined fourdirectional features of our proposed motif.

    The organization of the paper is as follows: in Section 1, a briefreview of CBIR and related work are given. Section 2 presents aconcise review of feature extraction strategies. The proposed imageretrieval system framework is given in Section 3. Experimentalresults of various methods are analyzed in Section 4 and finallyin Section 5, we conclude the summary of work.

    2. Feature extraction methods

    2.1. Local binary patterns (LBP)

    LBP (Ojala et al., 1996) was derived from the general definitionof texture in a local neighborhood. This method shows success interms of performance and speed, and is reported in many researchareas such as texture classification, face recognition, image retrie-val, etc.

    Given a center pixel in the 3 3 pattern, LBP value is computedby comparing its gray scale value with its surrounding neighbors asfollows:

    LBPN;R XN1n0

    2n f1gn gc 1

    f1x 1 x P 00 x < 0

    2

    where N is the number of neighbors, R is the radius of the neighbors,gc denotes the gray value of the center pixel and gn is the gray valueof its neighbors.

    2.2. Motif co-occurrence matrix

    Jhanwar et al. (2004) have proposed the motif co-occurrencematrix (MCM) for CBIR. The MCM is designed from the transformedmotif image which is computed by segmenting the whole imageinto non-overlapping 2 2 grid (pixel patterns). In our work, weused seven scan motif patterns with a size of 2 2. Each patternis replaced by a scan motif number as shown in Fig. 1. Further, thismethod has been converted into translation invariant by applyingmotif operation on overlapped 2 2 pixel patterns. Finally, thetransformed motif image Mi(x,y) formed and it minimizes the localgradient. More details on MCM can be found in the literature(Jhanwar et al., 2004; Lin et al., 2009).

    3. Proposed image retrieval system

    3.1. Motif XoR patterns (MXoR)

    In order to analyze the performance of proposed method, wealso incorporated XoR operation on the existing motif as follows.The MXoR is derived by applying the XoR operator on a trans-formed motif image. For a given 3 3 pattern, MXoR value is com-puted by comparing the center motif value with its neighboringmotif values as follows:

    MXoRN;R XN1i0

    MImpi;pc 2i 3

  • P Q

    R S

    P PP P

    Fig. 1. Different types of scanned motif using 2 2 grids.

    Fig. 2. Construction of 1 3 sub-grids from a given 3 3 grid, (a) horizontal direction, (b) vertical direction and (c) and (d) diagonal directions.

    Fig. 3. Proposed motif calculation in 0 direction for a given 1 3 sub-grid.

    8018 S.K. Vipparthi, S.K. Nagar / Expert Systems with Applications 41 (2014) 80168026MImpi;pc 1 pi pc0 pi pc

    4

    where N is the number of neighbors (N = 8) and R is the radius of theneighbor (R = 1), pc denotes motif value of a center pixel in a givenmotif image; pi is the motif value of neighbor.

    After computing the motif XoR pattern (MXoR) for each pixel(i, j), the whole image is represented by building a histogram usingEq. (5).

    HMXoRPl XN1i1

    XN2j1

    f2MXoRi; j; l; l 2 0; 2N 1 5

    f2x; y 1 x y0 else

    6

    where N1 N2 represents the size of input image and N is numberof neighbors.

    3.2. Proposed directional local motifs

    The image is divided into an overlapped 3 3 grid. From eachgrid, four 1 3 sub-grids are retrieved in 0, 45, 90 and 135directions as shown in Fig. 2. All four sub-grids are constructedbased on the center pixel of 3 3 grid (see in Fig. 2). Further, thedirectional motifs are derived from four sub-grids as shown inFig. 3. The detailed explanation of the 0 direction motif calculationis given in Section 3.3.

    3.3. Analysis

    Here our main aim is to determine a new transformed motifnumber based on the pixel behavior. The detailed explanation for0 direction motif calculation is given as follows.

    Let a, b and c are the gray values of 1 3 sub-grid in 0 direction(see in Fig. 3).

    1. For motif 1: when a = 1, b = 2 and C = 3. Here, b is the centerpixel value in 1 3 pattern. If it satisfies the condition of b > aand b < C then the 1 3 pattern is replaced by a new motifvalue 1. Similarly, motif 2 is coded when A > b and b > c.

    2. For motif 3: when A = 4, b = 3 and C = 5. Center pixel valueis smaller than the neighbours and A < C. Then, the new motifvalue with 3 is coded. Similarly, motif 4 is coded whenA > C.

    3. For motif 5: as if a = 3, b = 5 and c = 4. Center pixel value islarger than the neighbours and a < c. Then, new motif valuewith 5 is coded. Similarly, motif 6 is coded when a > c.

  • S.K. Vipparthi, S.K. Nagar / Expert Systems with Applications 41 (2014) 80168026 80194. For motif 7: if all three pixels bearing same gray scale values,then, new motif value with 7 is coded.

    In this paper following notations are used for better under-standing. Smaller and larger value is defined by using symbols. Whereas and symbols are referred to smaller ofsmallest and larger of largest values respectively among a andc pixels. If a value is larger of largest value then superscript ofA and symbol are used. Similarly for smaller of smallest value and subscript of c is used and vice versa in Fig. 3.

    ThImf ; a; b

    1; a P f & a 6 b2; a 6 f & a P b3; a 6 f & a 6 b & f 6 b4; a 6 f & a 6 b & b 6 f 5; a P f & a P b & f 6 b6; a P f & a P b & b 6 f 7; a f &a b&b f

    8>>>>>>>>>>>>>>>>>>>>>:

    jh0

    7

    ThImc; a; g

    1; a P c & a 6 g2; a 6 c & a P g3; a 6 c & a 6 g & c 6 g4; a 6 c & a 6 g & g 6 c5; a P c & a P g & c 6 g6; a P c & a P g & g 6 c7; a c&a g&g c

    8>>>>>>>>>>>>>>>>>>>>>:

    jh45

    8

    ThImd; a;h

    1; a P d & a 6 h2; a 6 d & a P h3; a 6 d & a 6 h & d 6 h4; a 6 d & a 6 h & h 6 d5; a P d & a P h & d 6 h6; a P d & a P h & h 6 d7; a d & a h & h d

    8>>>>>>>>>>>>>>>>>>>>>:

    jh90

    9

    ThIme; a; i

    1; a P e & a 6 i2; a 6 e & a P i3; a 6 e & a 6 i & e 6 i4; a 6 e & a 6 i & i 6 e5; a P e & a P i & e 6 i6; a P e & a P i & i 6 e7; a e&a i&i e

    8>>>>>>>>>>>>>>>>>>>>>:

    jh135

    10

    Finally, the whole image is converted into four directional trans-formed motif images.

    3.4. Directional local motif XoR patterns (DLMXoRP)

    Local motif XoR operator is derived by applying XoR operationon individual directional transformed motif images. For, a given3 3 pattern, DLMXoRP value is computed by comparing the cen-ter motif value with its surrounding neighbor motif values asfollows:

    DLMXoRPhN;R XN1i0

    ThImpi;pc 2i

    h0 ;45 ;90 ;135

    11

    ThImpi; pc 1 pi pc0 pi pc

    h0 ;45 ;90 ;135

    12where N is the number of neighbors, R is the radius of the neighbors,pc denotes center motif value, pi is the motif value of neighbor and hdenotes direction of motif image.

    After computing the DLMXoRP, the whole image is representedby building a histogram as follows:

    HhDLMXoRPl XN1i1

    XN2j1

    f2DLMXoRPhi; j; l; l 2 0; 2N 1h0 ;45 ;90 ;135

    13

    where N1 N2 is the size of input image.Finally, the feature vector for a given image is constructed by

    concatenating four directional histograms as given in Eq. (14).

    HDLMXoRP H0

    DLMXoRP;H45DLMXoRP;H

    90DLMXoRP;H

    135DLMXoRP

    h i143.5. Proposed system framework

    Fig. 4 illustrates the flowchart of proposed feature extractionmethod and algorithm for the same is given bellow:

    Algorithm

    Input: Image; Output: Retrieval results1. Load the image and convert into a gray scale (if it is RGB).2. Divide the image into 3 3 grids.3. Collect four 1 3 sub-grids from each 3 3 grid.4. Calculate the motif values (Fig. 4(a)).5. Collect the four motif images in 0, 45, 90 and 135directions.6. Apply XoR operation on four motif images to formDLMXoRPs (Fig. 4(b)).7. Construct the histograms for 0, 45, 90 and 135directional DLMXoRPs.8. Construct a feature vector by concatenating thehistograms (Fig. 4(c)).9. Compare the query feature with features in the databaseusing Eq. (14) (Fig. 4 (d)).10. Retrieve the images based on the best matches(Fig. 4(e)).

    3.6. Similarity measures

    In this paper, four types of similarity distance measures are usedas given below:

    Manhattan or L1 or city block Distance : DsQm; Tm

    XLgi1

    j f Tm ;i fQm ;ij 15

    Euclidean or L2 Distance : DsQm; Tm XLgi1

    jfTm ;i fQm ;ij2

    !1=2

    16

    d1 Distance : DsQm; Tm XLgi1

    fTm ;i fQm ;i1 fTm ;i fQm ;i

    17

    Canberra Distance : DsQm; Tm XLgi1

    jfTm ;i fQm ;ijjfTm ;i fQm ;ij

    18

    where Qm is query image, Lg is feature vector length, Tm is image indatabase; fTm ;i is i

    th feature of image Tm in the database, fQm ;i is ithfeature of query image Qm.

  • DLMXoRP Feature Database

    Similarity Measurement

    5 7 6 4 2

    2 1 3 6 5

    5 3 1 2 7

    3 5 2 1 5

    3 5 1 1 2

    Motif calculation using 1x3sub-grid in 0 degree direction

    127 223 248

    222 120 224

    184 254 152

    XOR patterns calculations

    Feature Vector

    Result

    DLMXoRP

    Database Images

    Query Image

    a

    b

    c

    c

    d

    e

    Fig. 4. Proposed image retrieval system framework.

    8020 S.K. Vipparthi, S.K. Nagar / Expert Systems with Applications 41 (2014) 801680264. Experimental results and discussions

    In this paper, the proposed method is evaluated on three bench-mark databases followed by brief description about experimentalconditions. Corel- database (http://wang.ist.psu.edu/docs/rela-ted.shtml) comprises thousands of images of various contents rang-ing from animals, and outdoor sports to natural images. By domainprofessionals this database is pre-classified into different categoriesof size 100. Because of its size and heterogeneous content Corel-database meets all the requirements to evaluate an image retrievalsystem. In this paper, we use the Corel-5000 and Corel-10000 dat-abaseswhich consist of 50 and 100 different categories respectively.Each category has NG (=100) images with resolution of either126 187 or 187 126.

    The retrieval performance of the proposed method is measuredin terms of precision (P), average retrieval precision (ARP), recall(R) and average retrieval rate (ARR) are shown below.

    For the query image Iq the precision (P) and Recall (R) aredefined as follows.Precision : PIq Number of Relevant Images RetrievedTotal Number of Images Retrieved 19

    ARP 1DB

    XDBi1

    PIin610

    20

    Recall : RIq Number of Relevant Images Retrieved

    Total Number of Relevant Images in the Database21

    ARR 1DB

    XDBi1

    RIinP10

    22where DB is the total number of images in the database.Table 1Performance of various methods in terms of ARP and ARR on Corel-5000 and Corel-10000

    Database Performance (%) Methods

    CS-LBP BLK-LBP

    Corel-5000 ARP 32.96 45.75ARR 13.99 20.29

    Corel-10000 ARP 26.43 38.13ARR 10.15 15.344.1. Experiment 1

    In this experiment, the performance of the proposed method(DLMXoRP) is tested on Corel-5000 database in terms of ARP andARR. Table 1 exemplifies the retrieval performance of DLMXoRPand other existing techniques (LBP, CS-LBP, BLK-LBP, and MXoR)on Corel-5000 and Corel-10000 databases in terms of ARP andARR. Fig. 5(a) and (b) illustrate the category wise performance ofvarious methods in terms of precision and recall. Fig. 5(c) and (d)illustrate the performance comparison between the proposedmethod and other existing methods in terms of ARP and ARRrespectively on Corel-5000 database. We also analyze the individ-ual performance of directional features of DLMXoRP as shown inTable 2. From Table 2, it is clear that the DLMXoRP with all direc-tional information outperforms the individual directional featuresof DLMXoRP and MXoR in terms of ARP and ARR on Corel-5000database. Table 3 illustrates the performance of DLMXoRP usingvarious distance measures in terms of ARP and ARR on Corel-5000 database. From Table 3, it is observed that d1 distance mea-sure shows a better performance as compared to other existing dis-tance measures. From Tables 13 and Fig. 5, it is clear that theproposed method outperforms the other existing methods onCorel-5000 database. The retrieval performance of the proposedmethod and other existing techniques are verified with precisionversus recall curve as shown in Fig. 6. From the Fig. 6 it is clear that,the proposed method show a significant improvement as com-pared to other techniques on Corel-5000 database. The queryresults of the proposed method on Corel-5000 database is (top leftimage is query image) shown in Fig. 7.4.2. Experiment 2

    In this experiment, Corel-10000 database is used to evaluateour proposed method. Table 4 illustrates the retrieval performancedatabases.

    LEPINV LBP MXoR DLMXoRP

    35.19 43.62 42.40 49.4714.84 19.22 18.97 21.76

    28.93 37.62 34.954 40.93 34.95 40.9311.22 14.97 14.12 16.27

    http://wang.ist.psu.edu/docs/related.shtmlhttp://wang.ist.psu.edu/docs/related.shtml

  • 0 10 20 30 40 500

    102030405060708090

    100

    Prec

    isio

    n (%

    )

    Category

    CS_LBP LEPINV BLK_LBP LBP MXOR DLMXoRP

    0 10 20 30 40 5005

    10152025303540455055

    Rec

    all (

    %)

    Category

    CS_LBP LEPINV BLK_LBP LBP MXOR DLMXoRP

    0 10 20 30 40 50 60 70 80 90 10010

    15

    20

    25

    30

    35

    40

    45

    50

    AR

    P (%

    )

    No of Top Matching Images

    CS_LBP LEPINV BLK_LBP LBP MXOR DLMXoRP

    0 10 20 30 40 50 60 70 80 90 1002468

    10121416182022

    AR

    R (%

    )

    No of Top Matching Images

    CS_LBP LEPINV BLK_LBP LBP MXOR DLMXoRP

    Fig. 5. Comparison of proposed method with other existing methods in terms of: (a) category wise precision, (b) category wise recall, (c) ARP and (d) ARR on Corel-5000database.

    Table 2Results of proposed method with individual and combined directional features in terms of ARP and ARR on Corel-5000 database.

    Database Performance (%) Methods

    MXoR DLMXoRPs

    0 45 90 135 PM

    Corel-5000 ARP 42.40 41.91 40.65 42.13 42.93 49.47ARR 18.98 18.71 18.14 18.69 18.94 21.76

    PM: proposed method with all directional motif information.

    S.K. Vipparthi, S.K. Nagar / Expert Systems with Applications 41 (2014) 80168026 8021of MXoR and DLMXoRP with individual and combined directionalfeatures on Core-10000 database. From Table 4, it is observed thatthe DLMXoRP with all directional information shows around 6%improvement in terms of ARP as compared to MXoR and DLMXoRPwith individual directional information. Table 5 illustrates the per-formance of the proposed method with different distance mea-sures on Corel-10000 database. From Table 5, it is observed thatthe distance measure (d1) shows better performance as comparedto the other existing distance measures. Fig. 8(a) and (b) illustratesthe category wise performance of the proposed method in terms ofprecision and recall. Fig. 8(c) and (d) exemplifies the performanceof various methods on Core-10000 database in terms of ARP andARR respectively. From the Tables 1, 4, 5 and Fig. 8, it is clear thatthe proposed method outperforms the other existing methods on

  • Table 3The performance of DLMXoRP with various distance measures in terms of ARP andARR on Corel-5000 database.

    Performance Distance measure

    L1 Canberra L2 d1

    ARP (%) 45.092 49.416 38.544 49.468ARR (%) 19.627 21.768 16.6134 21.76

    Fig. 6. Precision Vs Recall curve for the proposed method and other existingtechniques on Corel-5000 database.

    Table 4Results of proposed method with individual and combined directional features interms of ARP and ARR on Corel-10000 database.

    Database Performance(%)

    Methods

    MXoR DLMXoRPs

    0 45 90 135 PM

    Corel-10000

    ARP 34.95 33.75 33.28 34.08 34.59 40.93ARR 14.12 13.56 13.41 13.69 13.61 16.27

    PM: Proposed method with all directional motif information.

    Table 5The performance of DLMXoRP with various distance measures in terms of ARP andARR on Corel-10000 database.

    Performance Distance measure

    L1 Canberra L2 d1

    ARP (%) 37.15 40.69 31.28 40.93ARR (%) 14.52 16.25 11.96 16.27

    8022 S.K. Vipparthi, S.K. Nagar / Expert Systems with Applications 41 (2014) 80168026Corel-10000 database in terms of their evaluation measures. FromFig. 5, it is observed that the proposed method shows less retrievalperformance on categories 2, 11, 14, 17, 20, 27, 37, 39, 46 and 50 ascompared to the other existing methods. Similarly, From Fig. 8, it isobserved that, 53, 58, 60, 63, 89 and 100 as compared to the otherexisting methods. The reason behind this is, these categories hav-ing less discriminative information. However, the overall (average)Fig. 7. Two query results of the proposed method (top lefperformance of the proposed method shows a significant improve-ment as compared to the existing methods in terms of precision,recall, average and average retrieval rate on Corel-5000 andCorel-10000 databases. The retrieval performance of the proposedmethod and other existing techniques are verified with precisionversus recall curve as shown in Fig. 9. From Fig. 9 it is clear thatthe proposed method show a significant improvement as com-pared to other techniques on Corel-10000 database. Fig. 10 showsthe query results of the proposed method on Corel-10000database.t image is the query image) on Corel-5000 database.

  • 0 10 20 30 40 50 60 70 80 90 100

    102030405060708090

    Prec

    isio

    n (%

    )

    Category

    CS_LBP LEPINV BLK_LBP LBP MXOR DLMXoRP

    0 10 20 30 40 50 60 70 80 90 1000

    10

    20

    30

    40

    50

    60

    70

    Rec

    all (

    %)

    Category

    CS_LBP LEPINV BLK_LBP LBP MXOR DLMXoRP

    0 10 20 30 40 50 60 70 80 90 1008

    1012141618202224262830323436384042

    AR

    P (%

    )

    No of Top Matching Images

    CS_LBP LEPINV BLK_LBP LBP MXOR DLMXoRP

    0 10 20 30 40 50 60 70 80 90 10023456789

    1011121314151617

    AR

    R (%

    )

    No of Top Matching Images

    CS_LBP LEPINV BLK_LBP LBP MXOR DLMXoRP

    Fig. 8. Comparison of proposed method with other existing methods in terms of: (a) category wise precision, (b) category wise recall, (c) ARP and (d) ARR on Corel-10000database.

    Fig. 9. Precision Vs recall curve for proposed method and other existing techniqueson Corel-10000 database.

    S.K. Vipparthi, S.K. Nagar / Expert Systems with Applications 41 (2014) 80168026 80234.3. Experiment 3

    In this experiment, we use the MIT-VisTex database (http://vis-mod.media.mit.edu/pub/) which consists of 40 different textureswith size 512 512. These textures are divided into sixteen128 128 non-overlapping sub-images, thus creating a databaseof 640 (40 16) images. The performance of the proposed methodis measured in terms of average retrieval rate (ARR). Fig. 11 illus-trates the retrieval results of the proposed method and variousexisting methods on MIT-VisTex database in terms of ARR. FromFig. 11, it is observed that the proposed DLMXoRP shows betterperformance than the state-of-the-art features for image retrievalon MIT-VisTex database. Further, the performance of the proposedmethod is analyzed with different directional information asshown in Fig. 12. From Fig. 12, it observed that the 0 + 45 +90 + 135 directional information shows better performance than

    http://vismod.media.mit.edu/pub/http://vismod.media.mit.edu/pub/

  • Fig. 10. Two query results of the proposed method (top left image is the query image) on Corel-10000 database.

    8 16 24 32 40 48 56 64 72 80 88 96 104 11270

    75

    80

    85

    90

    95

    100

    AR

    R (%

    )

    No of Top Matching Images

    CS-LBP LEPINV BLK-LBP LBP MXOR PM

    Fig. 11. Comparison of proposed method with other existing methods in terms ofARR MIT-VisTex database.

    83

    85

    87

    89

    91

    93

    95

    97

    99

    16 32 48 6

    AR

    R (%

    )

    Number of

    0 De90 D0+4590+10+130+450+900+45

    Fig. 12. Retrieval performance of the DLMXoRP with the po

    8024 S.K. Vipparthi, S.K. Nagar / Expert Systems with Applications 41 (2014) 80168026other directional information. Fig. 13 shows the query results ofthe proposed method on MIT-VisTex database.5. Conclusions

    In image retrieval, the feature extraction and similarity mea-surement are two major factors which determining the accuracyrate of the retrieval system. In this paper, we mainly focused todesign/ develop an expert feature extraction approach using direc-tional local motif XoR patterns (DLMXoRP).

    The proposed scanned motif using 1 3 grids and its study iscompletely different from existing motif using 2 2 grids. In imageanalysis, extracting specific directional information is an importanttask. Thus, the existing motif representation fails to extract specificdirectional information from an image due to its grid alignmentwhereas; our proposed motif is able to collect four directional4 80 96 112 top matches

    gree 45 Degreeegree 135 Degree Degrees 45+90 Degrees35 Degrees 0+90 Degrees5 Degrees 45+135 Degrees+90 Degrees 0+45+135 Degrees+135 Degrees 45+90+135 Degrees+90+135 Degrees

    ssible directional information on MIT-VisTex database.

  • Fig. 13. Two query results of the proposed method (top left image is the query image) on MIT-VisTex.

    S.K. Vipparthi, S.K. Nagar / Expert Systems with Applications 41 (2014) 80168026 8025information. The existing motif demands six comparisons toencode the 2 2 grid whereas our proposed method requires onlythree comparisons for 1 3 grid. Further, the XoR operation wasperformed on transformed motif images.

    The retrieval performance of the proposed descriptor is testedon Corel-5000, Corel-10000 and MIT-VisTex databases. The retrie-val performance of DLMXoRP shows 7% and 6% improvement interms of ARP as compared to MXoR on Corel-5000 and Corel-10000 databases respectively. An improvement of 9.16% in termsof ARR as compared to MXoR on MIT-VisTex database wasachieved.References

    Ahonen, T., Hadid, A., & Pietikainen, M. (2006). Face description with local binarypatterns: Applications to face recognition. IEEE Transactions on Pattern Analysis,28(12), 20372041.

    Cheng-Hao, Y., & Shu-Yuan, C. (2003). Retrieval of translated, rotated and scaledcolor textures. Pattern Recognition, 36, 913929.

    Chuen-Horng, L., Chun-Chieh, C., Hsin-Lun & Jan-Ray, L. (2014). Fast K-meansalgorithm based on a level histogram for image retrieval. Expert Systems withApplications, 41, 32763283.

    Fatahiyah, M. A., Rossitza, S., & Yu-kun, L. (2013). Trademark image retrieval usingan integrated shape descriptor. Expert Systems with Applications, 40, 105121.

    Guo, Z., Zhang, L., & Zhang, D. (2010). Rotation invariant texture classification usingLBP variance with global matching. Pattern Recognition, 43(3), 706719.

    Huang, J., Kumar, S. R., & Mitra, M. (1997). Combining supervised learning withcolor correlograms for content based image retrieval. In Fifth ACM multimediaconf. (pp. 325334).Ja-Hwung, S., Yu-Ting, H., Hsin-Ho, Y., & Vincent, S. T. (2010). Effective content-based video retrieval using pattern-indexing and matching technique. ExpertSystems with Applications, 37, 50685085.

    Jhanwar, N., Chaudhuri, S., Seetharamanc, G., & Zavidovique, B. (2004). Contentbased image retrieval using motif co-occurrence matrix. Image and VisionComputing, 22, 12111220.

    Lin, C. H., Chen, R. T., & Chan, Y. K. A. (2009). Smart content-based image retrievalsystem based on color and texture feature. Image and Vision Computing, 27,658665.

    Lu, Z. M., & Burkhardt, H. (2005). Color image retrieval based on DCT-domain vectorquantization index histograms. IEEE Electronics Letters, 41(17), 956957.

    Marko, H., Matti, P., & Cordelia, S. (2009). Description of interest regions with localbinary patterns. Pattern Recognition, 42, 425436.

    Ojala, T., Matti, P., & Harwood, D. (1996). A comparative study of texture measureswith classification based on feature distributions. Pattern Recognition, 29(1),5159.

    Pass, G., Zabih, R., & Miller, J. (1997). Comparing images using color coherencevectors. In ACM multimedia conf., Boston, USA (pp. 6573).

    Pei-Cheng, C., Been-Chian, C., Hao-Ren, K., & Wei-Pang, Y. (2008). A two-levelrelevance feedback mechanism for image retrieval. Expert Systems withApplications, 34, 21932200.

    Persoon, E., & Fu, K. S. (1977). Shape discrimination using Fourier descriptors. IEEETransactions on Systems, Man and Cybernetics, 7(3), 170179.

    Rui, Y., She, A. C., & Huang, T. S. (1996). Modified Fourier descriptors for shaperepresentation a practical approach. In First international workshop on imagedatabases and multimedia search.

    Shufu, X., Shiguang, S., Xilin, C., & Jie, C. (2010). Fusing local patterns of gabormagnitude and phase for face recognition. Image Processing, IEEE Transactions,19(5), 13491361.

    Smith, J. R., & Chang, S. F. (1996). Automated binary texture feature sets for imageretrieval. IEEE int. conf. acoustics, speech and signal processing, Columbia Univ.,New York (pp. 22392242).

    Stricker, M., & Orengo, M., Similarity of color images. In SPIE, storage and retrieval forimage and video database, San Jose, USA (pp. 381392).

    http://refhub.elsevier.com/S0957-4174(14)00393-5/h0005http://refhub.elsevier.com/S0957-4174(14)00393-5/h0005http://refhub.elsevier.com/S0957-4174(14)00393-5/h0005http://refhub.elsevier.com/S0957-4174(14)00393-5/h0010http://refhub.elsevier.com/S0957-4174(14)00393-5/h0010http://refhub.elsevier.com/S0957-4174(14)00393-5/h0015http://refhub.elsevier.com/S0957-4174(14)00393-5/h0015http://refhub.elsevier.com/S0957-4174(14)00393-5/h0015http://refhub.elsevier.com/S0957-4174(14)00393-5/h0020http://refhub.elsevier.com/S0957-4174(14)00393-5/h0020http://refhub.elsevier.com/S0957-4174(14)00393-5/h0025http://refhub.elsevier.com/S0957-4174(14)00393-5/h0025http://refhub.elsevier.com/S0957-4174(14)00393-5/h0035http://refhub.elsevier.com/S0957-4174(14)00393-5/h0035http://refhub.elsevier.com/S0957-4174(14)00393-5/h0035http://refhub.elsevier.com/S0957-4174(14)00393-5/h0040http://refhub.elsevier.com/S0957-4174(14)00393-5/h0040http://refhub.elsevier.com/S0957-4174(14)00393-5/h0040http://refhub.elsevier.com/S0957-4174(14)00393-5/h0045http://refhub.elsevier.com/S0957-4174(14)00393-5/h0045http://refhub.elsevier.com/S0957-4174(14)00393-5/h0045http://refhub.elsevier.com/S0957-4174(14)00393-5/h0050http://refhub.elsevier.com/S0957-4174(14)00393-5/h0050http://refhub.elsevier.com/S0957-4174(14)00393-5/h0055http://refhub.elsevier.com/S0957-4174(14)00393-5/h0055http://refhub.elsevier.com/S0957-4174(14)00393-5/h0060http://refhub.elsevier.com/S0957-4174(14)00393-5/h0060http://refhub.elsevier.com/S0957-4174(14)00393-5/h0060http://refhub.elsevier.com/S0957-4174(14)00393-5/h0070http://refhub.elsevier.com/S0957-4174(14)00393-5/h0070http://refhub.elsevier.com/S0957-4174(14)00393-5/h0070http://refhub.elsevier.com/S0957-4174(14)00393-5/h0075http://refhub.elsevier.com/S0957-4174(14)00393-5/h0075http://refhub.elsevier.com/S0957-4174(14)00393-5/h0085http://refhub.elsevier.com/S0957-4174(14)00393-5/h0085http://refhub.elsevier.com/S0957-4174(14)00393-5/h0085

  • 8026 S.K. Vipparthi, S.K. Nagar / Expert Systems with Applications 41 (2014) 80168026Subrahmanyam, M., Maheshwari, R. P., & Balasubramanian, R. (2012a). Expertsystem design using wavelet and color vocabulary trees for image retrieval.International Journal of Expert systems with applications, 39, 51045114.

    Subrahmanyam, M., Maheshwari, R. P., & Balasubramanian, R. (2012b). Directionallocal extrema patterns: A new descriptor for content based image retrieval.International Journal of Multimedia Information Retrieval, 1(3), 191203.

    Subrahmanyam, M., Maheshwari, R. P., & Balasubramanian, R. (2012c). Local tetrapatterns: A new feature descriptor for content based image retrieval. IEEETransactions on Image Processing, 21(5), 28742886.

    Subrahmanyam, M., Maheshwari, R. P., & Balasubramanian, R. (2012d). Localmaximum edge binary patterns: A new descriptor for image retrieval and objecttracking. Signal Processing, 92, 14671479.

    Subrahmanyam, M., Maheshwari, R. P., & Balasubramanian, R. (2012e). Directionalbinary wavelet patterns for biomedical image indexing and retrieval. Journal ofMedical Systems, 36(5), 28652879.Swain, M. J., & Ballard, D. H. (1990). Indexing via color histograms. In Thirdinternational conference on computer vision (pp. 390393).

    Turgay, ., & Tardi, T. (2011). Multiscale texture classification and retrieval based onmagnitude and phase features of complex wavelet sub bands. Computers andElectrical Engineering, 37, 729743.

    Valtteri, T., Timo, A., & Matti, P. (2005). Block-based methods for image retrievalusing local binary patterns. In SCIA 2005. LNCS (Vol. 3450, pp. 882891). .

    Vipparthi, S., & Nagar, S. (in press). Directional local ternary patterns for multimediaimage indexing and retrieval. International Journal of Signal and Imaging SystemsEngineering.

    Vipparthi, S., & Nagar, S. (2014). Multi-joint histogram based modeling for imageindexing and retrieval. Computers and Electrical Engineering. http://dx.doi.org/10.1016/j.compeleceng.2014.04.018.

    http://refhub.elsevier.com/S0957-4174(14)00393-5/h0100http://refhub.elsevier.com/S0957-4174(14)00393-5/h0100http://refhub.elsevier.com/S0957-4174(14)00393-5/h0100http://refhub.elsevier.com/S0957-4174(14)00393-5/h0105http://refhub.elsevier.com/S0957-4174(14)00393-5/h0105http://refhub.elsevier.com/S0957-4174(14)00393-5/h0105http://refhub.elsevier.com/S0957-4174(14)00393-5/h0110http://refhub.elsevier.com/S0957-4174(14)00393-5/h0110http://refhub.elsevier.com/S0957-4174(14)00393-5/h0110http://refhub.elsevier.com/S0957-4174(14)00393-5/h0115http://refhub.elsevier.com/S0957-4174(14)00393-5/h0115http://refhub.elsevier.com/S0957-4174(14)00393-5/h0115http://refhub.elsevier.com/S0957-4174(14)00393-5/h0120http://refhub.elsevier.com/S0957-4174(14)00393-5/h0120http://refhub.elsevier.com/S0957-4174(14)00393-5/h0120http://refhub.elsevier.com/S0957-4174(14)00393-5/h0130http://refhub.elsevier.com/S0957-4174(14)00393-5/h0130http://refhub.elsevier.com/S0957-4174(14)00393-5/h0130http://refhub.elsevier.com/S0957-4174(14)00393-5/h0135http://refhub.elsevier.com/S0957-4174(14)00393-5/h0135http://dx.doi.org/10.1016/j.compeleceng.2014.04.018http://dx.doi.org/10.1016/j.compeleceng.2014.04.018

    Expert image retrieval system using directional local motif XoR patterns1 Introduction2 Feature extraction methods2.1 Local binary patterns (LBP)2.2 Motif co-occurrence matrix

    3 Proposed image retrieval system3.1 Motif XoR patterns (MXoR)3.2 Proposed directional local motifs3.3 Analysis3.4 Directional local motif XoR patterns (DLMXoRP)3.5 Proposed system framework3.6 Similarity measures

    4 Experimental results and discussions4.1 Experiment 14.2 Experiment 24.3 Experiment 3

    5 ConclusionsReferences

Recommended

View more >