HP Labs, Russia; nvassilieva@hp.com29 2007

I. II. CBIR - WTGF: Weighted Total with Gravitation Function

CBIR: CBIR Content Based Image Retrieval (CBIR)

Data

Cluster

(),

(content retrieval)

(color features)

(texture features)

(shape features)

(spatial layout)

(content retrieval)

(color features)

(texture features)

(shape features)

(spatial layout)

F(I) = (h1I, h2I, , hNI): L1, L2, LF(I) = (E1I,E2I,E3I, 1I,2I,3I, s1I,s2I,s3I). , , 3- : : ~L1Stricker M., Orengo M. Similarity of Color Images. Proceedings of the SPIE Conference, vol. 2420, p. 381-392, 1995

h2

hN

h1

(color features)

1. :d(H1, H2) > d(H1, H3)

Niblack W., Barber R., et al. The QBIC project: Querying images by content using color, texture and shape. In IS&T/SPIE International Symposium on Electronic Imaging: Science & Technology, Conference 1908, Storage and Retrieval for Image and Video Databases, Feb. 1993

2. :HA= HB = HC ., . . RCDL2005.i = 1..N ;(ai, bi, ci) i;weighti i ;(xi, yi) .

Stricker M., Dimai A. Spectral Covariance and Fuzzy Regions for Image Indexing. Machine Vision and Applications, vol. 10., p. 66-73, 1997

? (1)Stricker M., Orengo M. Similarity of Color Images. ... (3000 )

? (2) Corel Photo Set (285 ) .

:

(content retrieval)

(color features)

(texture features)

(shape features)

(spatial layout)

: Haraliks co-occurrence matrices Tamura Tamura features (Tamura image)

Voronoi tesselation featuresStructural methods

General statistics parametersHaralicks co-occurrence matricesTamura features

(texture features)

PWTTWTDCT, DST, DHT Complex waveletsGabor filters ICA filters

Markov random fieldsFractals

Grey Level Co-occurrence Matrices (GLCM): , . , ; I(p,q) , (p, q).

:

: , : - , - , , - , - ,

Tamura, : (coarseness) (contrast) (directionality) (line-likeness) (regularity) (roughness)

: -, ICA

Voronoi tesselation featuresStructural methods

General statistics parametersHaralicks co-occurrence matricesTamura features

(texture features)

PWTTWTDCT, DST, DHT Complex waveletsGabor filters ICA filters

Markov random fieldsFractals

-- : :

: : , S ,Uh, Ul .

ICAH. Borgne, A. Guerin-Dugue, A. Antoniadis. Representation of images for classification with independent features. Pattern Recognition Letters, vol. 25, p. 141-154, 2004

P. Howarth, S. Rger. Robust texture features for still image retrieval. In Proc. IEE Vis. Image Signal Processing, vol. 152, No. 6, December 2006 !

(2)Snitkowska, E. Kasprzak, W. Independent Component Analysis of Textures in Angiography Images. Computational Imaging and Vision, vol. 32, pages 367-372, 2006. v. s. ICA : ICA 13% Brodatz ICA 4%

:

(content retrieval)

(color features)

(texture features)

(shape features)

(spatial layout)

Centroid DistanceComplex CoordinatesCurvature signatureTurning Angle

(shape features)

(boundary-based methods)

(region-based methods)

TriangulationMedial Axis Transform(Skeleton Transform)

Fourier Descriptors UNL-FourierNFDWavelet Descriptors

B-Splines

Moment invariantsZernike momentsPseudo Zernike moments

Grid method

: (Chain Codes) (Fourier Descriptors)

(shape features)

(boundary-based methods)

(region-based methods)

Fourier Descriptors NFD...

: 03001033332322121111 : 70016665533222 4- 8- :: : :

1. (2D -> 1D): : z(t) = x(t) + iy(t)...

2. (s(t) ):3. (NFD Normalized Fourier Descriptors):4. :

: - (Grid-method) (Moment invariants)

(shape features)

(boundary-based methods)

(region-based methods)

Moment invariantsZernike momentsPseudo Zernike moments

Grid method

-: 001111000 011111111 111111111 111111111 111110111 0111000011 : 001100000 011100000 111100000 111101111 111111110 001111000: : ; ; .

(p+q) : f(x,y) : 7 , . :

Mehtre B. M., Kankanhalli M. S., Lee W. F. Shape measures for content based image retrieval: a comparison. Inf. Processing and Management, vol. 33, No. 3, pages 319-337, 1997.

QBICVisualSEEkNetraMars, HSV (HSV), Color codebook, (HSV) (HSV), Color Sets,Location infoTamura Image, Euclid dist Tamura Image, 3D Histo + Fourier-based ()MFD ()

I. II. CBIR - WTGF: Weighted Total with Gravitation Function

( ) ( ) ( ) CBIR (2)

CombMax, CombMin, CombSumCombAVGCombMNZ = CombSUM * number of nonzero similaritiesProbFuseHSC3D (CombSum ) : -:

, ColorMomentICAHistColorHist -

:CombMax, CombMin, CombSumCombAVGCombMNZ = CombSUM * number of nonzero similaritiesProbFuseHSC3D(x11, r11), (x12, r12), , (x1n, r1n)1(x21, r21), (x22, r22), , (x2n, r2n)2(xm1, rm1), (xm2, rm2), , (xmn, rmn)mi i- ; rik - k- ir0k = f(, Rk), ,Rk - k

([0..1], [0..1])N -> [0..1] [0..1]N -> [0..1]MinMax /CombMin, CombMax, CombAVG/: ( HSC3D): ( )

Weighted Total with Gravitation Function CombAVG, - () :

: Roverlap, Noverlap:Lee J. H. Analyses of multiple evidence combination. SIGIR '97: Proceedings of the 20th annual international ACM SIGIR conference on Research and development in information retrieval. New York, NY, USA: ACM Press, p. 267-276, 1997.

I: Flickr (~15000):Random MinMaxCombMNZWTGF_MTWeightedTotal

I: Roverlap) Roverlap delta=0.03 10 ) Roverlap delta=0.07 10

II ( ): (olorHist ) (olorMoment ) ICA (ICAHist):CombMNZWTGF_MTWTGF_MT_weighted: Corel Photo Set (285)

II Roverlap : ) ColorHist ColorMoment; b) ColorHist ICAHist; c) ColorMoment ICAHist.

III ICA : : 0, 0.25, 0.5, 0.75, 1 ( ) ,

: . .

:

:

: (1) .WTGF: ; ; .CombMNZ: ; . .

: (2) , . ? ?

: ? : , ICA : , ( ) ?

. . 952 -(1) How to merge fairly?

How to normalize the channels' evaluation scores in similar scale? Sometime rank can be used as a method of normalization. Tuning merging parameters in the best way.

(2) How to merge efficiently?

It is not necessary to merge the full result list. Truncated list merge is preferred.

Merge can be executed in parallel,hierarchy merge or pair-wise merge can be used.

(3) How to merge effectively?

Different merging algorithms perform differently for specific dataset. Merging algorithms such as COMBSUM, COMBMIN, COMBMAX (Shaw 1995). Fuzzy-AND merge and fuzzy-OR merge (Wu 2000) can be used.(1) How to merge fairly?

How to normalize the channels' evaluation scores in similar scale? Sometime rank can be used as a method of normalization. Tuning merging parameters in the best way.

(2) How to merge efficiently?

It is not necessary to merge the full result list. Truncated list merge is preferred.

Merge can be executed in parallel,hierarchy merge or pair-wise merge can be used.

(3) How to merge effectively?

Different merging algorithms perform differently for specific dataset. Merging algorithms such as COMBSUM, COMBMIN, COMBMAX (Shaw 1995). Fuzzy-AND merge and fuzzy-OR merge (Wu 2000) can be used.Previous explaination such as Lee's rational for fusion states "different runs retrieve similar sets of relevant documents but retrieve different sets of non-relevant documents" (Lee 1997). However, his coefficients R overlap and Noverlap are neither sufficient conditions nor necessary conditions for performance improvement. Later, Vogt (Vogt 1998) use a linear model to predict performance gain for different runs in TREC5. However, there are too many assumptions in his experimental settings and most are unjustified. It is still unclear in what condition could viewpoints enhance each other to generate a better result. And this question requires furthur research effort. Rcommon number of common relevant documentsR1 number of relevant documents in run1R2 number of relevant document in run2Ncommon number of common non relevant documentsN1 number of non relevant documents in run1N2 number of non relevant document in run2

So, we will compare our method with some of known others. There are: Classical methods: COMBSUM, COMBMIN, COMBMAX Probability methods: probFuseRandom method: random values that satisfied to merge properties.

Previous explaination such as Lee's rational for fusion states "different runs retrieve similar sets of relevant documents but retrieve different sets of non-relevant documents" (Lee 1997). However, his coefficients R overlap and Noverlap are neither sufficient conditions nor necessary conditions for performance improvement. Later, Vogt (Vogt 1998) use a linear model to predict performance gain for different runs in TREC5. However, there are too many assumptions in his experimental settings and most are unjustified. It is still unclear in what condition could viewpoints enhance each other to generate a better result. And this question requires furthur research effort. Rcommon number of common relevant documentsR1 number of relevant documents in run1R2 number of relevant document in run2Ncommon number of common non relevant documentsN1 number of non relevant documents in run1N2 number of non relevant document in run2

So, we will compare our method with some of known others. There are: Classical methods: COMBSUM, COMBMIN, COMBMAX Probability methods: probFuseRandom method: random values that satisfied to merge properties.

Previous explaination such as Lee's rational for fusion states "different runs retrieve similar sets of relevant documents but retrieve different sets of non-relevant documents" (Lee 1997). However, his coefficients R overlap and Noverlap are neither sufficient conditions nor necessary conditions for performance improvement. Later, Vogt (Vogt 1998) use a linear model to predict performance gain for different runs in TREC5. However, there are too many assumptions in his experimental settings and most are unjustified. It is still unclear in what condition could viewpoints enhance each other to generate a better result. And this question requires furthur research effort. Rcommon number of common relevant documentsR1 number of relevant documents in run1R2 number of relevant document in run2Ncommon number of common non relevant documentsN1 number of non relevant documents in run1N2 number of non relevant document in run2

So, we will compare our method with some of known others. There are: Classical methods: COMBSUM, COMBMIN, COMBMAX Probability methods: probFuseRandom method: random values that satisfied to merge properties.

Previous explaination such as Lee's rational for fusion states "different runs retrieve similar sets of relevant documents but retrieve different sets of non-relevant documents" (Lee 1997). However, his coefficients R overlap and Noverlap are neither sufficient conditions nor necessary conditions for performance improvement. Later, Vogt (Vogt 1998) use a linear model to predict performance gain for different runs in TREC5. However, there are too many assumptions in his experimental settings and most are unjustified. It is still unclear in what condition could viewpoints enhance each other to generate a better result. And this question requires furthur research effort. Rcommon number of common relevant documentsR1 number of relevant documents in run1R2 number of relevant document in run2Ncommon number of common non relevant documentsN1 number of non relevant documents in run1N2 number of non relevant document in run2

So, we will compare our method with some of known others. There are: Classical methods: COMBSUM, COMBMIN, COMBMAX Probability methods: probFuseRandom method: random values that satisfied to merge properties.

Previous explaination such as Lee's rational for fusion states "different runs retrieve similar sets of relevant documents but retrieve different sets of non-relevant documents" (Lee 1997). However, his coefficients R overlap and Noverlap are neither sufficient conditions nor necessary conditions for performance improvement. Later, Vogt (Vogt 1998) use a linear model to predict performance gain for different runs in TREC5. However, there are too many assumptions in his experimental settings and most are unjustified. It is still unclear in what condition could viewpoints enhance each other to generate a better result. And this question requires furthur research effort. Rcommon number of common relevant documentsR1 number of relevant documents in run1R2 number of relevant document in run2Ncommon number of common non relevant documentsN1 number of non relevant documents in run1N2 number of non relevant document in run2

So, we will compare our method with some of known others. There are: Classical methods: COMBSUM, COMBMIN, COMBMAX Probability methods: probFuseRandom method: random values that satisfied to merge properties.

Previous explaination such as Lee's rational for fusion states "different runs retrieve similar sets of relevant documents but retrieve different sets of non-relevant documents" (Lee 1997). However, his coefficients R overlap and Noverlap are neither sufficient conditions nor necessary conditions for performance improvement. Later, Vogt (Vogt 1998) use a linear model to predict performance gain for different runs in TREC5. However, there are too many assumptions in his experimental settings and most are unjustified. It is still unclear in what condition could viewpoints enhance each other to generate a better result. And this question requires furthur research effort. Rcommon number of common relevant documentsR1 number of relevant documents in run1R2 number of relevant document in run2Ncommon number of common non relevant documentsN1 number of non relevant documents in run1N2 number of non relevant document in run2

So, we will compare our method with some of known others. There are: Classical methods: COMBSUM, COMBMIN, COMBMAX Probability methods: probFuseRandom method: random values that satisfied to merge properties.

Previous explaination such as Lee's rational for fusion states "different runs retrieve similar sets of relevant documents but retrieve different sets of non-relevant documents" (Lee 1997). However, his coefficients R overlap and Noverlap are neither sufficient conditions nor necessary conditions for performance improvement. Later, Vogt (Vogt 1998) use a linear model to predict performance gain for different runs in TREC5. However, there are too many assumptions in his experimental settings and most are unjustified. It is still unclear in what condition could viewpoints enhance each other to generate a better result. And this question requires furthur research effort. Rcommon number of common relevant documentsR1 number of relevant documents in run1R2 number of relevant document in run2Ncommon number of common non relevant documentsN1 number of non relevant documents in run1N2 number of non relevant document in run2

So, we will compare our method with some of known others. There are: Classical methods: COMBSUM, COMBMIN, COMBMAX Probability methods: probFuseRandom method: random values that satisfied to merge properties.

Previous explaination such as Lee's rational for fusion states "different runs retrieve similar sets of relevant documents but retrieve different sets of non-relevant documents" (Lee 1997). However, his coefficients R overlap and Noverlap are neither sufficient conditions nor necessary conditions for performance improvement. Later, Vogt (Vogt 1998) use a linear model to predict performance gain for different runs in TREC5. However, there are too many assumptions in his experimental settings and most are unjustified. It is still unclear in what condition could viewpoints enhance each other to generate a better result. And this question requires furthur research effort. Rcommon number of common relevant documentsR1 number of relevant documents in run1R2 number of relevant document in run2Ncommon number of common non relevant documentsN1 number of non relevant documents in run1N2 number of non relevant document in run2

So, we will compare our method with some of known others. There are: Classical methods: COMBSUM, COMBMIN, COMBMAX Probability methods: probFuseRandom method: random values that satisfied to merge properties.