Image indexing and retrieving using histogram based methods

03/7/1503/7/15

資工研所資工研所陳慶鋒陳慶鋒

Outline

Histogram based methodsHistogram based methods Image retrieval using the three methodsImage retrieval using the three methods Experimental resultExperimental result Library of Image formatsLibrary of Image formats Future workFuture work ReferencesReferences

Histogram based features Color HistogramColor Histogram Histogram RefinementHistogram Refinement Color CorrelogramColor Correlogram

Color histogram

For a For a nnnn with with mm colors image colors image II,,

the color histogram is the color histogram is

wherewhere

pp 為屬於為屬於 II 的的 pixel, pixel, I(p)I(p) 為其顏色為其顏色

, , ,,forfor

Color histogram (cont.)

AdvantagesAdvantages

-trivial to compute-trivial to compute

-robust against small changes in camera -robust against small changes in camera

viewpointviewpoint DisadvantagesDisadvantages

-without any spatial information-without any spatial information

Histogram refinement

The pixels of a given bucket are subdivided The pixels of a given bucket are subdivided into classes based on local feature. Within a into classes based on local feature. Within a given bucket , only pixels in the same class given bucket , only pixels in the same class are compared.are compared.

The local feature which this paper used:The local feature which this paper used:

Color Coherence Vectors(CCVs)Color Coherence Vectors(CCVs)

Histogram refinement (cont.)

CCVsCCVs

For the discretized color For the discretized color ccii, the pixels with color , the pixels with color ccii

are coherence if the number of connected are coherence if the number of connected component>= component>= , indicated as , indicated as cici, otherwise are , otherwise are

incoherence, indicated as incoherence, indicated as cici, and total pixel with , and total pixel with

color color ccii = = cici+ + cici, , a threshold a threshold is defined as the is defined as the

condition of coherence or notcondition of coherence or not

for color for color jj, the coherence pair is (, the coherence pair is (cici,, cici) )

Histogram refinement (cont.)

Histogram refinement (cont.)

ExampleExample

Histogram refinement (cont.)

Example(cont.)Example(cont.)

Histogram refinement (cont.)

Example(cont.)Example(cont.)

Lable A B C D E

Color 1 2 1 3 2

Size 12 15 3 1 5

Histogram refinement (cont.)

Example(cont.)Example(cont.)

Color 1 2 3

α 12 20 0

β 3 0 1

Color correlograms

A table indexed by color pairs, where the A table indexed by color pairs, where the kk-th entry for -th entry for color pair color pair <i, j><i, j> specifies the probability of finding a pixel specifies the probability of finding a pixel of color of color jj at a distance at a distance kk from a pixel of color from a pixel of color ii in the image. in the image.

The correlogram isThe correlogram is

dd …… ……

…… …… …… …… ……

11 ……

(1,1)(1,1) (1,2)(1,2) ………… ((m,m)m,m)

1 0 0 1 1 11 1 1

Color correlograms(cont.)

The autocorrelogram isThe autocorrelogram is dd ……

…… …… …… ……

11 ……

11 …… mm

Color correlograms (cont.)

ExampleExample

Color correlograms (cont.)

Example(cont.)Example(cont.)

Color correlograms (cont.)

Example(cont.)Example(cont.)

Image retrieval using the three methods Similarity measureSimilarity measure -L1 distance similarity-L1 distance similarity

-relative distance-relative distance Performance measurePerformance measure -ranking measure-ranking measure

Similarity measure

L1 distance similarity L1 distance similarity Sim()Sim()

Sim(I,I’)Sim(I,I’) 愈大，兩張圖的相似度愈高愈大，兩張圖的相似度愈高

Similarity measure(cont.)

Relative distanceRelative distance

愈小愈小，，兩張圖的相似度愈高兩張圖的相似度愈高

Performance measure

Ranking measuresRanking measures令 為令 為 query imagesquery images 的集合，的集合， Q’Q’ii 為為 QQii 的的 answer imageanswer image

rr-measure: -measure:

average average rr-measure:-measure:

pp11-measure: -measure:

average average pp11-measure:-measure:

Experimental result

Experimental setupExperimental setup

--Image database of 180 gray level images with size Image database of 180 gray level images with size

192x128192x128

-Quantize gray level to 16 bins-Quantize gray level to 16 bins

-Set -Set of CCV as 1500 of CCV as 1500

-Set -Set d d of autocorrelogram as 30of autocorrelogram as 30

-A query set which consists 25 query images and 25 -A query set which consists 25 query images and 25 answer imagesanswer images

Experimental result(cont.)

ResultsResults

similarity hist: 1 ccv: 1 auto: 1 similarity hist: 1 ccv: 1 auto: 1 relative distance hist: 1 ccv: 1 auto: 1relative distance hist: 1 ccv: 1 auto: 1

similarity hist: 32 ccv: 26 auto: 44 similarity hist: 32 ccv: 26 auto: 44 relative distance hist: 33 ccv: 38 auto: 31relative distance hist: 33 ccv: 38 auto: 31

Experimental result(cont.)

Results(cont.)Results(cont.)

similarity hist: 41 ccv: 11 auto: 77similarity hist: 41 ccv: 11 auto: 77 relative distance hist: 10 ccv: 3 auto: 7relative distance hist: 10 ccv: 3 auto: 7

similarity hist: 55 ccv: 26 auto: 80similarity hist: 55 ccv: 26 auto: 80 relative distance hist: 2 ccv: 10 auto: 1relative distance hist: 2 ccv: 10 auto: 1

Experimental result(cont.)

Results(cont.)Results(cont.) performance measure in performance measure in similaritysimilarity and and relative distancerelative distance

Similarity Color histogram ccv auto

r-measure 266 203 387

avg r-measure 10.64 8.12 15.48

p1-measure 15.27 14.53 15.57

avg p1-measure 0.61 0.58 0.62

Relative distance Color histogram ccv auto

r-measure 155 185 133

avg r-measure 6.2 7.4 5.32

p1-measure 15.77 14.68 16.54

avg p1-measure 0.63 0.59 0.66

Experimental result(cont.)

Results(cont.)Results(cont.) performance measure in performance measure in similaritysimilarity and and relative distancerelative distance

Similarity Color histogram ccv auto

r-measure 266 203 387

avg r-measure 10.64 8.12 15.48

p1-measure 15.27 14.53 15.57

avg p1-measure 0.61 0.58 0.62

Relative distance Color histogram ccv auto

r-measure 155 185 133

avg r-measure 6.2 7.4 5.32

p1-measure 15.77 14.68 16.54

avg p1-measure 0.63 0.59 0.66

Experimental result(cont.)

Factors which affect performance Factors which affect performance - choice of image database- choice of image database

- choices between query images and answer images- choices between query images and answer images

- - of CCV of CCV

- - dd of color autocorrelogram of color autocorrelogram

Library of Image formats

Include: imgdata.hInclude: imgdata.h Formats: pgm, jpg, png, bmpFormats: pgm, jpg, png, bmp We can get: width, height, and raw dataWe can get: width, height, and raw data

Library of Image formats(cont.)

FunctionsFunctions

GetPGM(char, int*, int*, unsigned char**)GetPGM(char, int*, int*, unsigned char**)

GetPNG(char, int*, int*, unsigned char**)GetPNG(char, int*, int*, unsigned char**)

GetBMP(char, int*, int*, unsigned char**)GetBMP(char, int*, int*, unsigned char**)

GetJPEG(char, int*, int*, unsigned char**)GetJPEG(char, int*, int*, unsigned char**)

Library of Image formats(cont.)

ExampleExample

int width, heightint width, height

unsigned char* dataunsigned char* data

GetJPEG(“1.jpg”, &width, &height, &data)GetJPEG(“1.jpg”, &width, &height, &data)

Future work

Image indexing and retrieving of color Image indexing and retrieving of color images (debugging)images (debugging)

Further study Further study

References [1] [1] M. Swain and D. Ballard, “Color indexing,” International Journal of M. Swain and D. Ballard, “Color indexing,” International Journal of

Computer Visioin, 7(1):11-32, 1991Computer Visioin, 7(1):11-32, 1991 [2][2] G. Pass and R.Zabih, “Histogram refinement for content based G. Pass and R.Zabih, “Histogram refinement for content based

image retrieval,” IEEE Workshop on Applications of Computer Vision, image retrieval,” IEEE Workshop on Applications of Computer Vision, pp.96-102, 1996pp.96-102, 1996

[3] G Pass and R. Zabih, “Compare images using color coherence [3] G Pass and R. Zabih, “Compare images using color coherence vectors,” Applications of Computer Vision, 1996. WACV '96., vectors,” Applications of Computer Vision, 1996. WACV '96., Proceedings 3rd IEEE Workshop on , 2-4 Dec 1996 , Page(s): 96 -102Proceedings 3rd IEEE Workshop on , 2-4 Dec 1996 , Page(s): 96 -102

[4] J. Huang, S. R. Kumar, M. Mitra, W. J. Zhu, and R.Zabih, “Image [4] J. Huang, S. R. Kumar, M. Mitra, W. J. Zhu, and R.Zabih, “Image indexing using color correlograms,” Conf. Computer Vision and Pattern indexing using color correlograms,” Conf. Computer Vision and Pattern Recognit., pp.762-768,1997Recognit., pp.762-768,1997