1

MPEG-7 DCD using Merged Palette Histogram Similarity Measure

Lai-Man Po and Ka-Man Wong

ISIMP 2004Oct 20-22, Poly U, Hong Kong

Department of Electronic EngineeringCity University of Hong Kong

2

MPEG-7 Dominant Color Descriptor

CQ

Dominant Color Descriptor

Original Image Color Quantized Image

CQ

Dominant Color Descriptor

Original Image Color Quantized Image

A compact and effective descriptor Generated by GLA color quantization Maximum of 8 colors in storage

3

Dominant Color Descriptor Similarity measure

A modified Quadratic Histogram Distance Measure (QHDM) Since each DCD may have different set of colors, QHDM is

used to account for identical colors and similar colors.1 2 1 2

2 2 21 2 1 2 1 ,2 1 2

1 1 1 1

( , ) 2 N N N N

QHDM i j i j i ji j i j

D F F p p a p p

Percentage p

color

Percentage q

color

4

DCD-QHDM upper bound problem Limitations of QHDM - 1

Distance upper bound is varied by number of matching colors Completely different image cannot be identified by its upper bound

F2

1/2

I2

F31/3

I3

F1

1/2

I1

F1

1/2

I1

2 21 2 1 3( , ) 0.81389 > ( , ) 0.7093QHDM QHDMD F F D F F

5

DCD-QHDM upper bound problem Analysis of problem 1

The upper-bound of the distance measured varies by number of color in the descriptor

Maximum of positive part is not a constant Maximum of negative part is zero So, the maximum of QHDM result is not fixed This property makes DCD unable to identify completely different images

by the values measured

Positive part Negative part

1 2 1 22 2 2

1 2 1 2 1 ,2 1 21 1 1 1

( , ) 2 N N N N

i j i j i ji j i j

D F F p p a p p

6

DCD-QHDM Similarity coefficient problem Limitations of QHDM - 2

The similarity coefficient does not well model color similarity It does not balance between color distance and area of matching

I4

F4

1

F2

1/2

I2

F1

1/2

I1

F1

1/2

I1

2 21 2 1 4( , ) 0.81389 > ( , ) 0.5

QHDMQHDMD F F D F F

7

DCD-QHDM Similarity coefficient problem The similarity coefficient use the color distance to fine tune the

similarity Difficult to define a quantitative similarity between colors,

Sensitivity of human eye depends on many conditions (e.g. light source of the room, spatial layout of the image, etc.)

1 2 1 2

1 ,2

1 2

1 / ,

0,

i j d i j d

i j

i j d

c c T c c Ta

c c T

a = 16.67%

a = 44%

Td

a = 0%

1.2

d

8

Proposed Merged Palette Histogram Similarity Measure MPHSM Process - 1

Find the closest pair of colors using Euclidian distance in CIELuv color space

MPHSM process - 2 If the distance smaller than a threshold Td, merge them to form a

new common palette color

2 2 21 2 1 2 1 2 1 2( , ) ( ) ( ) ( ) d C C l l u u v v

1 1 2 2( , )

1 2

i i j jm i j

i j

p c p cc

p p

Common Palette

9

Proposed Merged Palette Histogram Similarity Measure MPHSM process - 3

A new common palette is then generated Form new descriptors based on the common palette

Dominant Color Descriptor

Common Palette

Merged Palette Histogram

10

Proposed Merged Palette Histogram Similarity Measure MPHSM process - 4

Histogram intersection is used to measure the similarity Count the non-overlapping area as the distance

1 2 1 2 1 21 1

1( , ) 1 min( , )

2

m mN N

m m mi mi mi mii i

D F F p p p p

11

Flow of MPHSMInitial DCDs

Step 1: Find a pair of colors with minimum

distance d

d<Td ?Step2: Merge colors having

minimum distance

Common Palette

N

Y

Step 3: Update each DCD basedon the common palette

Step 4: Histogram Intersection

12

Experiment Result of MPH-RF Experiment Methodology

ANMRR

Image Database 5466 Images from MPEG-7 common color dataset (CCD) 50 Pre-defined query and ground truth sets

)(5.05.0)(

2

)(5.0

)(

)(

)(

)(

1

qNGqK

qNG

qNG

kRank

qNMRR

qNG

k

NQ

q

qNMRRNQ

ANMRR1

)(1

13

Latest experimental results MPHSM without spatial coherence improves DCD by about 0.0

4 of ANMRR in average Very close to QHDM with spatial coherence

Significant improve in medium queries It gives significant improvement on visual results

*ANMRR (smaller means better)

QHDM QHDM-SC MPHSM MPHSM-SC

easy queries NMRR<0.2 (17)

0.0357 0.0267 0.0434 0.0363

hard queries NMRR > 0.4 (12)

0.5317 0.4754 0.5801 0.5571

medium queries (21)

0.3420 0.2862 0.2227 0.2040

average queries (50) 0.2834 0.2434 0.2475 0.2317

14

Experimental results Visual results - Query #32 from MPEG-7 CCD

Demo available in http://www.ee.cityu.edu.hk/~mirror/

Query image

QHDM results, ANMRR=0.4 MPHSM result, ANMRR=0.0111

15

Experimental results Visual results - Query #25 from MPEG-7 CCD

Demo available in http://www.ee.cityu.edu.hk/~mirror/

Query image

QHDM results, ANMRR=0.3935 MPHSM result, ANMRR=0.0481

16

Conclusion A new merged palette histogram similarity measure

for dominant color descriptor of MPEG-7 is proposed

The merged palette formed a common color space and used to redefine the new query histograms for histogram intersection similarity measure.

Can match identical colors as well as similar colors

Use area of matching for similarity measure

17

Conclusion Experimental results show that the proposed MPHSM

improve DCD-QHDM using ANMRR rating by about 0.04 and very close to the result of DCD-QHDM with spatial coherence

Our experiment result also found that the result of proposed method can be further improved by spatial coherence

The proposed method also provide better perceptually relevant image retrieval.