MPEG-7 DCD Based Relevance Feedback Using Merged Palette Histogram

Ka-Man Wong and Lai-Man Po

ISIMP 2004

Poly U, Hong Kong

Department of Electronic Engineering

City University of Hong Kong

MPEG-7 Dominant Color Descriptor

CQ

Dominant Color Descriptor

Original Image Color Quantized Image

CQ

Dominant Color Descriptor

Original Image Color Quantized Image

A compact and effective descriptor Generated by GLA color quantization Maximum of 8 colors in storage Each color have a minimum distance (Td) of 15 in CIELuv

color space

Commonly used RF techniques and limitations

Feature weighting relevance feedback technique Assumes a fixed feature space (histograms) Taking liner combinations on matching histogram bins. Simple approach: Histogram averaging

1'

k

jij

i

pH

k

+( ) / 2 =

Commonly used RF technique and limitations

But DCDs of images might have different set of colors, similar images might not have any exactly matched colors.

The number of colors in updated query may greatly exceed the limit of the number of colors defined by MPEG-7

Similar colors are separated. By definition of DCD, similar colors should be grouped together.

H1 2

1

H2 2

1

H’4

1

Limitation of feature weighting relevance feedback technique

The Merged Palette Histogram Relevance Feedback The updated query contains common

colors among selected images Represent the selected images

efficiently

Proposed Merged Palette Histogram for Relevance Feedback

Merged Palette Histogram Relevance Feedback

(MPH-RF) process - initialize Obtain all DCD from selected images

Proposed Merged Palette Histogram for Relevance Feedback

Merged Palette Histogram Relevance Feedback

(MPH-RF) process - 1 Link all DCD together

+ + =

6 colors 8 colors 6 colors 20 colors

Proposed Merged Palette Histogram for Relevance Feedback Merged Palette Histogram Relevance Feedback

(MPH-RF) process - 2 Palette Merging

Find the closest pair of colors based on Euclidian distance in CIELuv

If minimum distance smaller than Td merge the color pair and sum up the percentages of merged colors

Iterate until minimum distance > Td

1 1 2 2

1 1 2 2

1 1 2 2

i i j jm

i j

m

w p c w p cc

w p w p

p w p w p

20 colors 9 colors

Proposed Merged Palette Histogram for Relevance Feedback

Merged Palette Histogram Relevance Feedback

(MPH-RF) process - 3 Approximation

Cut the least significant colors if number of colors >8

9 colors 8 colors

Proposed Merged Palette Histogram for Relevance Feedback

Merged Palette Histogram Relevance Feedback

(MPH-RF) process - 4 Re-normalization

Adjust the histogram sum into 1 An updated query is generated

Approximated MPH Updated QueryHistogram Sum =1

Experimental Results of MPH-RF

Experiment Methodology ANMRR

Image Database 5466 Image from MPEG-7 common color dataset

(CCD) Pre-defined queries and ground truth set

Relevance Feedback Ground truth images are selected as relevant images

)(5.05.0)(

2

)(5.0

)(

)(

)(

)(

1

qNGqK

qNG

qNG

kRank

qNMRR

qNG

k

NQ

q

qNMRRNQ

ANMRR1

)(1

Latest experimental results MPH-RF gives improvement on both similarity

measure methods Combination of MPHSM and MPH-RF gives a

significant improvement Three iterations of relevance feedback give a

significant result*ANMRR – smaller means better

Initial After 3 RFRF

Improvement

DCD-MPHSM

0.2475 0.1838 0.0637

DCD-QHDM

0.2834 0.2117 0.0717

Experimental results Visual results – Query #50 from MPEG-7 CCD, MPHSM

Visit http://www.ee.cityu.edu.hk/~mirror for more results

Query image

Ground truth images

Initial retrieval, 4 of 8 ground truths hit, NMRR=0.5

First RF retrieval, 6 of 8 ground truths hit, NMRR=0.2782

Second RF retrieval, 7 of 8 ground truths hit, NMRR=0.1541

Experimental results Visual results – Query #24 from MPEG-7 CCD, QHDM

Visit http://www.ee.cityu.edu.hk/~mirror for more results

First RF retrieval, 6 of 12 ground truths hit, NMRR=0.3738

Query image

Ground truth images

Initial retrieval, 5 of 12 ground truths hit, NMRR=0.5125 Second RF retrieval, 9 of 12 ground truths hit, NMRR=0.1963

Conclusions on Merged Palette Histogram Relevance Feedback

A new MPH-RF for MPEG-7 DCD is proposed

MPH-RF generates a new DCD query using palette merging technique

Represents the selected relevant images naturally and effectively

Experiment result also found that proposed method improve DCD-QHDM by 0.0717 and MPHSM by 0.0637 using MPEG-7 Common Color Dataset

The proposed method also provide better perceptually relevant image retrieval.