selection of the proper compact composite descriptor for improving content based image retrieval
TRANSCRIPT
SELECTION OF THE PROPER COMPACT COMPOSITE DESCRIPTOR FOR IMPROVING CONTENT BASED IMAGE RETRIEVAL
Savvas Chatzichristofis, Mathias Lux and Yiannis Boutalis
Department of Electrical & Computer Engineering Democritus University of Thrace – GreeceInstitute of Information Technology ‐ Klagenfurt University Klagenfurt, Austria
Signal Processing, Pattern Recognition and Applications SPPRA 2009
Presenter: Savvas A. Chatzichristofis
• Compact Composite Descriptors (CCD) are global image descriptors capturing more than one feature at the same time, in a very compact representation.
Natural ImagesCEDDFCTH
Artificial ImagesSpCL
Medical ImagesBTDH
SELECTION OF THE PROPER COMPACT COMPOSITE DESCRIPTOR FOR IMPROVING CONTENT BASED IMAGE RETRIEVAL
Overview• In this paper we propose a combination of two
recently introduced CCDs (CEDD and FCTH) into a Joint Composite Descriptor (JCD).
• We further present a method for auto descriptor selection.
• Similar techniques were applied to select the most appropriate MPEG-7 descriptor, by extracting information from all the images of a dataset.
SELECTION OF THE PROPER COMPACT COMPOSITE DESCRIPTOR FOR IMPROVING CONTENT BASED IMAGE RETRIEVAL
CEDD and FCTH Descriptors• The CEDD length is 54 bytes per image while FCTH
length is 72 bytes per image.
• The structure of these descriptors consists of n texture areas. In particular, each texture area is separated into 24 sub regions, with each sub region describing a color.
• CEDD and FCTH use the same color information, as it results from 2 fuzzy systems that map the colors of the image in a 24-color custom palette.
SELECTION OF THE PROPER COMPACT COMPOSITE DESCRIPTOR FOR IMPROVING CONTENT BASED IMAGE RETRIEVAL
CEDD and FCTH Descriptors
SELECTION OF THE PROPER COMPACT COMPOSITE DESCRIPTOR FOR IMPROVING CONTENT BASED IMAGE RETRIEVAL
CEDD and FCTH Descriptors
0 1 2 3 4 5 6 7
CEDD
Linear
Non
Directional
Horizontal
Activation
VerticalA
ctivation
45 Degree
Diagonal
135 Degree
Diagonal
- -
FCTH
Linear L
owE
nergy
Horizontal L
owE
nergy
Vertical Low
Energy
Both D
irectionsL
ow E
nergy
Linear
High E
nergy
Horizontal
High E
nergy
Vertical High
Energy
Both D
irectionsH
igh Energy
SELECTION OF THE PROPER COMPACT COMPOSITE DESCRIPTOR FOR IMPROVING CONTENT BASED IMAGE RETRIEVAL
CEDD and FCTH Descriptors
WANG UCID NISTERCCDCEDD 0.25283 0.28234 0.11297FCTH 0.27369 0.28737 0.09463MPEG-7DCD MPHSM 0.39460 - -DCD QHDM 0.54680 - -SCD 0.35520 0.46665 0.36365CLD 0.40000 0.43216 0.2292CSD 0.32460 - -EHD 0.50890 0.46061 0.3332HTD 0.70540 - -
SELECTION OF THE PROPER COMPACT COMPOSITE DESCRIPTOR FOR IMPROVING CONTENT BASED IMAGE RETRIEVAL
Joint Composite Descriptor (JCD)• Based on the fact that the color information
given by the 2 descriptors comes from the same fuzzy system, we can assume that joining the descriptors will result in the combining of texture areas carried by each descriptor.
• JCD is made up of 7 texture areas, with each area made up of 24 sub regions that correspond to color areas.
SELECTION OF THE PROPER COMPACT COMPOSITE DESCRIPTOR FOR IMPROVING CONTENT BASED IMAGE RETRIEVAL
Joint Composite Descriptor (JCD)• The texture areas are as follows:
▫ JCD(0) Linear Area▫ JCD(1) Horizontal Activation▫ JCD(2) 45 Degrees Activation▫ JCD(3) Vertical Activation▫ JCD(4) 135 Degrees Activation▫ JCD(5) Horizontal and Vertical Activation▫ JCD(6) Non directional Activation
SELECTION OF THE PROPER COMPACT COMPOSITE DESCRIPTOR FOR IMPROVING CONTENT BASED IMAGE RETRIEVAL
Descriptor Implementation• We model the problem as follows:• CEDD and FCTH be available for an image. The
indicator m symbolises the bin of the color of each descriptor.
• The indicators n and n’ determine the texture area for the CEDD and FCTH respectively
CONTENT BASED MEDICAL IMAGE INDEXING AND RETRIEVAL USING A FUZZY COMPACT COMPOSITE DESCRIPTOR
[0, 23]m∈
[0,5]n∈ ' [0,7]n ∈
SELECTION OF THE PROPER COMPACT COMPOSITE DESCRIPTOR FOR IMPROVING CONTENT BASED IMAGE RETRIEVAL
Descriptor Implementation• Each descriptor can be described in the
following way:
'( ) , ( )m mn nCEDD j FCTH j
The algorithm for the Joint Composite Descriptor can be analysed as follows:
54( ) (2 24 5) (53)CEDD j bin bin= × + =
SELECTION OF THE PROPER COMPACT COMPOSITE DESCRIPTOR FOR IMPROVING CONTENT BASED IMAGE RETRIEVAL
Descriptor Implementation
SELECTION OF THE PROPER COMPACT COMPOSITE DESCRIPTOR FOR IMPROVING CONTENT BASED IMAGE RETRIEVAL
Auto Descriptor Selection (ADS)
• (i) The descriptor for search is chosen based on the query image.
• (ii) The most appropriate descriptor is chosen at similarity assessment time, so within a single query the chosen descriptor may be different for different image pairs.
SELECTION OF THE PROPER COMPACT COMPOSITE DESCRIPTOR FOR IMPROVING CONTENT BASED IMAGE RETRIEVAL
Auto Descriptor Selection (ADS)• In retrieval scenarios a
combination of different feature spaces within a single query is often not possible.
• Experiments on the Wang data set have shown that with normalized similarities (mean of 0 and standard derivation of 1) distributions are similar enough to be combined.
SELECTION OF THE PROPER COMPACT COMPOSITE DESCRIPTOR FOR IMPROVING CONTENT BASED IMAGE RETRIEVAL
Distribution of (a) CEDD, (b) FCTH and (c) JCD similarities / Wang 1000 image
database.
Auto Descriptor Selection (ADS)• Given that the color information in all two
descriptors is the same, the factor that will determine the suitability and capability of each descriptor is mainly found in the texture information.
• The system that determines the most appropriate descriptor is a Mamdani fuzzy system of three inputs and one fuzzy output. The centroid method was used to defuzzify the output of the Mamdanimodel.
SELECTION OF THE PROPER COMPACT COMPOSITE DESCRIPTOR FOR IMPROVING CONTENT BASED IMAGE RETRIEVAL
Criterion 1: Maximum amount of information.
• The first criterion shows which CCD contains the largest quantity of information.
SELECTION OF THE PROPER COMPACT COMPOSITE DESCRIPTOR FOR IMPROVING CONTENT BASED IMAGE RETRIEVAL
Criterion 2: Percentage of information in non-uniform texture areas.
• The most appropriate descriptor is the one that contains the smallest percentage of non uniform image blocks.
SELECTION OF THE PROPER COMPACT COMPOSITE DESCRIPTOR FOR IMPROVING CONTENT BASED IMAGE RETRIEVAL
Criterion 3: The percentage of information in texture areas.
• The third criterion considers the most appropriate descriptor to be the one that has the smallest percentage of image blocks present in linear areas.
SELECTION OF THE PROPER COMPACT COMPOSITE DESCRIPTOR FOR IMPROVING CONTENT BASED IMAGE RETRIEVAL
Experiments• The proposed methods have been implemented and are available as
open source libraries under GNU - General public License (GPL) in the image retrieval system img(Rummager) the on line application img(Anaktisi) and image retrieval library LIRe.
SELECTION OF THE PROPER COMPACT COMPOSITE DESCRIPTOR FOR IMPROVING CONTENT BASED IMAGE RETRIEVAL
•• CEDD• FCTH• JCD• Ranking
SELECTION OF THE PROPER COMPACT COMPOSITE DESCRIPTOR FOR IMPROVING CONTENT BASED IMAGE RETRIEVAL
For use of multiple different descriptors within one query, the ADS unit also needs to normalize the similarities based on their distribution. Based on
experiments we used the normalization values given in paper.
Experiments• To evaluate the performance of the proposed methods, the objective measure called ANMRR is used.
WANG UCID NISTERCEDD 0.25283 0.28234 0.11297
FCTH 0.27369 0.28737 0.09463
JCD 0.25606 0.26832 0.085486
ADSBased on Query descriptor 0.24948 0.27952 0.09291
ADSBased on Pair wise descriptor 0.24876 0.27722 0.09291
SELECTION OF THE PROPER COMPACT COMPOSITE DESCRIPTOR FOR IMPROVING CONTENT BASED IMAGE RETRIEVAL
Conclusions• JCD and ADS methods are not suggested to improve the
retrieval procedure.
• The goal is to approach the best ANMRR that would result from CEDD and FCTH.
• Nevertheless, the new JCD shows an increase in retrieval performance.
• The methods for automatic selection of the most appropriate descriptor (ASD) for retrieval increases retrieval performance in all 3 experiments.
SELECTION OF THE PROPER COMPACT COMPOSITE DESCRIPTOR FOR IMPROVING CONTENT BASED IMAGE RETRIEVAL