young deok chun, nam chul kim, member, ieee, and ick hoon jang, member, ieee ieee transactions on...
Post on 19-Dec-2015
215 views
TRANSCRIPT
Content-Based Image Retrieval Using Multiresolution Color and Texture Features
Young Deok Chun, Nam Chul Kim, Member, IEEE, and Ick Hoon Jang, Member, IEEE
IEEE TRANSACTIONS ON MULTIMEDIA,OCTOBER 2008
Outline
Introduction Conventional features Proposed image retrieval method Experimental results Conclusion
Introduction
Typical CBIR Extract features related to visual content from a
query image Compute similarity between the features of the
query image and target images in DB Target images are next retrieved which are most
similar to the query image Extraction of good features is one of the
important tasks in CBIR Shape▪ Describes the contours of objects in an image▪ Usually extracted from segmenting the image into
meaningful regions or objects
Introduction
Color▪ Most widely used visual features ▪ Invariant to image size and orientation
Texture▪ Visual feature that refers to innate surface properties of an object ▪ Relationship to the surrounding environment
A feature extracted from an image is generally represented as a vector of finite dimension Feature vector dimension is one of the most important
factors that determine▪ The amount of storage space for the vector▪ Retrieval accuracy▪ Retrieval time (or computational complexity)
Conventional features
Explain the conventional features which are used in the proposed retrieval method color feature▪ color autocorrelogram
texture features▪ BDIP ▪ BVLC
Color Autocorrelogram
Color autocorrelogram Captures the spatial correlation between identical colors
only Probability of finding a pixel p’ of the identical color at a
distance k from a given pixel p of the lth color
measure the distance between pixels
As k varies, spatial correlation between identical colors in an image can be obtained in various resolutions
IlIpforlIpandkppIplk ''' |Pr
1,.....1,0 LI
''' ,max yyxxpp
BDIP
Block Difference of Inverse Probabilities Texture feature that effectively extracts
edges and valleys
Object boundaries are extracted well , and the intensity variation in dark regions is emphasized
yxI
yxIyxIB
lkl
kl
kl
Byx
Byx Byxklk
,max
,,max1
,
, ,
Block size (k+1)*(k+1)
Intensity of pixel(x,y)
Representative (maximum) of intensity variation in a block
Representative value in a block
Original image BDIP operator
BVLC
Block Variation of Local Correlation coefficients Represents the variation of block-based local
correlation coefficients according to four orientations
Local correlation
BVLC
0,,,0,,0,0,4 kkkkO
klkll k
Ok
k
Ok
k
,min,max44
kll
Byx kllyxklk
kl
kykxIyxIB
kl
,
,,1
,standard deviation ofthe lth block shifted by k
Local covariance normalized by local variance
Difference between the max and min values of block-based local correlation coefficients (four orientations)
Proposed image retrieval method
Overall structure
A texture feature vector ft ofdimension Nt with BDIP and BVLC moments extracted fromV
nmW ,
Each component image is waveletdecomposed into a wavelet image
4-band wavelet decomposition
configuration of 2-level wavelet decomposed images
A color feature vector fc of dimension Nc is then formed with color autocorrelograms extracted from the and
HnmW ,
SnmW ,
Color Feature Extraction
Color Feature Extraction
The and are first quantized into and LL band▪ uniformly quantized
other subbands ▪ non-uniformly quantized by the generalized Lloyd
algorithm Given the total number of quantization levels L of
all subbands, Li is allocated to the ith subband
HnmW ,
SnmW ,
HnmQ ,
SnmQ ,
1
0
K
iiLL KK
j j
ii K
LL 1
1
0
2
2
22
2 log2
1loglog
Color Feature Extraction
Numbers of quantization levels for all subbands are decided
Color autocorrelogram is extracted To reduce computational complexity, we modify the color
autocorrelogram
Cnm
Cnm
Cnm
Cnmk
nmQlQpforlQpand
pNpandkppQplC
,,,'
2''
,'
, Pr,
ZnHHLHHLLLm
LlSHC nm
,...,1,,,,
,1,...,1,0,, ,
the set of pixels having the lth color
Color Feature Extraction
The color feature vector fc is finally formed with the color autocorrelogram
color autocorrelogram probability of finding a pixel p’ of the lth color among
the two causal neighboring pixels at a distance k from a given pixel p of the lth color in the
lCf knmc ,,
ZnHHLHHLLLm
LlSHC nm
,...,1,,,,
,1,...,1,0,, ,
CnmQ ,
Texture Feature Extraction
Texture Feature Extraction
is first divided into nonoverlapping blocks of a given size, where the BDIP and BVLC are computed
BDIP the denominator in may yield negative BDIP values in
wavelet domain, which leads to invalid measurement of intensity variation
yxW Vnm ,,
yxW
yxWyxWB
lVnLLByx
Byx
Vnm
VnmByxk
lk
kl
kl
kl
,max
,,max1
,,
, ,,,
ZnHHLHHLLLm ,...,1,,,, Pixel values are nonnegative in LL band
Texture Feature Extraction BVLC
Reduce computational complexity▪ Local covariance▪ Mean absolute difference of pixels between two
blocks
▪ Local variance ▪ Mean absolute difference of four end pixels in the
block
2
,,1
,, ,,
,kll
Byx yxVnm
Vnmk
lknm
kl
kykxWyxWB
kl
kykxWkyxWykxWyxW
kykxWykxWkyxWyxW
Vnm
Vnm
Vnm
Vnm
Vnm
Vnm
Vnm
Vnm
l,,,,
,,,,
4
1
,,,,
,,,,
Texture Feature Extraction Modified BVLC
Difference between the max and min values of the local correlation coefficients according to two orientations
klkll k
Ok
k
Ok
knm
,min,max
22,
kkO ,0,0,2
Texture Feature Extraction The first and second moments of the BDIP
and BVLC for each subband are extracted
The texture feature vector
2,,,2
,,
2
,,,2
,,
knm
knm
knm
knm
knm
knm
knm
knm
knm
knm
l
l
l
l
knm
knm
knm
knmTf ,,,, ,,,
ZnHHLHHLLLm ,...,1,,,,
Feature Vector Combination and Similarity Measurement
Feature Vector Combination After color and texture feature vectors are extracted,
the retrieval system combines these feature vectors
Each of the color and texture feature components is normalized by its dimension and standard deviation reducing the effect of different feature vector dimensions and
component variances in the similarity computation
knm
knm
knm
knmTf ,,,, ,,,
lSlHf knm
knmc ,,, ,,
ZnHHLHHLLLm ,...,1,,,,
TT
T
CC
C
N
f
N
ff
,
Similarity Measurement
Use generalized Minkowski-form distance of metric order one
Feature dimension Color feature
▪ NC : determined as the total number of quantization levels L
Texture feature▪ NT : 16Z
N= NC + NT The number of additions for a query image in the
similarity measurement of the retrieval is given as
N
i
tqtq ififffD1
,
120 NK
Similarity Measurement
Huge Database Progressively implement▪ reduce the computational complexity in CBIR
Set of candidate images is selected by feature matching at the lowest level
Progressive refinement is performed as the level increases
Progressive retrieval is composed of Z+1 steps▪ First step▪ the color feature vector fC and the texture feature vector fT are combined for
m=LL and n=1
▪ Second step▪ m={LL,HL,LH,HH} and n=1
▪ Z+1 step▪ m={LL,HL,LH,HH} and n={1,….,Z}
Similarity Measurement
At each step (jth step) query ▪ the combined feature vector fj
q of dimension Nj
Target▪ the combined feature vector fj
t of the same dimension
▪ for each of kj-1 target images
kj target images with the best similarity are retrieved Total number of additions for a query in the similarity
measurement of the progressive retrieval
where kj is determined to decrease and the Nj increases as the retrieval step j increases
Z
jjj NK
01 12
EXPERIMENTAL RESULTS
Image Database The Corel DB▪ 990 RGB color images▪ 192 x 128 pixels▪ 11 classes, each of 90 images
VisTex DB ▪ 1200 RGB color images▪ 128 x 128 pixls▪ 75 classes, each of 16 images
MPEG-7 common color dataset (CCD) ▪ 5420 color images▪ 332 ground truth sets (GTS)
where the number of images in each GTS varies
EXPERIMENTAL RESULTS
Corel MR DB▪ directly from a third of all the images for each class in the
Corel DB▪ Ratio of (1.5:1)▪ Ratio of (2:1)
VisTex MR DB▪ Ratio of (1:1),(1.5:1),(1.75:1),(2:1)
MPEG-7 CCD MR DB▪ Ratio of (1:1),(1.5:1),(2:1)
The sizes and numbers of classes of the three derived DBs are the same as those of the original DBs
Contain images of various resolutions
EXPERIMENTAL RESULTS
Performance Measures For a query q
A(q) : a set of retrieved images in a DB B(q) : images relevant to the query q
Precision
Recall
ANMRR (average normalized modified retrieval rank)▪ measure of retrieval accuracy used in almost all of the MPEG-7 color
core experiments▪ The ANMRR gives just one value for a DB▪ Lower ANMRR value means more accurate retrieval performance
qA
qBqAqP
qB
qBqAqR
EXPERIMENTAL RESULTS
Specifications of Retrieval Methods wavelet decomposition level was chosen as Z=2 total number of quantization levels L for each of the wavelet
decomposed H and S component images was also chosen as L=30▪ {Lm,1} = {8,4,4,4} , {Lm,2} = {4,2,2,2}
Dimensions of feature vector▪ NC = 60
▪ NT = 32
Z=2 , Proposed progressive retrieval has 3 steps▪ At the step of j=1,2,3 , the total number of quantization levels for
each of the wavelet decomposed H and S component images is given as L = 8,20, and 30
feature vector dimensions▪ (N1,N2,N3) =(20,56,92)
EXPERIMENTAL RESULTS
The proposed method with nonprogressive scheme and that with progressive scheme
(a) Corel DB (b) VisTex DB.
former yields the average precision loss of 1.5% and that of 1.1% over the latter for Corel DB and for VisTex DB
Proposed retrieval method with progressive scheme is about 1.2 times faster than nonprogressive scheme
EXPERIMENTAL RESULTS
Precision versus recall of the proposed method with progressive scheme according to each step
(a) Corel DB (b) VisTex DB.
EXPERIMENTAL RESULTS
Single features and the proposed progressive retrieval method
Combination of color and texture features and the proposed progressive retrieval method
EXPERIMENTAL RESULTS
ANMRR of the retrieval methods
The proposed method almost always yields better performance in precision versus recall and in ANMRR over the other methods for the six test DBs
EXPERIMENTAL RESULTS
Retrieval ranks of the relevant images for the query image
Query imageResolution is identical with the query image
The proposed method is more effective for multiresolution image DBs
Conclusion
The feature vector is scalable according to the decomposition level Z in the wavelet transform domain It was found in some experiments that the retrieval
accuracies of Z>2 are slightly better than those of Z=2
Experimental results for six test DBs showed that the proposed method yielded higher retrieval accuracy than the other conventional methods
It was all the more so for multiresolution image Databases