Integration of Feature Distributions for Colour Texture Segmentation

Abstract

This paper proposes a new framework for colour texturesegmentation and determines the contribution of colour andtexture. The distributions of colour and texture features pro-vides the discrimination between different colour texturedregions in an image. The proposed method was tested usingdifferent mosaic and natural images. From the results, itis evident that the incorporation of colour information en-hanced the colour texture segmentation and the developedframework is effective.

1. Introduction

Texture and colour are the two important innate featuresthat can be employed to detect the colour texture variationsin an image. Segmentation was based on grey scale im-ages until recently, when the concern changed to the jointrepresentation of texture and colour for colour texture seg-mentation. The main objective of this paper is to develop aframework for colour texture segmentation and to determinethe use of colour information in the colour texture segmen-tation process.In this paper, the colour and texture features were extractedseparately and combined for colour texture segmentation.The grey scale algorithms were employed in the luminanceplane with colour information as an extra cue. The devel-oped colour texture segmentation method integrates colourand texture by using their feature distributions. Chen et al.[1] proposed a method using the distributions of colour andlocal edge patterns which is used to derive a homogeneitymeasure for colour texture segmentation. Jain and Healey[4] introduced a method for colour texture classificationbased on unichrome features computed from the three spec-tral bands independently and opponent colour features thatutilise the spatial correlation between spectral bands. Drim-barean and Whelan [2] examined the contribution of colourinformation to the overall classification performance by ex-tracting texture features and colour features. It was foundthat the inclusion of colour increases the classification re-sults without significantly complicating the feature extrac-tion algorithms. Pietiekainen et al. [8] presented a colour

texture classification based on separate processing of colourand pattern information. From the classification results itwas concluded that colour and texture have complementaryroles.In this study, Local Binary Pattern (LBP) [5] is used asthe texture feature extraction technique. The LBP operatoralone cannot properly detect large scale textural structuresthat exists in real world applications. In addition, LBP is notinvariant to rotation. The limitation of LBP in the segmenta-tion procedure was overcome by combining the colour fea-tures. Colour features were derived based on k-means algo-rithm which clusters the input image by organising the datainto k different clusters. The distributions of LBP and thedistributions of colour clustered labels were combined forcolour texture segmentation. This unification of colour andtexture increases the efficiency of colour texture segmenta-tion. Any of the available feature extraction techniques canbe used in this framework.

2. Feature Distributions

LBP is invariant against any monotonic grey scale trans-formation. This provides robust pattern related informationand knowledge about the spatial structure of the local imagetexture. LBP is combined with the contrast of the texturewhich is a measure of local variations present in an imagefor the texture description. The distributions of LBP andcontrast (256 ∗ 8) were used for texture description.This study uses the unsupervised clustering technique basedon the k-means algorithm to cluster the colour features.The k-means algorithm [3] is the simplest and most popu-lar among the iterative clustering algorithms. The k-meansalgorithm organises the objects into an efficient representa-tion that characterises the population being sampled. Thenumber of clusters is generally image dependent so the ini-tial guess is 10 clusters, this number is sufficient to captureall the relevant clusters. The distribution of the colour clus-ters is used for colour description.

Modified Kolmogorov Smirnov (M-KS) A non-parametric test M-KS statistic was used for comparingLBP/C with colour clustered labels. This tests the hy-

pothesis that two empirical feature distributions havebeen generated from the same population. M-KS has thedesirable property that it is invariant to arbitrary monotonicfeature transformations [9]. The M-KS statistic is definedas the sum of the absolute value of the discrepanciesbetween the normalised cumulative distributions.

D(s,m) =∑

i

∣

∣

∣

∣

∣

Fs(i)

ns

−

Fm(i)

nm

∣

∣

∣

∣

∣

(1)

where Fs(i) and Fm(i) represents the sample cumulativedistribution functions; ns and nm represents the number ofpixels in the sample regions. Since M-KS is normalised, itis advantageous over other statistical measures such as: Gstatistic and the Chi square statistic.

3. Segmentation Method

The unsupervised colour texture segmentation methodinvolves three steps: hierarchical splitting, agglomerativemerging and the boundary refinement.

Hierarchical Splitting The hierarchical splitting recur-sively splits the input image into four subblocks. The sixpairwise M-KS values between the LBP/C of the 4 sub-blocks were calculated. The uniformity of the region wastested by a decision factor

R =MKSmax

MKSmin

> X (2)

where X is a threshold value, MKSmax and MKSmin rep-resents the highest and the lowest M-KS values.

Agglomerative Merging An agglomerative merging pro-cedure was applied on the image which has been splitinto blocks of roughly uniform textures. This proceduremerges similar adjacent regions until a stopping rule is sat-isfied. The pair of adjacent segments which has the smallestMerger Importance (MI) value were merged. The MI valuebetween two regions is calculated as followed,

MI = w1 ∗MKS1 + w2 ∗MKS2 (3)

where w1 and w2 represents corresponding weights for theLBP histogram and the colour clustered histogram respec-tively. MKS1 and MKS2 represents the M-KS statisticfor texture and colour histograms respectively. The weightsare automatically detected using an uniformity factor de-fined as the maximum of the ratio between colour clusteredhistogram and number of pixels in the two regions underconsideration, the sample and the model regions.

kj = max

{

CLj [i]

Np

}

(4)

where kj represents the uniformity factor for the sampleand the model regions respectively. If the difference be-tween k1 and k2 is less than 0.1, i.e., both the sampleand the model weights are more or less the same, thenw2 = (k1 + k2)/2 and w1 = 1 − w2. This indicates thatcolour influences more than texture, hence colour statisticis given more importance. On the other hand, if the differ-ence between k’s is high, both the texture and the colourare given equal weights. This method followed a simplestopping rule, MinMI > Y , where MinMI represents theminimum merger importance value. If this is greater than athreshold value then the merging procedure is halted.

(a) (b)

(c) (d)

Figure 1. (a), (c) Segmentation before bound-ary refinement, (b), (d) Segmentation afterboundary refinement

Boundary Refinement The agglomerative merging pro-cedure resulted in blocky segmented image. A new bound-ary refinement algorithm was developed and used for theimprovement at the boundaries between various regions. Apixel is regarded as a boundary point if it is on the boundaryof at least two distinct regions, i.e., its region label is differ-ent from at least one of its four neighbours. For an exam-ined point P, a discrete square with a dimension d aroundthe pixel was placed and the colour histogram for this re-gion was computed. The corresponding colour histogramsfor the different neighbouring points were calculated. Thehomogeneity of the square region and the ith neighbouringregion, i=1,2,...l...n region was computed. The pixel is re-classified if the MI value between adjacent regions and theregion around the pixel under consideration is lower thanthe merge threshold. This procedure is iterative and pro-ceeds until no pixels are relabelled. Reassigning pixels in

this way improves the accuracy of the segmentation pro-cess. Figure 1 shows the segmentation result before andafter boundary refinement.

4. Experimental Results

The experimental evaluation of the proposed method wasbased on 46 images from VisTex [10] image database con-sisting of mosaics and natural images. The mosaics wereconstructed using a random selection of different texturesfrom the VisTex texture image database. In addition, 2 mo-saic images used by Petrou et al. [7] and 2 natural imagesused by Panjwani et al. [6] were also used for the experi-ments.

WeightsColour(w2)

Texture(w1)

e (%)

1.0 0.0 4.840.6 0.4 4.530.2 0.8 11.710.0 1.0 26.71automaticselection

automaticselection

< 1.0

Table 1. Average segmentation error (%)based on the colour and the texture weights

Tests were carried out to demonstrate the importance of thecolour and texture descriptors for colour texture segmen-tation. Table 1 illustrates the segmentation error based oncolour and texture features. The weights in the mergingstage were replaced with different values ranging from 0.0to 1.0. The mean error rate for different colour and textureweights were calculated. The error rates in Table 1 showsthat up to a weight of 0.6 for colour the segmentation erroris minimal beyond which the error increased significantly.The errors at the colour weights 0.2 and 0.0 was found to besubstantial. From Table 1 it is apparent that colour weights0.6 and texture weights 0.4 resulted in a good segmentationwith minimum error. In addition, the automatic selection ofcolour and texture weights provided a good result with min-imum segmentation error. The following conclusions canbe drawn from these experimental results. Texture aloneor colour alone cannot provide a good segmentation. Theproper inclusion of both colour and texture is necessary fora more accurate colour texture segmentation.The results in Figure 2 shows that the error in segmentationbased on colour alone or texture alone is higher compared tothe segmentation based on colour and texture. In addition,the results depicted in Figure 2 illustrate that the incorpora-tion of colour with texture increases the accuracy of colour

texture segmentation.

(a) (b) (c)

(d) (e) (f)

(g) (h) (i)

Figure 2. (a), (d), (g) Segmentation based oncolour, (b), (e), (h) Segmentation based ontexture, (c), (f), (i) Segmentation based oncolour and texture

The boundaries obtained after the merging stage and theboundary refinement stage were presented in Figure 3 andFigure 4. These images shows the detected boundary em-bedded over the original image. Figure 3-M1 illustrates aproper segmentation result. Figure 3-M2 and M3 showssegmentation of small regions due to the difference in thecolour.Figure 4-N1 represents the segmented beach image. Thesegmentation method merged the small features within thebird as a single region. Figure 4-N2 shows the segmentedimage of the rocks and sea. It can be noted that the regionrepresenting the clouds and the sky is considered as a singleregion. In addition, the regions on the rock and sea weremerged together as a single region. It is evident from thesegmented results that it is not possible to differentiate re-gions with similar colour and different textures.

M1(a) M1(b)

M2(a) M2(b)

M3(a) M3(b)

Figure 3. Colour texture segmentation of mo-saic images. (a) represents segmented imageand (b) represents boundary refined image.

N1(a) N1(b)

N2(a) N2(b)

Figure 4. Colour texture segmentation of nat-ural images

5. Conclusions

A new framework was developed for colour texture seg-mentation which integrates the colour and the texture fea-tures. The distribution of colour features and the distri-bution of the texture features were used for colour texturediscrimination. The distribution of the derived features en-compasses both the structural pattern and the colour of theimage. The method was applied to various mosaic and nat-ural images and the importance of colour in colour texturesegmentation was demonstrated.

References

[1] K. M. Chen and S. Y. Chen. Color texture segmentationusing feature distributions. Pattern Recognition Letters,23:755–771, 2002.

[2] A. Drimbarean and P. F. Whelan. Experiments in colour tex-ture analysis. Pattern Recognition Letters, 22:1161–1167,2001.

[3] A. K. Jain and R. C. Dubes. Algorithms for clusteringdata. Prentice Hall, Advanced Reference Series, New Jer-sey, 1988.

[4] A. K. Jain and G. Healey. A multiscale representationincluding opponent color features for texture recognition.IEEE Trans. on Image Processing, 7(1):124–128, 1998.

[5] T. Ojala and M. Pietikainen. Unsupervised texture seg-mentation using feature distributions. Pattern Recognition,32:477–486, 1999.

[6] D. K. Panjwani and G. Healey. Markov random field mod-els for unsupervised segmentation of textured color images.IEEE Trans. on Pattern Analysis and Machine Intelligence,17(10):939–954, 1995.

[7] M. Petrou, M. Mirmehdi, and M. Coors. Perceptual smooth-ing and segmentation of colour textures. Proc. of the 5th Eu-ropean Conference on Computer Vision, I:623–639, 1998.

[8] M. Pietikainen, T. Maenpaa, and J. Viertola. Color textureclassification with color histograms and local binary pattern.In Proc.Texture2002. 2nd International Workshop on TextureAnalysis and Synthesis. In Conjunction with ECCV2002,2002.

[9] J. Puzicha, J. M. Buhmann, Y. Rubner, and C. Tomasi. Em-pirical evaluation of dissimilarity measures for colour andtextrue. In Proc. of IEEE Int. Conf. on Computer Vision,ICCV’99, pages 1156–1173, 1999.

[10] VisTex. Colour texture image database, 2000.