improving sampling-based image matting with cooperative coevolution differential ... · 2018. 5....

14
Soft Comput (2017) 21:4417–4430 DOI 10.1007/s00500-016-2250-7 METHODOLOGIES AND APPLICATION Improving sampling-based image matting with cooperative coevolution differential evolution algorithm Zhao-Quan Cai 1 · Liang Lv 2 · Han Huang 2 · Hui Hu 1 · Yi-Hui Liang 2 Published online: 6 July 2016 © Springer-Verlag Berlin Heidelberg 2016 Abstract Image matting is a fundamental operator in image editing and has significant influence on video production. This paper explores sampling-based image matting technol- ogy, with the aim to improve the accuracy of matting result. The result of sampling-based image matting technology is determined by the selected samples. Every undetermined pixel needs both a foreground and background pixel to esti- mate whether the undetermined one is in the foreground region of the image. These foreground pixels and background pixels are sampled from known regions, which form sample pairs. High-quality sample pairs can improve the accuracy of matting results. Therefore, how to search for the best sample pairs for all undetermined pixels is a key optimization prob- lem of sampling-based image matting technology, termed “sample optimization problem.” In this paper, in order to improve the efficiency of searching for high-quality sam- ple pairs, we propose a cooperative coevolution differential evolution (DE) algorithm in solution to this optimization problem. Strong-correlate pixels are divided into a group to Communicated by V. Loia. B Liang Lv [email protected] Zhao-Quan Cai [email protected] Han Huang [email protected] Hui Hu [email protected] Yi-Hui Liang [email protected] 1 College of Huizhou, Huizhou 516007, China 2 School of Software Engineering, South China University of Technology, Guangzhou 510006, China cooperatively search for the best sample pairs. In order to avoid premature convergence of DE algorithm, a scattered strategy is used to keep the diversity of population. Besides, a simple but effective evaluation function is proposed to distinguish the quality of various candidate solutions. The existing optimization method, original DE algorithm and a popular evolution algorithm are used for comparison. The experimental results demonstrate that the proposed coopera- tive coevolution DE algorithm can search for higher-quality sample pairs and improve the accuracy of sampling-based image matting. Keywords Image matting · Sample selection · Cooperative coevolution · Differential evolution algorithm · Diversity of population 1 Introduction Image and video processing is a popular and useful field of research (Gu et al. 2015; Xia et al. 2016, 2014; Xie and Wang 2014). The major research branches include video coding (Pan et al. 2015), image segmentation (Zheng et al. 2015; Miao et al. 2013), image fusion (Shi et al. 2013; Miao et al. 2011), edge detection (Xu et al. 2012) and image matting. This paper focuses on image matting. Image matting was originally applied in film and video production (Fieldling 1970) and plays a central role in image processing. It refers to the problem of smooth-and-exact extracting the foreground region from an image, which is implemented by determining whether undetermined pixel is in the foreground region of the image. The foreground region is ordered by a user. In general, the information can be obtained by a corresponding trimap image. This trimap image (Wang and Cohen 2005; Lee and Wu 2011) divides the tackled image into three regions: 123

Upload: others

Post on 21-Feb-2021

3 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Improving sampling-based image matting with cooperative coevolution differential ... · 2018. 5. 23. · Soft Comput (2017) 21:4417–4430 DOI 10.1007/s00500-016-2250-7 METHODOLOGIES

Soft Comput (2017) 21:4417–4430DOI 10.1007/s00500-016-2250-7

METHODOLOGIES AND APPLICATION

Improving sampling-based image matting with cooperativecoevolution differential evolution algorithm

Zhao-Quan Cai1 · Liang Lv2 · Han Huang2 · Hui Hu1 · Yi-Hui Liang2

Published online: 6 July 2016© Springer-Verlag Berlin Heidelberg 2016

Abstract Imagematting is a fundamental operator in imageediting and has significant influence on video production.This paper explores sampling-based image matting technol-ogy, with the aim to improve the accuracy of matting result.The result of sampling-based image matting technology isdetermined by the selected samples. Every undeterminedpixel needs both a foreground and background pixel to esti-mate whether the undetermined one is in the foregroundregion of the image. These foreground pixels and backgroundpixels are sampled from known regions, which form samplepairs. High-quality sample pairs can improve the accuracy ofmatting results. Therefore, how to search for the best samplepairs for all undetermined pixels is a key optimization prob-lem of sampling-based image matting technology, termed“sample optimization problem.” In this paper, in order toimprove the efficiency of searching for high-quality sam-ple pairs, we propose a cooperative coevolution differentialevolution (DE) algorithm in solution to this optimizationproblem. Strong-correlate pixels are divided into a group to

Communicated by V. Loia.

B Liang [email protected]

Zhao-Quan [email protected]

Han [email protected]

Hui [email protected]

Yi-Hui [email protected]

1 College of Huizhou, Huizhou 516007, China

2 School of Software Engineering, South China Universityof Technology, Guangzhou 510006, China

cooperatively search for the best sample pairs. In order toavoid premature convergence of DE algorithm, a scatteredstrategy is used to keep the diversity of population. Besides,a simple but effective evaluation function is proposed todistinguish the quality of various candidate solutions. Theexisting optimization method, original DE algorithm and apopular evolution algorithm are used for comparison. Theexperimental results demonstrate that the proposed coopera-tive coevolution DE algorithm can search for higher-qualitysample pairs and improve the accuracy of sampling-basedimage matting.

Keywords Image matting · Sample selection · Cooperativecoevolution · Differential evolution algorithm · Diversity ofpopulation

1 Introduction

Image and video processing is a popular and useful field ofresearch (Gu et al. 2015; Xia et al. 2016, 2014; Xie andWang2014). The major research branches include video coding(Pan et al. 2015), image segmentation (Zheng et al. 2015;Miao et al. 2013), image fusion (Shi et al. 2013; Miao et al.2011), edge detection (Xu et al. 2012) and image matting.This paper focuses on image matting. Image matting wasoriginally applied in film and video production (Fieldling1970) and plays a central role in image processing. It refers tothe problem of smooth-and-exact extracting the foregroundregion from an image, which is implemented by determiningwhether undetermined pixel is in the foreground region ofthe image. The foreground region is ordered by a user. Ingeneral, the information can be obtained by a correspondingtrimap image.This trimap image (WangandCohen2005;Leeand Wu 2011) divides the tackled image into three regions:

123

Page 2: Improving sampling-based image matting with cooperative coevolution differential ... · 2018. 5. 23. · Soft Comput (2017) 21:4417–4430 DOI 10.1007/s00500-016-2250-7 METHODOLOGIES

4418 Z.-Q. Cai et al.

respectively, known foreground region, known backgroundregion and undetermined region. The goal of image mattingis to determine which pixels of undetermined region belongto foreground region. In the literature, various methods havebeen tested to implement matting in order that the extractiveforeground region can be satisfied. These methods can becategorized into sampling-based image matting technologyand propagation-based image matting technology.

This present study focuses on sampling-based imagematting technology. It collects a set of foreground and back-ground pixels from known regions to build sample sets.Then it uses these samples to estimate undetermined pix-els’ regions. In sampling-based image matting technology,the basic assumption is that for every undetermined pixel, itsregion can be accurately estimated by a pair of foreground–background pixels, which are sampled from known regions.Therefore, in order to obtain more accurate matting results,various sampling-basedmattingmethods havebeenproposedto collect high-quality samples. Earlier methods are localsampling-based methods (Wang and Cohen 2007; Rhemannet al. 2008; Gastal and Oliveira 2010). They collect fore-ground and background pixels from the nearby known regionof every undetermined pixel, according to spatial or colorcharacteristics of undetermined pixel. Usually, the size ofsample set is not big. Therefore, they can search for thebest sample pair for every undetermined pixel by brute-forcemethod. Different evaluation criteria consisting of spatial,photometric or probabilistic characteristics of the image areused to evaluate the quality of sample pairs (Wang andCohen2007; Gastal and Oliveira 2010). However, the best samplepair of every undetermined pixel is not always in samplesets because the number of samples is rather limited. Theaccuracy of matting result will degrade when the best sam-ple pair is not collected in the sample set. We call this themissing problem. In order to avoid this problem, a larger-scale sampling method is needed for covering more pixels ofknown region. The global sampling-based method (He et al.2011) collects all known pixels on the boundary of unde-termined region to build the sample set. These samples areshared among undetermined pixels. This sample set is hugeenough so that the best sample pair of every undeterminedpixel can be collected from it. However, the time complexityof searching for the best samples is growing rapidly at thesame time. Brute-force method is not an acceptable methodfor searching for the best sample pairs. Therefore, how toreduce the time complexity and find the best sample pairs forall undetermined pixels is a key optimization problem. Theproblem is called “sample optimization problem.” Existingoptimization method is a random search algorithm (Barneset al. 2009), which has been used in He et al. (2011). It treatsthe sample optimization problem as a correspondence searchprocess among undetermined pixels. The matting result (Heet al. 2011)was proved to be better than local sampling-based

methods. But the exploration and the robustness of this ran-dom search algorithm are not strong because of the propertyof random searching. Therefore, the best sample pairs cannotalways be found in search process.

In this paper, in order to search for best sample pairs for allundetermined pixels, we formulate the sample optimizationproblem with a large-scale numerical optimization problemand propose a new DE algorithm to solve it. Historically,various algorithms and frameworks have been proposed tosolve the large-scale numerical optimization problem withthe aim to improve the efficiency of searching (Mhlenbein1989; Liu et al. 2001; Cheng and Jin 2014). Consideringthe property of sample optimization problem, we use thecooperative coevolution framework to enhance the perfor-mance of originalDEalgorithm.The cooperative coevolution(CC) is a promising framework and based on the divide-and-conquer strategy,whichwas first proposed byPotter and Jong(1994). It tackles a high-dimensional optimization problemby decomposing this problem into several low-dimensionalsubproblems. Then it obtains an overall solution from thecombinations of sub-solutions, which are obtained by solv-ing each subproblems. In this original CC framework study(Potter and Jong 1994), the maximum dimension of opti-mization problems is only thirty. The first attempt to useCC framework to solve large-scale optimization problemsis made in Liu et al. (2001). Later, in order to improve theperformance of CC framework, lots of works have devotedto study decomposition strategy (Yang et al. 2008; Omid-var et al. 2014) and contributions of subproblems (Omidvaret al. 2011). Clearly, the cooperative coevolution frameworkperforms better on separable problems. Usually, pixels of animage, whose spatial distance is close to each other, havethe same best foreground–background sample pair. There-fore, the sample optimization problem is actually a separableproblem and adjoining pixels can be divided into a separategroup to cooperatively search for the best sample pair. Thecooperative coevolution mechanism can enhance the perfor-mance of DE algorithm on sample optimization problem.In order to address the drawback of existing random searchalgorithm (Barnes et al. 2009), we choose the DE/rand/1/bin(Storn and Price 1997) as the mutation strategy of our DEalgorithm. This mutation strategy has a strong explorationcapacity and is suitable for global search (Qin et al. 2009;Price et al. 2005). In order to prevent the DE algorithm frompremature convergence and find high-quality sample pairs,we add an additional scattered strategy to our DE algorithm,which can keep the diversity of population. Besides, a simplebut effective evaluation function is proposed to distinguishthe quality of various candidate solutions.

The remainder of this paper is organized into four sections.Sect. 2 defines the sample optimization problem. Section 3describes in detail the cooperative coevolution DE algorithmwith a scattered strategy and the evaluation function. Our

123

Page 3: Improving sampling-based image matting with cooperative coevolution differential ... · 2018. 5. 23. · Soft Comput (2017) 21:4417–4430 DOI 10.1007/s00500-016-2250-7 METHODOLOGIES

Improving sampling-based image matting with cooperative coevolution differential… 4419

experiment, the used image data set, the compared algo-rithms and the results of comparison, are given in Sect. 4.We conclude this paper in Sect. 5.

2 Sample optimization problem

2.1 Problem definition

This section will describe the sample optimization problemof sampling-based image matting technology, which wasfirst introduced by Porter and Duff (1984). This optimiza-tion problem can be mathematically modeled to be a linearcombination equation. Themost common combination equa-tion is the over equation, where the color of a undeterminedpixel is regarded as the combination of a foreground colorand a background color.We consider an imagewith Nu unde-termined pixels. Specifically, for the kth undetermined pixel,k = 1, 2, . . . , Nu, the over equation is shown by Expression(1).

Ik = αk Fk + (1 − αk)Bk (1)

where Ik is the color of the kth undetermined pixel, which canbe obtained directly from the given image. Fk is the color offoreground pixel, while Bk is the color of background pixel.They are chosen for kth undetermined pixel to calculate itsalpha value αk by expression (1). The alpha value for everypixel is typically called its opacity for foreground. Its valueis in the range of [0, 1]. The kth undetermined pixel is aforeground pixel of the given image if its αk = 1, while itis a background pixel if its αk = 0. Therefore, Fk and Bk

are the optimized variables for the kth undetermined pixel.The goal of optimization is to search for the best foreground–background colors for every undetermined pixel. Each pair ofbest foreground–backgroundcolors can accurately determinewhether corresponding undetermined pixel is a foregroundpixel. Therefore, in order to obtain accurate matting result,we must find the best foreground–background colors (F, B)

for every undetermined pixel. As mentioned above, the basicassumption of sampling-based image matting technology isthat the best foreground–background colors of all undeter-mined pixels can be sampled from the unknown region of agiven image. Therefore, once the sample sets are built, thegoal of sample optimization problem is to search for the bestforeground–background sample pair for every undeterminedpixel in built sample sets.

Historically, in order to definite the best foreground–background sample pair for every undetermined pixel, vari-ous evaluation criteria have been proposed (Wang and Cohen2007; Gastal and Oliveira 2010). In this study, we use thisevaluation model (He et al. 2011) to evaluate the quality ofdifferent foreground–background sample pairs for one unde-

termined pixel. This evaluation model is based on spatialand color fitness of selected sample pair. In particular, for kthundetermined pixel, we assume that the selected foreground–background sample pair is (Fi , B j ) and the evaluation valueof this sample pair is calculated by:

f(Fi , B j

)= fc

(Fik , B

jk

)+ fs(F

i ) + fs(Bj ), (2)

where Fik is the color of i th foreground sample in built fore-

ground sample set and B jk is the color of j th background

sample in built background sample set. fc is the color fitnessfunction of sample pair (Fi , B j ), which is used to describehow selected sample pair fits (1):

fc(Fik , B

jk ) =

∥∥∥Ik −(α̂Fi

k + (1 − α̂B jk )

)∥∥∥ , (3)

α̂ = (Ik − B jk )(Fi

k − B jk )∥∥∥Fi

k − B jk

∥∥∥2 , (4)

A smaller fc() indicates that the selected sample pair canbetter explain the corresponding undetermined pixel’s color.A similar evaluation function was used in Wang and Cohen(2007) andGastal andOliveira (2010).However, it is possiblethat a bad sample pair occasionally well explains the corre-sponding undetermined pixel’s color. Therefore, the spatialinformation is also considered in f (). fs(Fi ) is used to cal-culate the spatial distance between the i th foreground sampleand the corresponding undetermined pixel:

fs(Fi ) =

∥∥XFi − Xk∥∥

DF(5)

where XFi and Xk are the spatial coordinates of the i th fore-ground sample and the corresponding undetermined pixel.The factor DF = mini

∥∥XFi − Xk∥∥ is the nearest distance of

the corresponding undetermined pixel to foreground bound-ary, which is used to ensure the fc is independent fromthe absolute distance. A smaller fc(Fi ) indicates that theselected foreground sample is closer to the correspondingundetermined pixel. The fs(B j ) of the j th background sam-ple is defined similarly. Therefore, a smaller f () indicatesthat the selected sample pair (Fi , B j ) can better explain thecorresponding undetermined pixel’s color and is spatially asclose as possible. Therefore, a high-quality sample pair forone undetermined pixel means that its f () value is small. Thebest sample pair for the corresponding undetermined pixelhas smallest f () value. For a certain undetermined pixel, thesample optimization problem can be regarded as a searchproblem in two-dimensional foreground–background searchspace, as shown in Fig. 1. The left part of Fig. 1 is the Trimap,where the white region is known foreground region, the grayregion is undetermined region, and the black region is known

123

Page 4: Improving sampling-based image matting with cooperative coevolution differential ... · 2018. 5. 23. · Soft Comput (2017) 21:4417–4430 DOI 10.1007/s00500-016-2250-7 METHODOLOGIES

4420 Z.-Q. Cai et al.

Fig. 1 Sample optimization problem. Left The Trimap, where Ik iscolor of the kth undetermined pixel. Right Foreground–backgroundsample pair search space, where Fi is the i th foreground sample inforeground sample set and B j is the j th background sample in back-ground sample set

background region. The right part of Fig. 1 is the foreground–background sample pair search space.

2.2 Large-scale optimization model

Obviously, the sample optimization problem is a correspon-dence search problem among undetermined pixels. There-fore, in order to improve the search efficiency, we modelthe sample optimization problem in two-dimensional searchspace into a large-scale optimization problem in multidi-mensional search space. The goal of this new optimizationproblem is to search for best foreground–background samplepairs for all undetermined pixels at once. In this multidi-mensional search space, every position represents a group ofcandidate foreground–background sample pairs for all unde-termined pixels and every dimensional value represents aforeground sample or a background sample. The best positionobtains the best foreground–background sample pairs for allundetermined pixels.We assume that the given image has Nu

undetermined pixels, the sizes of built foreground sample setand built background sample set are NF and NB , respectively.Thus, there exist NFNB

Nu candidate solutions for all undeter-minedpixels.Obviously, the timecomplexity of searching forthe best sample pairs shows rapid growth with the increase ofthe number of the undetermined pixels and samples. Usually,the number of undetermined pixels is huge and uncertain.Besides, as mentioned above, in order to avoid the best sam-ple pair missing problem, the size of sample set is huge sothat the best sample pair of every undetermined pixel canbe collected in the built sample set. Therefore, the large-scale sample optimization problem is a complex one. Anefficient search algorithm is needed to solve this optimizationproblem. Existent random search algorithm is insufficient inexploration and robustness. In order to improve the efficiencyof searching for high-quality sample pairs, we propose inthis paper a cooperative coevolution differential evolution

algorithm with a scattered strategy (CC-DE-S) to solve thisproblem. The implementation of DE algorithm is describedin next section.

3 The cooperative coevolution differentialevolution algorithm with a scattered strategy

Differential evolution (DE) is a population-based paral-lel evolution algorithm, which was proposed by Storn andPrice (1997). DE has been successfully applied in manyimage processing areas such as Sarangi et al. (2014), Yuet al. (2015) and Guo and Li (2015). It is initialized witha population. Every individual of population represents acandidate solution of the problem. DE updates the posi-tion of every individual by using the differences amongthe individuals, which makes DE exploratory. The proposedcooperative coevolution differential evolution algorithmwitha scattered strategy (CC-DE-S) is based on the original DE(Storn and Price 1997). It consists of four basic operators:the initialization, the mutation, the crossover, and the selec-tion, respectively. The detailed formulation of CC-DE-S isdescribed as follows.

3.1 The formulation of proposed CC-DE-S

In our CC-DE-S algorithm, adjoining undetermined pixelswere divided into a group to cooperatively search for thebest sample pairs. In order to achieve this goal, we used acommon image segmentation tool, which is calledmean shiftimage segmentation (Comaniciu and Meer 1997, 2002), forgrouping. This image segmentation tool can divide pixelsinto different groups by considering their color and spatialcharacteristics. User needs to set spatial parameter and colorparameter for controlling the bandwidth (Comaniciu andMeer 2002).Asmentioned above, these undetermined pixels,whose spatial distances are close to each other, are often-times believed to have the same best foreground–backgroundsample pair. Therefore, in this study, the spatial parameterwas adjusted to obtain a better result and the color para-meter was set to seven, according to Comaniciu and Meer(2002). Therefore, the sizes of different groups consisted ofadjoining pixels were determined by the spatial parameter.The critical step in CC framework is decomposition strategy.Therefore, the efficiency of CC-DE-S algorithm is sensi-tive to the sizes of groups. Smaller size and larger size willdegrade performance of our algorithm. Finally, the spatialparameter was set to eight, which we found to be sufficient inexperiments.We assume that an given image has Nu undeter-mined pixels and these undetermined pixels are divided intoM groups by setting the spatial parameter to eight and colorparameter to seven. In the initialization, the CC-DE-S algo-rithm will be initialized with M subpopulations randomly

123

Page 5: Improving sampling-based image matting with cooperative coevolution differential ... · 2018. 5. 23. · Soft Comput (2017) 21:4417–4430 DOI 10.1007/s00500-016-2250-7 METHODOLOGIES

Improving sampling-based image matting with cooperative coevolution differential… 4421

positioned in problem space. Every individual of M subpop-ulations is sub-solution, which moves in this problem spacefor looking for the optimal solution of corresponding groupconsisted of adjoining undetermined pixels. As mentionedabove, we modeled the sample optimization problem into alarge-scale numerical optimization problem. Therefore, theproblem space is a multidimensional search space. Actually,this is a 2Nu dimensional search space, according to ourprevious assumption. Every position of this search space isan overall candidate solution for Nu undetermined pixels,which is a 2Nu dimensional vector and obtains Nu candidateforeground–background sample pairs. The initialized resultof CC-DE-S algorithm is shown as follows,

X =

⎧⎪⎪⎪⎪⎪⎨⎪⎪⎪⎪⎪⎩

x11,g x21,g ...

... ... ...

x1i,g x2i,g ...

... ... ...

x1Np,gx2Np,g

...︸ ︷︷ ︸

1

x j1,g...

x ji,g...

x jNp,g︸ ︷︷ ︸...

... x2Nu−11,g x2Nu

1,g... ... ...

... x2Nu−1i,g x2Nu

i,g... ... ...

... x2Nu−1Np,g

x2NuNp,g

⎫⎪⎪⎪⎪⎪⎬⎪⎪⎪⎪⎪⎭

︸ ︷︷ ︸M

(6)

where X is the overall population, which consists of M sub-populations. g is the number of generation in the evolutionaryprocess. i is the index of the individual, i = 1, 2, 3, . . . , Np.Np is the size of every subpopulation. j is the j th dimensionof Xi,g. The overall population was divided into M subpop-ulations, and every individual of subpopulations representsa candidate solution for corresponding group consisted ofadjoining undetermined pixels. In our formulation, everydimensional value of Xi,g is integer. The reason for usingthe integer in individual rather than floating-point numbersis that every dimensional value of Xi,g represents the indexof a foreground sample or a background sample in sampleset. Specifically, if the size of foreground sample set is NF

and the size of background sample set is NB, every odd-dimensional value of Xi,g represents an index of foregroundsample and randomly initialized in {1, 2, . . . , NF} and everyeven-dimensional value of Xi,g represents an index of back-ground sample and randomly initialized in {1, 2, . . . , NB},as follows:

⎧⎪⎨⎪⎩

x1i,g, x3i,g, . . . , x

2Nu−1i,g = rand(1, NF)

x2i,g, x4i,g, . . . , x

2Nui,g = rand(1, NB)

(7)

In every individual, an adjacent pair of odd-dimensionalvalue and even-dimensional value represents a candidatesample pair for an undetermined pixel.

As a simple example, Fig. 2 shows an image with fiveundetermined pixels. The size of foreground sample set isthree, and the size of background sample set is five. Thesample sets are sorted by their color, which is grayscale.

Fig. 2 Example of the proposedDE algorithm. Fi is the i th foregroundsample. B j is the j th background sample

It has been proved that there is no difference betweenthe color and other sorting criteria (He et al. 2011). Forthese five undetermined pixels, we assume that they aredivided into two groups by considering their spatial char-acteristics. The first group consists of three undeterminedpixels, the second group consists of two undetermined pix-els, and the best foreground–background sample pairs foreach group is: [2, 4, 1, 3, 3, 2] and [1, 4, 2, 1], respectively.We randomly initialize two subpopulations, each with Np

individuals, in ten dimensional search space. Then, the CC-DE-S algorithm is used to search for the best solution:[2, 4, 1, 3, 3, 2, 1, 4, 2, 1].

After the initialization, the next operator is mutation.Mutation generates a new subpopulation for every subpopu-lation by using the weighted difference between individualsin current subpopulation. Different mutation strategies havea significant influence on the behavior of the algorithm. Inthe literature, several studies have shown that some mutationstrategies are good for global search (Qin et al. 2009; Priceet al. 2005; Gong et al. 2011), while some others are goodfor local fine tuning (Zhang and Sanderson 2009; Mezura-Montes et al. 2006). In order to improve the exploration androbustness of CC-DE-S algorithm in solving this large-scaleoptimization problem, we choose the DE/rand/1/bin (Stornand Price 1997) as the mutation strategy. It is good for globalsearch (Qin et al. 2009; Price et al. 2005), which is shown asfollows.

Vi,g = Xr1,g + F(Xr2,g − Xr3,g

)(8)

where Vi,g is the i th individual of the new subpopulation atthe g generation. The indexes r1, r2, r3 are randomly cho-sen from the set {1, 2, . . . , Np}. They are different from theindex i . F is a weighted parameter. To make sure that everydimensional value of Vi,g is integer, we used the integerpart of every dimensional value of Vi,g as the final value

(�v1i,g� �v2i,g� . . . , �vNui,g �).

In the crossover operator, an effective mechanism is usedto prevent individuals of every trial subpopulation over the

123

Page 6: Improving sampling-based image matting with cooperative coevolution differential ... · 2018. 5. 23. · Soft Comput (2017) 21:4417–4430 DOI 10.1007/s00500-016-2250-7 METHODOLOGIES

4422 Z.-Q. Cai et al.

boundary of search space. Trial subpopulation is generatedby mixing the new subpopulation in mutation and currentcorresponding subpopulation. It is used to generate betterindividuals in comparison with current corresponding sub-population. The individual of trial subpopulation is termedas Ui,g, i = 1, 2, . . . , Np. If a dimensional value of Ui,g

is over boundary, it will be modified into the correspondingdimensional value of Xi,g. This operator is conducive to holdthe good sample.

3.2 The scattered strategy

In the selection step, in order to prevent the population frompremature convergence, we designed a scattered strategy tokeep the diversity of population. For every subpopulation,if the best individual did not have an update after the selec-tion, the scattered strategy will randomly redistribute thissubpopulation in search space. In order to decrease the timecomplexity of CC-DE-S algorithm and make good use ofthe convergence ability of DE algorithm, a threshold n wasdesigned for executing this scattered strategy. Only if after nselections the best individual still did not have any update,the scattered strategy would be executed for recovering thediversity of this subpopulation. So, the scattered strategy wasexecuted conditionally. We found that setting n to five iseffective to produce good results. The pseudo-code of theCC-DE-S algorithm is shown in Algorithm 1.

3.3 The evaluation function

Aswe have discussed, in our large-scale numerical optimiza-tion problem, every position of problem space represents agroup of candidate foreground–background sample pairs forall undetermined pixels. In order to distinguish the qualityof various individuals in our CC-DE-S algorithm and definethe best position, we proposed a simple but effective eval-uation function. This evaluation function was on the basisof the evaluation model (2). As shown in (6), we assumethat Nu undetermined pixels are divided into M groups bymean shift image segmentation. The size of every group isdenoted as Nm,m = 1, 2, 3, . . . , M. In particular, for the i thoverall individual at g generation, i = 1, 2, 3, . . . , Np, wecalculated its overall evaluation value by:

ϕ(Xi,g) =Nu∑k=1

f(x2k−1i,g , x2ki,g

), i = 1, 2, . . . , Np. (9)

where f () is the evaluation model (2). x2k−1i,g represents a

selected foreground sample and x2ki,g represents a selectedbackground sample for a undetermined pixel in one group.According to the strategy of cooperative coevolution, whenmth subpopulation is searching for best foreground–backg-

Algorithm 1 The CC-DE-S algorithmInput: The number of the undetermined pixels: Nu . The size of the

ordered foreground sample set and the ordered background sampleset: NF and NB , respectively;

1: Use the mean shift image segmentation to divide Nu undeterminedpixels into M groups. The size of every group is denoted asNm ,m = 1, 2, 3, ..., M ; randomly initialize Np individuals,each with 2Nu dimensions in 2Nu search space; set the numberof generation g to 0 and the thresholds of the scattered strategythrm to 0,m = 1, 2, ..., M ; use an integer count to record whichgroup is cooperative coevolution and initialize it with 0. Integerk is a parameter in crossover step, which is used to mark whichdimension must be replaced.

2: while termination criterion is not met do3: for m = 1 to M do4: for i = 1 to Np do5: Randomly choose three different indexes6: r1, r2, r3.7: from the set

{1, 2, ..., Np

}.

8: Set the k = �count + 2Nm ∗ rand(0, 1)�.9: Set ui,g = xi,g .10: for j = count + 1 to count + 2Nm do

11: Set v = x jr1,g + F

(x jr2,g − x j

r3,g

).

12: Set v = �v�.13: if rand(0, 1) ≤ CR or j = k then14: u j

i,g = v.15: else16: u j

i,g = x ji,g .

17: end if18: if u j

i,g > NF or NB then

19: u ji,g = x j

i,g .

20: else if u ji,g < 1 then

21: u ji,g = x j

i,g .22: end if23: end for24: if Ui,g is better that Xi,g then25: Xi,g = Ui,g .26: end if27: end for28: if The best individual does not update then29: thrm = thrm + 1.30: end if31: if thrm > n then32: redistribute current subpopulation33: in search space; thrm = 0.34: end if35: count = count + 2Nm ;36: end for37: g = g + 1.38: end whileOutput: The optimal overall individual;

round sample pairs at g generation, only the evaluationvalues of individuals in mth subpopulation will be changedand the evaluation values of individuals in other subpop-ulations remain unchanged. Corresponding to (6), in thesearch step of mth subpopulation, the values of 1th to(2N1 + · · · + 2Nm−1)th dimension in overall individualsremain unchanged. Besides, the values of (2N1+· · ·+2Nm+1)th to (2Nu)th dimension in overall individuals remain

123

Page 7: Improving sampling-based image matting with cooperative coevolution differential ... · 2018. 5. 23. · Soft Comput (2017) 21:4417–4430 DOI 10.1007/s00500-016-2250-7 METHODOLOGIES

Improving sampling-based image matting with cooperative coevolution differential… 4423

unchanged. Only the values of (2N1+· · ·+2Nm−1+1)th to(2N1+· · ·+2Nm)th dimension in overall individuals will bechanged. The evaluation values of all the dimensions in oneoverall individual contribute to the overall evaluation valuesof this individual.

As shown in (9), we built our evaluation function ϕ() bylinearly adding the f () of every candidate sample pair inone overall individual. As we discussed in Sect. 2, the eval-uation model f () indicates the quality of selected samplepair for one undetermined pixel and a smaller f () meansthat the selected sample pair has higher quality. There-fore, the ϕ() indicates the overall quality of all candidateforeground–background sample pairs in an overall individ-ual. The CC-DE-S algorithm searches for the best solutionfor every subpopulation by using this evaluation function.The best solution obtains all the best sample pairs for cor-responding subpopulations, which consisted of adjoiningundetermined pixels.

4 Experiments and results

The experiments and results are presented in this sec-tion. The goal of our experiments was to prove thatthe proposed CC-DE-S algorithm can find higher-qualityforeground–background sample pairs in sample optimizationprocess, thereby obtaining more accurate matting results forsampling-based image matting. The existing random searchalgorithm (Barnes et al. 2009) was used to compare in exper-iments. To further prove the advantage of our algorithm, wealso compared with original DE algorithm and another pop-ular population-based evolution algorithm: particle swarmoptimization (PSO). The PSOwas proposed by Eberhart andKennedy (1995) and Kennedy and Eberhart (1995) and hasbeen proved to be a fast convergence algorithm. In this exper-iment, we used these four search algorithms to solve thelarge-scale sample optimization problem of the sampling-based image matting technology and compared the searchperformance of four algorithms by contrasting the accuracyofmatting results. In order to avoidmissing best sample pairs,we used the global sampling method (He et al. 2011) to buildsample sets.

For a fair comparison, only the sample optimization stepwas replaced with our CC-DE-S algorithm, original DEalgorithm, PSO algorithm and random search algorithm andkept other steps unchanged. Therefore, the different mattingresults were only caused by different search algorithms usedin sample optimization process. An open image date set wasused to be experimental images (Rhemann et al. 2009, whichcontained twenty-seven suit images. These images coverobjects photographed in front of a natural 3-dimensionalscene and a monitor showing natural images. This openimage data set was also used in Wang and Cohen (2005),

Wang and Cohen (2007), Rhemann et al. (2008), Gastal andOliveira (2010) and He et al. (2011) to test the performanceof imagematting. To evaluate the accuracy of variousmattingresults, this experiment shows visual matting results and thecorresponding mean squared error (MSE) metric. The MSEis a quantitative evaluation, which measures the differencebetween the experimental matting result and the accuratematting result. If the experimental matting result is closer tothe accurate matting result, its MSE value is smaller.

In addition, the compared PSO algorithm followed amod-ified particle swarm optimizer (Shi and Eberhart 1998),where a new parameter, called inertia weight, was added.The random search algorithm was implemented (He et al.2011) by ourselves. The CC-DE-S algorithm increases sometime cost in cooperative coevolution process, compared tothe algorithm without cooperative coevolution mechanism.Therefore, in order to balance the searching performance andtime cost, we set the maximum iteration number of the CC-DE-S algorithm, original DE algorithm and PSO is 3× 105;the maximum iteration number of random search algorithmis ©(10Nulog(NFNB)) He et al. (2011). For the experimenttools, the methods were implemented inMATLAB program-ming language on windows.

The visual matting results of the twenty-seven suit images(Rhemann et al. 2009) are shown in Figs. 3, 4. Intuitively,by comparing the matting results among the CC-DE-Salgorithm, the random search algorithm, the original DEalgorithm and the PSO algorithm, the discovery is shownas follows. The matting quality of the CC-DE-S algorithmand original DE algorithm is almost higher than that of ran-dom search algorithm on all images. Although the overallmatting quality of PSO is also higher than that of randomsearch algorithm. However, PSO obviously performs worseon the 2nd image. By further observing the matting resultsbetween the CC-DE-S algorithm and original DE algorithm,we discovered that the overall matting quality of the CC-DE-S algorithm is almost equal to original DE algorithm.However, the CC-DE-S algorithm obviously performs bet-ter than original DE algorithm on some images, such as 1st,5th, 13th and 16th images. These findings are understandablethat the CC-DE-S algorithm is a more effective and suitablealgorithm for the sampling-based image matting technology.Some detailed differences of different matting results areshown in Figs. 5, 6.

To further confirm our conclusion, we also conductedquantitative comparisons of the matting results by calculat-ing their MSE values. For every experimental image, eachalgorithm was repeated 15 times. For each algorithm, after15 runs, the smallest MSE value and average MSE valuewith standard deviation were recorded in Tables 1 and 2.By comparing the smallest MSE values obtained by CC-DE-S algorithm, the random search algorithm, the originalDE algorithm and the PSO algorithm, the discovery is as

123

Page 8: Improving sampling-based image matting with cooperative coevolution differential ... · 2018. 5. 23. · Soft Comput (2017) 21:4417–4430 DOI 10.1007/s00500-016-2250-7 METHODOLOGIES

4424 Z.-Q. Cai et al.

Fig. 3 Visual comparisons of matting results. 1st column Input image.2nd column Input trimap. 3rd column The matting results with randomsearch algorithm. 4th column The matting results with PSO. 5th col-

umn The matting results with the original DE algorithm. 6th columnThe matting results with the CC-DE-S algorithm

123

Page 9: Improving sampling-based image matting with cooperative coevolution differential ... · 2018. 5. 23. · Soft Comput (2017) 21:4417–4430 DOI 10.1007/s00500-016-2250-7 METHODOLOGIES

Improving sampling-based image matting with cooperative coevolution differential… 4425

Fig. 4 Visual comparisons of matting results. 1st column Input image.2nd column: Input trimap. 3rd column The matting results with randomsearch algorithm. 4th column The matting results with PSO. 5th col-

umn The matting results with the original DE algorithm. 6th columnThe matting results with the CC-DE-S algorithm

123

Page 10: Improving sampling-based image matting with cooperative coevolution differential ... · 2018. 5. 23. · Soft Comput (2017) 21:4417–4430 DOI 10.1007/s00500-016-2250-7 METHODOLOGIES

4426 Z.-Q. Cai et al.

Fig. 5 Detailed comparisons of the 1st matting results. 1st columnStandard result. 2nd column The matting result with random searchalgorithm. 3rd column The matting result with PSO. 4th column Thematting result with the original DE algorithm. 5th column The mattingresult with the CC-DE-S algorithm

Fig. 6 Detailed comparisons of the 13th matting results. 1st columnStandard result. 2nd column The matting result with random searchalgorithm. 3rd column The matting result with PSO. 4th column Thematting result with the original DE algorithm. 5th column The mattingresult with the CC-DE-S algorithm

follows. The smallest MSE values of the proposed DE algo-rithm and original DE algorithm are less than the values ofthe random search algorithm on all images. However, thesmallest MSE values of PSO are less than the values of therandom search algorithm on twenty-six suit images, exceptfor the 2nd image. A smaller value of MSE indicates thatthe experimental matting result is closer to the accurate mat-ting result. Therefore, the smallest MSE results of differentalgorithms support our above first discovery: It is the perfor-mance of the CC-DE-S algorithm, original DE algorithm andPSO algorithm is higher than existing random search algo-rithm. This is because that these three algorithms used theheuristic information between the individuals to search forthe better individual instead of randomly searching. The cur-rent position information of population can be used to searchfor the better solution. Therefore, the exploration of thesethree algorithms is better than the existing random searchalgorithm. By further comparing the smallest MSE valuesamong the CC-DE-S algorithm, original DE algorithm andPSO algorithm, we discovered that the smallest MSE valueof original DE algorithm is only less than the smallest MSEvalue of PSO algorithm on ten images, which are 2nd, 4th,6th, 7th, 12th, 17th, 21th, 22th, 23th and 27th images. If weadd the proposed cooperative coevolutionmechanismand theconditional scattered strategy into original DE algorithm, thesmallest MSE value of CC-DE-S algorithm is less than thesmallest MSE value of PSO algorithm on nineteen images.Besides, the smallest MSE value of CC-DE-S algorithm isless than the value of original DE algorithm on all images.Therefore, the smallest MSE results also support our abovesecond discovery: The overall performance of the CC-DE-Salgorithm is higher than originalDE algorithmandPSOalgo-rithm. Despite the advantage of PSO as rapid convergence,which enabled PSO rapidly find a good solution, PSO is usu-

ally still fallen into premature convergence. The original DEalgorithm can improve the exploration of population and pre-vent population from premature convergence by using themutation strategy. However, its search efficiency is less thanPSO algorithm. In order to improve the search efficiency ofalgorithm, we added the cooperative coevolution mechanisminto original DE algorithm by utilizing the characteristic ofsample optimization problem. In order to prevent proposedalgorithm from premature convergence, we designed a con-ditional scattered strategy to keep the diversity of population.Therefore, the CC-DE-S algorithm is a more effective searchalgorithm for sampling-based image matting technology.

In order to further confirm the robustness of the CC-DE-Salgorithm, we also recorded the average MSE values withstandard deviations, which were obtained by different algo-rithms after 15 runs. We discovered that the average MSEvalues of the CC-DE-S algorithm are less than the valuesof random search algorithm and original DE algorithm onall images. Besides, they are less than the values of PSOalgorithm on twenty images. Therefore, we can get a prelim-inary conclusion: The CC-DE-S algorithm has robustness.In order to further prove our conclusion, the Wilcoxon rank-sum test was conducted between the CC-DE-S algorithm andother three algorithms. The results are also shown in Tables 1and 2. If theMSE values obtained by CC-DE-S algorithm arestatistically different, compared with other three algorithms,the average MSE value with standard deviation of CC-DE-S algorithm is highlighted in bold. We discovered that theMSE values obtained by CC-DE-S algorithm are statisti-cally different on eighteen images, where the MSE valuesof CC-DE-S algorithm are significant less than the values ofother three algorithms on fourteen images. Therefore, thesediscoveries can prove that the proposed CC-DE-S algorithmis a more effective and robustness algorithm for sampling-based image matting technology.

5 Conclusion

The sample optimization problem is key optimization ofsampling-based image matting technology. The time com-plexity of searching for the best foreground–backgroundsample pairs shows rapid growth with the increase in thenumber of the undeterminedpixels and samples. In this paper,in order to find the best sample pairs for all undeterminedpixels in sample sets, we modeled this sample optimiza-tion problem into a large-scale optimization problem andproposed a cooperative coevolution DE algorithm with ascattered strategy (CC-DE-S) to solve this problem. In orderto improve the search efficiency, we added a cooperativecoevolution mechanism into original DE algorithm to makeadjoining pixels cooperatively search for the best sample pair.In order to prevent CC-DE-S from premature convergence,

123

Page 11: Improving sampling-based image matting with cooperative coevolution differential ... · 2018. 5. 23. · Soft Comput (2017) 21:4417–4430 DOI 10.1007/s00500-016-2250-7 METHODOLOGIES

Improving sampling-based image matting with cooperative coevolution differential… 4427

Table 1 Comparison of themean squared error (part I)

No. Random search PSO algorithm DE algorithm CC-DE-S algorithm

Image01

Best 13.9 × 10−3 1.25 × 10−3 1.27 × 10−3 1.18 × 10−3

Average 28.35 × 10−3 1.37 × 10−3 1.37 × 10−3 1.27 × 10-3

(Standarddeviation) (7.68 × 10−3) (0.06 × 10−3) (0.07 × 10−3) (0.06 × 10-3)

Image02

Best 7.02 × 10−3 8.23 × 10−3 3.12 × 10−3 2.72 × 10−3

Average 11.06 × 10−3 8.63 × 10−3 3.16 × 10−3 3.03 × 10-3

(Standarddeviation) (1.36 × 10−3) (0.37 × 10−3) (0.04 × 10−3) (0.12 × 10-3)

Image03

Best 13.00 × 10−3 4.21 × 10−3 12.34 × 10−3 10.02 × 10−3

Average 14.34 × 10−3 11.43 × 10−3 13.23 × 10−3 10.49 × 10−3

(Standarddeviation) (0.49 × 10−3) (3.80 × 10−3) (0.49 × 10−3) (0.27 × 10−3)

Image04

Best 100.17 × 10−3 20.10 × 10−3 17.45 × 10−3 13.16 × 10−3

Average 110.29 × 10−3 26.87 × 10−3 18.97 × 10−3 13.59 × 10-3

(Standarddeviation) (1.86 × 10−3) (1.35 × 10−3) (1.29 × 10−3) (0.33 × 10-3)

Image05

Best 15.80 × 10−3 2.20 × 10−3 2.54 × 10−3 2.01 × 10−3

Average 16.04 × 10−3 2.25 × 10−3 2.76 × 10−3 2.31 × 10−3

(Standarddeviation) (0.22 × 10−3) (0.03 × 10−3) (0.12 × 10−3) (0.13 × 10−3)

Image06

Best 10.22 × 10−3 2.67 × 10−3 1.96 × 10−3 1.95 × 10−3

Average 10.89 × 10−3 2.86 × 10−3 2.03 × 10−3 1.98 × 10-3

(Standarddeviation) (0.14 × 10−3) (0.12 × 10−3) (0.05 × 10−3) (0.02 × 10-3)

Image07

Best 15.65 × 10−3 2.57 × 10−3 2.29 × 10−3 2.06 × 10−3

Average 16.50 × 10−3 3.17 × 10−3 3.90 × 10−3 2.22 × 10-3

(Standarddeviation) (0.50 × 10−3) (0.21 × 10−3) (1.03 × 10−3) (0.10 × 10-3)

Image08

Best 57.26 × 10−3 12.32 × 10−3 26.47 × 10−3 23.23 × 10−3

Average 61.58 × 10−3 13.89 × 10−3 27.04 × 10−3 26.35 × 10-3

(Standarddeviation) (2.38 × 10−3) (1.73 × 10−3) (0.41 × 10−3) (0.61 × 10-3)

Image09

Best 12.54 × 10−3 2.50 × 10−3 4.58 × 10−3 4.58 × 10−3

Average 13.45 × 10−3 3.27 × 10−3 4.70 × 10−3 4.61 × 10-3

(Standarddeviation) (0.60 × 10−3) (0.43 × 10−3) (0.12 × 10−3) (0.07 × 10-3)

Image10

Best 8.12 × 10−3 3.11 × 10−3 3.40 × 10−3 3.08 × 10−3

Average 8.86 × 10−3 3.62 × 10−3 3.58 × 10−3 3.24 × 10-3

(Standarddeviation) (0.12 × 10−3) (0.32 × 10−3) (0.14 × 10−3) (0.10 × 10-3)

Image11

Best 12.97 × 10−3 4.04 × 10−3 5.51 × 10−3 4.26 × 10−3

Average 13.32 × 10−3 4.87 × 10−3 5.58 × 10−3 4.97 × 10−3

(Standarddeviation) (0.14 × 10−3) (0.31 × 10−3) (0.02 × 10−3) (0.21 × 10−3)

123

Page 12: Improving sampling-based image matting with cooperative coevolution differential ... · 2018. 5. 23. · Soft Comput (2017) 21:4417–4430 DOI 10.1007/s00500-016-2250-7 METHODOLOGIES

4428 Z.-Q. Cai et al.

Table 1 continuedNo. Random search PSO algorithm DE algorithm CC-DE-S algorithm

Image12

Best 4.86 × 10−3 2.82 × 10−3 2.72 × 10−3 2.33 × 10−3

Average 4.91 × 10−3 3.04 × 10−3 2.96 × 10−3 2.55 × 10-3

(Standarddeviation) (0.03 × 10−3) (0.08 × 10−3) (0.11 × 10−3) (0.15 × 10−3)

Image13

Best 57.17 × 10−3 21.58 × 10−3 24.03 × 10−3 19.74 × 10−3

Average 64.73 × 10−3 22.65 × 10−3 25.64 × 10−3 22.03 × 10−3

(Standarddeviation) (7.39 × 10−3) (0.77 × 10−3) (1.04 × 10−3) (0.68 × 10−3)

Image14

Best 4.60 × 10−3 2.00 × 10−3 2.36 × 10−3 1.94 × 10−3

Average 4.69 × 10−3 2.21 × 10−3 2.62 × 10−3 2.01 × 10-3

(Standarddeviation) (0.04 × 10−3) (0.07 × 10−3) (0.21 × 10−3) (0.08 × 10-3)

Image15

Best 9.25 × 10−3 3.88 × 10−3 6.13 × 10−3 4.33 × 10−3

Average 9.87 × 10−3 4.22 × 10−3 7.17 × 10−3 4.46 × 10-3

(Standarddeviation) (0.15 × 10−3) (0.05 × 10−3) (0.53 × 10−3) (0.08 × 10-3)

Image16

Best 80.49 × 10−3 62.30 × 10−3 63.10 × 10−3 61.85 × 10−3

Average 85.71 × 10−3 64.71 × 10−3 73.20 × 10−3 62.60 × 10-3

(Standarddeviation) (2.65 × 10−3) (0.80 × 10−3) (6.46 × 10−3) (0.32 × 10-3)

Image17

Best 10.92 × 10−3 2.58 × 10−3 2.48 × 10−3 2.32 × 10−3

Average 11.25 × 10−3 2.66 × 10−3 3.24 × 10−3 2.49 × 10-3

(Standarddeviation) (0.18 × 10−3) (0.07 × 10−3) (0.50 × 10−3) (0.11 × 10-3)

Image18

Best 11.64 × 10−3 3.81 × 10−3 3.82 × 10−3 3.61 × 10−3

Average 12.10 × 10−3 4.48 × 10−3 3.87 × 10−3 2.79 × 10-3

(Standarddeviation) (0.28 × 10−3) (0.25 × 10−3) (0.03 × 10−3) (0.05 × 10-3)

Image19

Best 6.18 × 10−3 0.94 × 10−3 1.25 × 10−3 0.97 × 10−3

Average 6.21 × 10−3 1.18 × 10−3 1.26 × 10−3 1.00 × 10-3

(Standarddeviation) (0.02 × 10−3) (0.04 × 10−3) (0.01 × 10−3) (0.01 × 10-3)

Image20

Best 4.45 × 10−3 1.94 × 10−3 2.24 × 10−3 2.15 × 10−3

Average 4.67 × 10−3 2.13 × 10−3 2.32 × 10−3 2.21 × 10−3

(Standarddeviation) (0.10 × 10−3) (0.11 × 10−3) (0.06 × 10−3) (0.04 × 10−3)

The Wilcoxon rank-sum test was conducted between the CC-DE-S algorithm and other three algorithms. Ifthe MSE values obtained by CC-DE-S algorithm are statistically different, the average MSE value withstandard deviation of CC-DE-S algorithm is highlighted in bold

we used a conditional scattered strategy to keep the diversityof population. Furthermore, a simple but effective evaluationfunction was proposed to distinguish the quality of variousindividuals. We compared the matting results of the CC-DE-S algorithm, existing random search algorithm, original DEalgorithm with PSO algorithm on a public image data set.The visual matting results and quantitative evaluation metricshow that the CC-DE-S algorithm can obtain higher-quality

sample pairs in sample optimization process and improve thematting quality of sampling-based image matting.

However, the CC-DE-S algorithm is still not the most effi-cient to all experimental images. For the futurework, it wouldbe helpful to design a more reasonable grouping methodby considering more information among pixels. Meanwhile,whether there is an adaptive strategy to control the diversityof population is also worthy of discussion.

123

Page 13: Improving sampling-based image matting with cooperative coevolution differential ... · 2018. 5. 23. · Soft Comput (2017) 21:4417–4430 DOI 10.1007/s00500-016-2250-7 METHODOLOGIES

Improving sampling-based image matting with cooperative coevolution differential… 4429

Table 2 Comparison of themean squared error (Part II)

No. Random search PSO algorithm DE algorithm CC-DE-S algorithm

Image21

Best 22.50 × 10−3 4.84 × 10−3 4.69 × 10−3 4.56 × 10−3

Average 22.92 × 10−3 5.29 × 10−3 5.70 × 10−3 4.90 × 10-3

(Standarddeviation) (0.39 × 10−3) (0.18 × 10−3) (0.83 × 10−3) (0.23 × 10-3)

Image22

Best 11.15 × 10−3 1.94 × 10−3 1.86 × 10−3 1.84 × 10−3

Average 12.07 × 10−3 2.07 × 10−3 1.96 × 10−3 1.95 × 10−3

(Standarddeviation) (0.19 × 10−3) (0.04 × 10−3) (0.09 × 10−3) (0.06 × 10−3)

Image23

Best 4.49 × 10−3 3.76 × 10−3 3.60 × 10−3 3.15 × 10−3

Average 4.58 × 10−3 3.87 × 10−3 4.68 × 10−3 3.36 × 10-3

(Standarddeviation) (0.07 × 10−3) (0.04 × 10−3) (0.61 × 10−3) (0.13 × 10-3)

Image24

Best 6.29 × 10−3 4.45 × 10−3 5.06 × 10−3 4.76 × 10−3

Average 7.04 × 10−3 4.88 × 10−3 5.12 × 10−3 5.05 × 10−3

(Standarddeviation) (0.32 × 10−3) (0.08 × 10−3) (0.05 × 10−3) (0.20 × 10−3)

Image25

Best 24.96 × 10−3 15.50 × 10−3 15.98 × 10−3 15.20 × 10−3

Average 25.37 × 10−3 15.80 × 10−3 16.34 × 10−3 15.50 × 10-3

(Standarddeviation) (0.22 × 10−3) (0.19 × 10−3) (0.34 × 10−3) (0.24 × 10-3)

Image26

Best 47.48 × 10−3 25.44 × 10−3 29.41 × 10−3 24.87 × 10−3

Average 48.01 × 10−3 27.44 × 10−3 30.47 × 10−3 27.12 × 10−3

(Standarddeviation) (0.34 × 10−3) (0.49 × 10−3) (1.06 × 10−3) (0.79 × 10−3)

Image27

Best 20.88 × 10−3 16.33 × 10−3 16.15 × 10−3 15.90 × 10−3

Average 21.16 × 10−3 17.65 × 10−3 17.37 × 10−3 17.31 × 10−3

(Standarddeviation) (0.15 × 10−3) (0.64 × 10−3) (0.65 × 10−3) (0.63 × 10−3)

The Wilcoxon rank-sum test was conducted between the CC-DE-S algorithm and other three algorithms. Ifthe MSE values obtained by CC-DE-S algorithm are statistically different, the average MSE value withstandard deviation of CC-DE-S algorithm is highlighted in bold

Funding This study was funded by National Natural Science Founda-tion of China (61370102, 61170193, 61370185), Guangdong NaturalScience Foundation (2014A030306050, S2012010009865, s2013010013432, S2013010015940), the Fundamental Research Funds for theCentral Universities, SCUT (2015PT022), Science and TechnologyPlanning Project of Huizhou City (2011P002, 2011g012, 2011P005,2011P003, 2011g011, 2013B020015008) and Science and Technol-ogy Planning Project of Guangdong Province (2011B090400041,2012B010100039, 2012B040305011, 2012B010100040, 2015B010129015). Education and Science Programs of Guangdong Province(11JXZ012, 14JXN065), Discipline Construction Programs of Guang-dong Province (2013LYM00874), Key Technology Research andDevelopment Programs of Huizhou (2013-13, 2013B020015008,2014B050013016), Science and Technology Plan Project of HuizhouUniversity (2012QN09),DistinguishedYoungScholars FundofDepart-ment of Education (No. Yq2013126).

Compliance with ethical standards

Conflicts of interest All authors of this paper declare that they haveno conflict of interest.

Ethical approval This article does not contain any studies with humanparticipants or animals performed by any of the authors.

References

Barnes C, Shechtman E, Finkelstein A, Goldman DB (2009) Patch-match: a randomized correspondence algorithm for structuralimage editing. ACM Trans Gr (TOG) 28(3):24

123

Page 14: Improving sampling-based image matting with cooperative coevolution differential ... · 2018. 5. 23. · Soft Comput (2017) 21:4417–4430 DOI 10.1007/s00500-016-2250-7 METHODOLOGIES

4430 Z.-Q. Cai et al.

Cheng R, Jin Y (2014) A competitive swarm optimizer for large scaleoptimization. IEEE Trans Cybern 45(2):191–204

Comaniciu D, Meer P(1997) Robust analysis of feature spaces: colorimage segmentation. In: Proceedings of IEEE computer societyconference on computer vision and pattern recognition, pp 750–755

Comaniciu D, Meer P (2002) Mean shift: a robust approach towardfeature space analysis. IEEE Trans Pattern Anal Mach Intell24(5):603–619

Eberhart R, Kennedy J (1995) A new optimizer using particle swarmtheory. In: Proceedings of sixth international symposium on micromachine and human science, pp 39–43

Fielding R (1972) The technique of special effects cinematography, 3rdedn. Focal/Hastings House, London, pp 220–243

Gastal ESL, Oliveira MM (2010) Shared sampling for real-time alphamatting. Comput Gr Forum 29(2):575–584

Gong W, Cai Z, Ling XC, Li H (2011) Enhanced differential evolutionwith adaptive strategies for numerical optimization. IEEE TransSyst Man Cybern 41(2):397–413

Gu B, Sheng VS, Tay KY, RomanoW, Li S (2015) Incremental supportvector learning for ordinal regression. IEEE Trans Neural NetwLearn Syst 26(7):1403–1416

Guo P, Li N (2015) Self-adaptive threshold based on differential evolu-tion for image segmentation. In: 2015 2nd international conferenceon information science and control engineering, pp. 466–470

He K, Rhemann C, Rother C, Tang X, Sun J (2011) A global samplingmethod for alphamatting. In: IEEE conference on computer visionand pattern recognition (CVPR), pp. 2049–2056

Kennedy J, Eberhart R (1995) Particle swarm optimization. In: Pro-ceedings of IEEE International Conference Neural Networks, p1942C1948

Lee P,WuY (2011)Nonlocalmatting. In: IEEE conference on computervision and pattern recognition CVPR 2011, pp 2193–2200

Liu Y, Yao X, Zhao Q, Higuchi T (2001) Scaling up fast evolutionaryprogramming with cooperative coevolution. In: Proceedings of the2001 congress on evolutionary computation, vol 2, pp 1101–1108

Mezura-Montes E, Velazquez-Reyes J, Coello CAC (2006) A compara-tive study of differential evolution variants for global optimization.In: Genetic and Evolutionary Computation Conference (GECCO),pp 485–492

Mhlenbein H (1989) Parallel genetic algorithms, population genetics,and combinatorial optimization. In: The workshop on parallelism,learning, evolution, workshop on evolutionary MODELS andstrategies, vol 565, Neubiberg, Germany, pp 398–406

Miao Q, Shi C, Xu P, YangM, Shi Y (2011) A novel algorithm of imagefusion using shearlets. Opt Commun 284(6):1540–1547

Miao Q, Xu P, Liu T, Yang Y, Zhang J, Li W (2013) Linear featureseparation from topographic maps using energy density and sheartransform. IEEE Trans Image Process 22(4):1548–1558

Omidvar MN, Li X, Mei Y (2014) Cooperative co-evolution with dif-ferential grouping for large scale optimization. IEEE Trans EvolComput 18(3):378–393

OmidvarMN, LiX,YaoX (2011) Smart use of computational resourcesbased on contribution for cooperative co-evolutionary algorithms.In: Proceedings of the 13th annual conference on genetic and evo-lutionary computation(GECCO’11), pp 1115–1122

Pan Z, Zhang Y, Kwong S (2015) Efficient motion and disparity esti-mation optimization for low complexity multiview video coding.IEEE Trans Broadcast 61(2):166–176

Porter T, Duff T (1984) Compositing digital images. Siggraph18(3):253–259

Potter MA, Jong KAD (1994) A cooperative coevolutionary approachto function optimization. In: Proceedings of 3rd conference onparallel problem solving Natue, pp 249–257

Price K, Storn RM, Lampinen JA (2005) Differential evolution: a prac-tical approach to global optimization (natural computing series).Springer, Berlin 538

Qin AK, Huang VL, Suganthan PN (2009) Differential evolution algo-rithm with strategy adaptation for global numerical optimization.IEEE Trans Evol Comput 13(2):398–417

Rhemann C, Rother C, Gelautz M (2008) Improving color modelingfor alpha matting. BMVC 1(2):1155–1164

Rhemann C, Rother C, Wang J, Gelautz M, Kohli P, Rott P (2009)A perceptually motivated online benchmark for image matting.IEEE Conference on Computer Vision and Pattern Recognition,pp 1826–1833

Sarangi PP, Mishra BSP, Majhi B, Dehuri S (2014) Gray-level imageenhancement using differential evolution optimization algorithm.In: 2014 international conference on signal processing and inte-grated networks (SPIN), pp 95–100

Shi C, Miao Q, Xu P (2013) A novel algorithm of image fusion basedon shearlets and pcnn. Neurocomputing 117(10):47–53

Shi Y, Eberhart R (1998) Amodified particle swarm optimizer. In: IEEEinternational conference on evolutionary computation, pp 69–73

Storn R, Price K (1997) Differential evolution—a simple and efficientheuristic for global optimization over continuous spaces. GlobOptim 11(4):341–359

Wang J, Cohen MF (2005) An iterative optimization approach for uni-fied image segmentation andmatting. In: Tenth IEEE internationalconference on computer vision (ICCV), vol 2, pp 936–943

Wang J, CohenMF (2007)Optimized color sampling for robustmatting.In: IEEE computer society conference on computer vision andpattern recognition (CVPR), pp 1–8

Xia Z, Wang X, Sun X, Liu Q, Xiong N (2016) Steganalysis of LSBmatching using differences between nonadjacent pixels.MultimedTools Appl 75(4):1947–1962

Xia Z, Wang X, Sun X, Wang B (2014) Steganalysis of least significantbit matching using multi-order differences. Secur Commun Netw7(8):1283–1291

Xie S, Wang Y (2014) Construction of tree network with limited deliv-ery latency in homogeneous wireless sensor networks. Wirel PersCommun 78(1):231–246

Xu P,MiaoQ, Shi C, Zhang J, LiW (2012) An edge detection algorithmbased on themulti-direction shear transform. JVisCommun ImageRepresent 23(5):827–833

Yang Z, Tang K, Yao X (2008) Large scale evolutionary optimizationusing cooperative coevolution. Inf Sci 178(15):2985–2999

Yu F, Li Y, Wei B, Kuang L (2015) Image retrieval based on interac-tive differential evolution. In: 2015 IEEE congress on evolutionarycomputation (CEC), pp 3124–3131

Zhang J, Sanderson AC (2009) Jade: Adaptive differential evolu-tion with optional external archive. IEEE Trans Evol Comput13(5):945–958

Zheng Y, Jeon B, Xu D, Wu QJ, Zhang H (2015) Image segmentationby generalized hierarchical fuzzy c-means algorithm. Intell FuzzySyst 28(2):961–973

123