Superpixels via Pseudo-Boolean Optimization

Yuhang Zhang, Richard HartleyThe Australian National University

{yuhang.zhang, richard.hartley }@anu.edu.au

John Mashford, Stewart BurnCSIRO

{john.mashford, stewart.burn }@csiro.au

Abstract

We propose an algorithm for creating superpixels. Themajor step in our algorithm is simply minimizing twopseudo-boolean functions. The processing time of our al-gorithm on images of moderate size is only half a second.Experiments on a benchmark dataset show that our methodproduces superpixels of comparable quality with existingalgorithms. Last but not least, the speed of our algorithm isindependent of the number of superpixels, which is usuallythe bottle-neck for traditional algorithms of the same type.

1. Introduction

Superpixel segmentation is an important preprocess-ing step in many image parsing applications [6, 10, 11].Through semantically grouping pixels in local neighbour-hoods, superpixels provide a more compact representationof the original image, which usually leads to great im-provement in computational efficiency. We here proposea new superpixel segmentation method which runs fasterthan most existing fast algorithms, yet achieves comparableor even better results.

The idea and the first superpixel algorithm was proposedby Ren and Malik [12] based on Normalized Cuts [13].After that, Normalized Cuts became the major means ofsuperpixel segmentation [10, 11]. Despite its high accu-racy, the heavy computation required by Normalized Cutsusually makes superpixel segmentation a long procedure.By simply perceiving superpixels as an over-segmentationto the original image, some less expensive segmentationmethods like Mean Shift [4] and Graph-based Segmenta-tion [5] could be used. However, superpixels produced inthat way are usually arbitrary in size and shape, hence nolonger as comparable as ‘pixels’. More recently, several fasthigh quality superpixel algorithms like SuperLattices [9],TurboPixels [7] and Superpixels via Expansion-Moves [14]have shortened the processing time of superpixel segmenta-tion from minutes to seconds.

Our work is most close to that of Veksleret al. [14],

in which superpixel segmentation is modelled as a ‘very-many-label’ energy minimization problem and solved withExpansion-Moves [2]. Although Expansion-Moves isknown to be efficient and reliable, we seek even better ef-ficiency through modelling the segmentation problem withonly two labels, hence no need for heuristic expansion. Bydefault, our two-label energy function is submodular, hencecan be readily minimized. To enforce more constraints forbetter quality, the energy function can be modified to benonsubmodular. However, we still find efficient algorithmsto compute a quality suboptimal solution. The result is, ourmethod can produce quality superpixels in times of less thana second.

2. Related Work

To help understand the difference between our methodand that of Veksleret al. [14], we here mainly review theirmethod. The other methods will appear only in the experi-mental comparison section. Readers are referred to the orig-inal paper for the information of the other superpixel algo-rithms. A brief review of pseudo-boolean optimization willbe conducted here as well.

Figure 1. Four square patchesY , G, B, P are half overlapping.Pixels is covered by all of the four patches.

2.1. Superpixels via Expansion-Moves

Veksleret al. assume the input image is intensively cov-ered by half overlapping square patches of the same size(Figure 1). Each square patch corresponds to a label. There-fore, hundreds of labels are generated. They then assign

each pixel to a unique patch via assigning it a label. Theassignment is accomplished through minimizing an energyfunction composed of data terms and smoothing terms. Ifa patch does not cover a pixel, the data cost for the pixelto belong to that patch will be positive infinite; otherwise,the data cost is constant one. A smoothing cost will be in-curred if two neighbouring pixels are assigned to differentpatches. This smoothing cost is a decreasing function ofthe colour difference between the two pixels. The effectof minimizing such an energy function is that, every pixelhas equal chance to belong to each of the four patches cov-ering it, but no chance to the other patches; pixels tend tohave the same label everywhere, yet to avoid the infinitecost, smoothness tends to be compromised at sharp colourdiscontinuities. The superpixels produced in this way arenamed as Compact Superpixels.

The above design has several merits. The size of an in-dividual patch is an upper bound for the size of a singlesuperpixel. The smooth costs suppress appearance of super-pixels of small sizes or irregular shapes which will result inmore discontinuities. Although the number of labels is quitelarge, during Expansion-Moves, particularly in each singlegraph cut iteration, the number of pixels actually involvedis quite small. That is because most of the pixels can neverhave most of the labels, and therefore may be excluded fromthe graph. The authors pointed out that, the running speed isreally proportional to the number of patches. The more thepatches, the smaller each patch is, the smaller each graph is.Accordingly they proposed a method named Variable PatchSuperpixels, which increases the density of patches in ar-eas of high image variance, to boost the speed. Thereforeprocessing a moderate image only takes seconds.

The major problem with the above design is that, neigh-bouring pixels are encouraged to have the same label every-where. Even if two pixels are completely different in colour,having different labels will still increase but not decrease thecost. Label discontinuity will only be encouraged, when thesize of a single superpixel becomes larger than the patch.In other words, energy minimization tries to conceal edgesas long as the maximum size of the superpixel is not vio-lated. Therefore, some strong edges can be observed insidethe resulting superpixels. To tackle this problem, Veksleret al. proposed to first assign each patch a color, which isthe colour of the pixel at the center of the patch. Then thedata cost for each pixel to belong to a patch covering itselfis proportional to the colour difference between the digitalpixel and the patch. Moreover, to make the colour of thepatch reasonable, the pixel on the center of each patch mustbelong to the patch if the patch is not empty. To enforce thisconstraint, four extra terms are added for each pixel in theimage. The optimization process is hence slowed down, butthe resulting superpixels become more coherent in colour.The superpixels produced in this way are called Constant

Intensity Superpixels.

2.2. Pseudo-Boolean Optimization

Many low-level computer vision problems can be formu-lated as minimizing discrete energy functions, which mea-sure the energy in Markov Random Fields (MRFs). A sim-ple case is when the function is quadratic and the variablesin the function are Boolean-valued:

E(x) = θc +∑p∈V

θp(xp) +∑

(p,q)∈E

θpq(xp, xq) . (1)

wherexp ∈ {0, 1}, θp andθpq are usually referred to asthe data term and smoothing term respectively. If for all(p, q) ∈ E it holds that,

θpq(0, 0) + θpq(1, 1)− θpq(0, 1)− θpq(1, 0) ≤ 0 , (2)

we say this pseudo-boolean function is submodular.We can always convert a submodular function into a di-

rected graphG = (V, E) containing no negative edges,and optimize it exactly with algorithms such as Max-Flow.However, if there exists(p, q) ∈ E , which violates con-straint (2), the function becauses nonsubmodular. Optimiz-ing a nonsubmodular function is in general NP-hard evenwhen only two labels are involved. We usually need an ap-proximate algorithm for nonsubmodular functions.

In this work, we use the Elimination algorithm [3] pro-posed by Carr and Hartley to optimize the pseudo-Booleanenergy functions. Through recursive elimination and backsubstitution, the minimum of a quadratic pseudo-Booleanfunction can be approximated efficiently no matter if it issubmodular or not.

3. Superpixels via Binary Labelling

The basic idea in the work of Veksleret al. was devel-oped into three different variants,i.e. Compact Superpixels,Variable Patch Superpixels and Constant Intensity Super-pixels. All the three variants finally appeal to multi-labeloptimization. Now we explain our basic design and de-velop it into two variants. Our first algorithm producesresults similar to the Compact Superpixels, through min-imizing two-label submodular functions. Our second al-gorithm produces results similar to the Constent IntensitySuperpixels, through minimizing two-label nonsubmodularfunctions. We do not have a counterpart for Variable PatchSuperpixels, because the processing time of our method isindependent of the size or number of superpixels. There-fore, varying superpixel density in the image will not in-crease our running speed, or should we say it in a positiveway, producing more or fewer superpixels does not slowdown our algorithm.

3.1. The Submodular Formulation

First we cover an image with two sets of half overlap-ping stripsHi andVi as shown in Figure 2. Either set ofstrips fully covers the image by itself. The first set of stripsare horizontal and have equal width with the image. Thesecond set of strips are vertical and have equal height withthe image. Obviously, each pixel in the image is containedby two horizontal strips as well as two vertical strips.

Figure 2. Two sets of strips used to cover image.H0, H1 andH2

are the horizontal strips;V0, V1 andV2 are the vertical strips.

What we are really interested in, however, is the mem-bership of each pixel with respect to another two sets oflatent strips{H ′

i} and {V ′i }. This time, each pixel only

belongs to one strip in{H ′i} and one strip in{V ′

i }. In par-ticular, we define that:

• if p ∈ H2i andlabel(p) = 0, thenp ∈ H ′2i;

• if p ∈ H2i+1 andlabel(p) = 1, thenp ∈ H ′2i+1.

We can see that everyH ′i is a subset ofHi. The intersection

of any twoH ′is is empty. The union of allH ′

i contains all thepixels. Therefore,{H ′

i} is a non-overlapping segmentationof the image. Via assigning label0 or 1 to each pixel in theimage, we can accomplish this segmentation. Equivalentdefinition and statement holds for{V ′

i }.We now define the energy function for label assignment.

As the processing of the two sets of latent strips are the sameand independent, we here focus on{H ′

i} only. By havingone of the two alternative labels, each pixel has the chanceto be assigned to two alternative latent strips. We make thechance equal, so the data cost for any pixel having eitherlabel is constant0. Recall the energy function in (1), theabove statement really suggests

∀p, θp(xp) = 0 , (3)

so there is indeed no data terms in our energy function.For the smoothing term, we assume the image is 4-

connected. We encourage neighbour pixels to belong to thesame latent strips and penalize the opposite case. However,following our previous design, identical labels do not al-ways guarantee identical strips. Given a pair of neighbour-ing pixelp andq, without losing generality, we assumep is

aboveq or on the same horizontal line withq in the image.There are three complementary situations forp andq:

• p ∈ H2i ∩H2i+1,q ∈ H2i ∩H2i+1;

• p ∈ H2i ∩H2i+1,q ∈ H2i+1 ∩H2i+2;

• p ∈ H2i+1 ∩H2i+2,q ∈ H2i+2 ∩H2i+3.

Particularly, in the first situation, the same label always in-dicates the same strip. In the second situation, label0 in-dicates different strips for different pixels. In the third sit-uation, label1 indicates different strips for different pixels.Combining the above three situations with the four possiblelabel states of the two pixels, we obtain12 possible smoothcosts, as shown in Table 1, wherec is a positive decreasingfunction of the colour difference between pixelp andq:

c = exp(−|Ip − Iq|2σ2

) , (4)

and∆ is the submodularity criterion:

∆ = θpq(0, 0) + θpq(1, 1)− θpq(0, 1)− θpq(1, 0) . (5)

As the table shows, the submodular criterion is negative un-der all situations. Therefore, our energy function is sub-modular. Based on existing algorithms, minimizing such atwo-label problem is easy and fast [1, 3].

(xp, xq)p ∈ H2i ∩ H2i+1 p ∈ H2i ∩ H2i+1 p ∈ H2i+1 ∩ H2i+2

q ∈ H2i ∩ H2i+1 q ∈ H2i+1 ∩ H2i+2 q ∈ H2i+2 ∩ H2i+3

(0, 0) 0 c 0(0, 1) c c c(1, 0) c c c(1, 1) 0 0 c

∆ -2c -c -cTable 1. value of the smoothing term in our submodular formula-tion.

Function c is deliberately chosen to be convex aboutthe colour difference. While minimizing the energy in anMRF, the smoothing cost tends to condense the length oflabel discontinuity. Such an effect can not only regularizethe shape of superpixels, but also unwantedly flatten zigzagedges and sharp corners. A convex cost makes a detouralong sharp colour discontinuities preferable to a short-cutthrough colour coherent areas.

Figure 3(a) is an input image. After assigning pixels to{H ′

i}, the boundaries between pixels belonging to differentstrips are presented in Figure 3(d). The same treatment canbe implemented for{V ′

i } and we obtain Figure 3(e). Byoverlaying the two layers of memberships, we obtain the

superpixels in Figure 3(b). The boundaries between super-pixels in Figure 3(f) are exactly the overlaid horizontal andvertical strip boundaries.

The superpixels produced by our method are regular inboth size and shape. The maximum size of a superpixel islimited by the width of the stripsHi andVi. Small sizes andirregular shapes are suppressed by smoothing cost. Most ofthe superpixels generally exist in a regular lattice configu-ration, while adjusting their boundaries to the curves in theimage content.

An intuitive explanation of our method might be, whileassigning pixels sequentially to two sets of strips, we re-ally find the most probable horizontal and vertical splittinglines on the input image. Then by breaking the image alongthese splitting lines, we obtain the superpixels. A similaridea appeared in the work of Mooreet al. [9], but led toa completely different solution. In the proposed method,we estimate the membership of pixels which then form theboundaries. However, in the work of Mooreet al., bound-aries are estimated first which then determine the member-ship of pixels. Experiment will show that the superpixelsproduced by our method is much more accurate.

(a) (b) (c)

(d) (e) (f)

Figure 3. (a) input image; (b) superpixels produced by submodularformulation; (c) superpixels produced by nonsubmodular formu-lation; (d) boundaries between horizontal strips; (e) boundariesbetween vertical strips; (f) boundaries between superpixels in (b).

It might cause some concern that, our method only ex-plicitly detects the horizontal and vertical edges in the im-age, but risks leaving diagonal edges unattained. We no-ticed this issue while developing our algorithm. How-ever, experiment shows our algorithm is in fact quite ro-

bust against diagonal edges as well (see any Figure contain-ing superpixels produced by our method). That is because,although we place the strips horizontally and vertically, itdoes not prevent the splitting lines between strips from aris-ing in any direction, especially via the convex smoothingcost. Even if it really forms a limitation of our method, wecan simply fix it by adding another two sets of strips in thediagonal directions.

3.2. The Nonsubmodular Formulation

As we have discussed while reviewing the work ofsuperpixels via Expansion-Moves [14], using overlappingpatches to split pixels does not guarantee colour coherencewithin superpixels. This defect remains as we formulate theproblem with two labels. Figure 4(a) shows an enlargedpart of Figure 3(b), where one superpixel contains both treetrunk and sea water. To improve the situation, we proposeour second formulation.

(a) (b) (c)

Figure 4. (a) superpixels via submodular formulation; (b) super-pixels via nonsubmodular formulation; (c) more intensive super-pixels via submodular formulation.

In the previous formulation, we encourage neighbouringpixels to belong to the same strip regardless of their colours.See the zero entries in Table 1. We now modify these zeroentries by adding the constraint that,

• if the colour difference between two neighbour pix-els exceeds a thresholdτ , the cost of belonging to thesame strip is1− c.

With this constraint, the more different two neighbouringpixels are, the smallerc will be, the larger the cost for themto belong to the same strip will be. Hence colour differ-ence in the same superpixels is suppressed. Although thischange may cause some of the smoothing terms to becomenonsubmodular, just observe what if the zero entries are re-placed with1−c, we can still minimize the energy functionefficiently with the Elimination algorithm [3].

Using the second formulation, we obtain the superpixelsin Figure 3(c) and Figure 4(b). As we can see, the tree trunkand the sea water are now separated.

On one hand, our second formulation has a similar draw-back to Constant Intensity Superpixels,i.e. the additionalconstraint makes the superpixels more irregular in shapeand size. In particular, a number of tiny superpixels arecreated and the total number of superpixels is substantially

increased. On the other hand, our second formulation treatsregions with gradual colour change differently from Con-stant Intensity Superpixels. Our method allows gradualchanging regions to be contained in one superpixel, becauseonly sharp colour differences incur costs. Nevertheless,Constant Intensity Superpixels forbids both sharp changeand gradual change. It is hard to claim either mechanism tobe better, because whether regions of gradual change shouldbe treated as a single image patch varies from case to case.Our second formulation and Constant Intensity Superpixelsreally provide two alternative options.

Another concern is that, if the nonsubmodular formula-tion produces superpixels smaller in size and more in quan-tity, it is not surprising that they can recall more edges. Canwe do the same thing simply via the submodular formu-lation through increasing the number of superpixels? Theanswer is partly yes. We can always retrieve more edgepixels simply via increasing the number of superpixels, asFigure 4(c) shows. In fact, no superpixel algorithm can re-trieve all edges with an insufficient number of superpixels.Whereas for most of the algorithms producing more super-pixels means longer processing time, for our algorithm thetime is a constant. However, at the same time, we cannotsimply perceive the functionality of the nonsubmodular for-mulation as retrieving more edges by way of segmentingimages into more pieces. We will see the disproof soon inthe experiment.

Here we give the outline of the proposed algorithms:

Submodular formulation

1. cover the image with half overlapping horizontal strips{Hi};

2. assign binary labels to each pixel so as to minimize thecost specified in Table 1, which generates{H ′

i};

3. implement Step 1 and 2 to vertical strips{Vi}, whichgenerates{V ′

i };

4. let Sij = H ′i ∩ V ′

j , if Sij 6= ∅, thenSij is a superpixel.

Nonsubmodular formulation

1. follow all the steps of submodular formulation exceptfor replacing the zero entries in Table 1 with a if-elsejudgement:{

1− c , if |Ip − Iq| < τ0 , otherwise

,

whereIp andIq are the colour values of pixelp andq.

4. Experiment

We consider four superpixel algorithms here, the pro-posed method, the Expansion-Moves based method [14],SuperLattices [9] and the Normalized-Cuts basedmethod [10]. The code for the other algorithms wasobtained from the corresponding authors. The pseudo-Boolean optimization step in our algorithm is implementedby the Elimination algorithm [3]. Detailed comparisonis conducted among the first three algorithms which areall designed for speedy superpixel segmentation. TheNormalized-Cuts based method is included only for thepurpose of reference.

The images are downloaded from the Berkeley Segmen-tation Dataset [8]. The edges on these images have beenlabelled by human. If a pixel on these edges is also adoptedas the boundaries between superpixels, we say this pixel isrecalled. We use the recall rate to measure the segmenta-tion accuracy of a superpixel algorithm. To compensate forthe errors due to human subjective bias, a tolerance factort is used. That is, an edge pixel is recalled as long as apixel within distancet from it is adopted as the boundarybetween superpixels. Whereas each image was labelled bymultiple users, we always choose the most labelled result asour benchmark. Each image is481× 321 in resolution.

4.1. Submodular Formulation

We first evaluate our submodular formulation againstCompact Superpixels [14] and SuperLattices [9]. All thethree algorithms have a parameter defining the size ofpatches or strips. We vary these parameters together andcompare their performance. The size of patches, or equiv-alently the width of strips are set to10, 11, 12, 15 and18pixels respectively. With each of these values, the three al-gorithms are implemented on200 images with the other pa-rameters fixed. The average recall rate is measured whilesetting the tolerance factort to one and two pixels respec-tively. The result is plotted in Figure 5.

(a) (b)

Figure 5. recall rate against patch size with tolerance factor (a)t = 1 pixel; (b) t = 2 pixels.

From the figures we see that, based on the same sizeor number of patches, our submodular formulation recallsmore edges than the other two. SuperLattices is the second

whereas Compact Superpixels is the last. Nevertheless, therecall rates of all the three algorithms are quite close.

The size of patches and strips sets the lower bound andupper bound of the number of superpixels. The lower boundis enforced via setting the largest size of a superpixel, whichcannot exceeds the size of a patch or the overlapping ofa horizontal and a vertical strips. The upper bound is en-forced by the number of patches or overlapping strips. Nev-ertheless, the actual number of superpixels is really deter-mined by the algorithm itself according to its optimizationtarget. We observe that, when based on patches or strips ofthe same size, different algorithms usually divide the sameimage into quite different numbers of superpixels. Usually,SuperLattices produces the most superpixels, followed byour submodular formulation and then Compact Superpix-els. If we plot the recall rate directly against the number ofsuperpixels, we obtain Figure 6.

(a) (b)

Figure 6. recall rate against number of superpixels with(a)tolerance factort = 1 pixel; (b)tolerance factort = 2 pixels.

In Figure 6 the three algorithms still present similar per-formance, yet Compact Superpixels becomes the first, fol-lowed by our submodular formulation and then SuperLat-tices. To conclude, based on the same patch size param-eter, SuperLattices produces the most superpixels, but oursubmodular formulation can recall the most edges. Basedon the same number of superpixels, Compact Superpixelscan recall the most edges, but our submodular formulationkeeps quite close.

The Normalized-Cuts based method uses a completelydifferent way to control the number of superpixels. Espe-cially due to its slow speed we can hardly implement it arbi-trarily as the other three algorithms to produce many resultsfor various comparisons. A single run on all200 imagestook us33 hours. The produced superpixels are1104 perimage on average and the recall rate is71.51 when the tol-erance factor is two pixels. If we plot it on Figure 6(b), itis below our method and slightly above the curve of Super-Lattice. It might be a bit surprising to see Normalized Cutsbased method being defeated by fast algorithms in quality.However, we are not the first to report this result [9]. Oneexplanation might be that, all superpixel algorithms needto balance two optimization criteria, namely the colour co-

herence within each superpixel as well as the regularity ofshape and size among superpixels; the optimization pro-cess of Normalized Cuts regards the regularity of the shapeand size of superpixels more importantly than the fast algo-rithms do. That is why the superpixels produced by Nor-malized Cuts are more neat and tidy. That is also why Nor-malized Cuts is outperformed by fast algorithms in accuracymeasurement from time to time.

4.2. Nonsubmodular Formulation

To reduce the edge pixels concealed by superpixels, wedeveloped our submodular formulation into the nonsub-modular formulation. We now check how much improve-ment can be achieved.

Figure 7 compares the recall of submodular and non-submodular formulation. Based on the images used in ourexperiment, the nonsubmodular formulation generally pro-duces30% more superpixels than the submodular formu-lation does. The recall rate can be increased by about fivepercent. From Figure 7(a) we see, when the patch size islarge, namely when patches are sparse and more edges tendto be concealed, the improvement due to nonsubmodularformulation is more significant. Figure 7(b) suggests withthe same number of superpixels, our nonsubmodular for-mulation performs better than the submodular formulation.That proves, the improvement is not obtained via more su-perpixels only.

(a) (b)

Figure 7. Improvement in recall rate due to nonsubmodular formu-lation, with tolerance factort = 2.

We conduct the same comparison between Compact Su-perpixels and Constant Intensity Superpixels [14], and plotthe result in Figure 8. One observation is that, Constant In-tensity Superpixels usually double the number of superpix-els generated by Compact Superpixels. This is obviouslymore aggressive a mechanism than our nonsubmodular for-mulation. It is also predictable as Constant Intensity Super-pixels does not allow gradual change whereas our methoddoes. Hence an improvement of around ten percent in therecall can be obtained. However, if based on the same num-ber of superpixels, Constant Intensity superpixels and ournonsubmodular formulation obtain quite similar recall rate.

(a) (b)

Figure 8. Improvement in recall rate due to Constant Intensity Su-perpixels, with tolerance factort = 2.

4.3. Efficiency

In the accuracy comparison, our methods always achievecomparable performance with Expansion-Moves basedmethods. However, in the efficiency aspect, our methodspossess a much more obvious advantage. To segmenting an481 × 321 image, whereas the Compact Superpixels takesfour or five seconds to converge, our submodular methodneeds only0.5 seconds. To obtain a better accuracy, Con-stant Intensity Superpixels requires eight or nine secondson the same image, but our nonsubmodular formulation stillfinishes the work in0.5 seconds.

An even faster speed can be found in SuperLattices,which can usually beat our method by10% in speed on thesame image. However, its quality is lower than our meth-ods, especially considering the regularity of the shape andsize of the resulting superpixels (see Figure 9).

We look forward to even faster implementation of ourmethod via dual-core CPU programming. In our method,assigning pixels to horizontal and vertical strips are two in-dependent processes, hence can be executed at the sametime to double the speed.

Some more examples of superpixel produced by ourmethods can be found in Figure 10 and 11.

5. Conclusion

We proposed two new formulations to create superpix-els based on pseudo-boolean optimization. In particular,our method can achieve the accuracy of Expansion-Movesbased superpixels with the speed of SuperLattices. The ef-ficiency and quality of our method makes a more valuablepresegmentation tool for applications in need of superpix-els.

References

[1] Y. Boykov and V. Kolmogorov. An experimental comparisonof min-cut/max-flow algorithms for energy minimization invision. IEEE Trans. Pattern Anal. Mach. Intell., 26:1124–1137, September 2004.

Figure 9. from top to bottom are superpixels produced byNormalized-Cuts, proposed submodular formulation, CompactSuperpixels and SuperLattice.

Figure 10. More superpixels produced by submodular formulation.

[2] Y. Boykov, O. Veksler, and R. Zabih. Fast approximateenergy minimization via graph cuts.Pattern Analysis andMachine Intelligence, IEEE Transactions on, 23(11):1222 –1239, Nov. 2001.

[3] P. Carr and R. Hartley. Minimizing energy functions on 4-connected lattices using elimination. InICCV, pages 2042–2049, 2009.

[4] D. Comaniciu and P. Meer. Mean shift: a robust approachtoward feature space analysis.Pattern Analysis and Ma-chine Intelligence, IEEE Transactions on, 24(5):603 –619,May 2002.

[5] P. F. Felzenszwalb and D. P. Huttenlocher. Efficient graph-based image segmentation.Int. J. Comput. Vision, 59:167–181, September 2004.

[6] B. Fulkerson, A. Vedaldi, and S. Soatto. Class segmentationand object localization with superpixel neighborhoods. InProceedings of the International Conference on ComputerVision, October 2009.

[7] A. Levinshtein, A. Stere, K. Kutulakos, D. Fleet, S. Dick-inson, and K. Siddiqi. Turbopixels: Fast superpixels usinggeometric flows.Pattern Analysis and Machine Intelligence,IEEE Transactions on, 31(12):2290 –2297, 2009.

[8] D. Martin, C. Fowlkes, D. Tal, and J. Malik. A databaseof human segmented natural images and its application to

Figure 11. More superpixels produced by submodular formulation.

evaluating segmentation algorithms and measuring ecologi-cal statistics. InProc. 8th Int’l Conf. Computer Vision, vol-ume 2, pages 416–423, July 2001.

[9] A. Moore, S. Prince, J. Warrell, U. Mohammed, andG. Jones. Superpixel lattices. InComputer Vision and Pat-tern Recognition, 2008. CVPR 2008. IEEE Conference on,pages 1 –8, 2008.

[10] G. Mori. Guiding model search using segmentation. InCom-puter Vision, 2005. ICCV 2005. Tenth IEEE InternationalConference on, volume 2, pages 1417 –1423 Vol. 2, 2005.

[11] G. Mori, X. Ren, A. A. Efros, and J. Malik. Recoveringhuman body configurations: Combining segmentation andrecognition. InCVPR (2), pages 326–333, 2004.

[12] X. Ren and J. Malik. Learning a classification model forsegmentation. InComputer Vision, 2003. Proceedings. NinthIEEE International Conference on, pages 10 –17 vol.1, 2003.

[13] J. Shi and J. Malik. Normalized cuts and image segmen-tation. InComputer Vision and Pattern Recognition, 1997.Proceedings., 1997 IEEE Computer Society Conference on,pages 731 –737, June 1997.

[14] O. Veksler, Y. Boykov, and P. Mehrani. Superpixels andsupervoxels in an energy optimization framework. InECCV, ECCV’10, pages 211–224, Berlin, Heidelberg, 2010.Springer-Verlag.