stable mean-shift algorithm and its application to the...

952 IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 53, NO. 2, FEBRUARY 2015

Stable Mean-Shift Algorithm and Its Applicationto the Segmentation of Arbitrarily Large

Remote Sensing ImagesJulien Michel, Member, IEEE, David Youssefi, and Manuel Grizonnet

Abstract—Segmentation of real-world remote sensing images ischallenging because of the large size of those data, particularlyfor very high resolution imagery. However, a lot of high-levelremote sensing methods rely on segmentation at some point andare therefore difficult to assess at full image scale, for real remotesensing applications. In this paper, we define a new property calledstability of segmentation algorithms and demonstrate that piece-or tile-wise computation of a stable segmentation algorithm canbe achieved with identical results with respect to processing thewhole image at once. We also derive a technique to empiricallyestimate the stability of a given segmentation algorithm and ap-ply it to four different algorithms. Among those algorithms, themean-shift algorithm is found to be quite unstable. We propose amodified version of this algorithm enforcing its stability and thusallowing for tile-wise computation with identical results. Finally,we present results of this method and discuss the various trendsand applications.

Index Terms—Image processing, image segmentation, mean-shift, object-based image analysis, remote sensing, stability ofsegmentation.

I. INTRODUCTION

FOR the past decade, the number of very high resolution(VHR) optical sensors orbiting around the Earth and imag-

ing it has been constantly increasing. USA satellite programssuch as IKONOS, QuickBird, GeoEye, or WorldView, as wellas the new French constellation of Pleiades satellites, provideimages with a ground sampling distance between 0.41 and 1 mand a field of view between 15 and 20 km. When includingground processing such as pan-sharpening or mosaics, theend users receive tremendously large images whose size canexceed several tenths of thousands of pixels in both dimen-sions, challenging both their manual and automatic processingcapabilities. Meanwhile, the fine spatial resolution of theseimages drastically increases not only the richness but alsothe complexity of the information they contain and thereforeenforces the need for more complex processing methods andalgorithms.

Manuscript received October 22, 2013; revised March 17, 2014; acceptedJune 3, 2014. Date of publication July 8, 2014; date of current versionAugust 12, 2014.

J. Michel and M. Grizonnet are with the Centre National d’Études Spatiales(French Space Agency), 31400 Toulouse, France.

D. Youssefi was with the Centre National d’Études Spatiales (French SpaceAgency), 31400 Toulouse, France. He is now with Communications et Sys-tèmes, 31506 Toulouse, France.

Color versions of one or more of the figures in this paper are available onlineat http://ieeexplore.ieee.org.

Digital Object Identifier 10.1109/TGRS.2014.2330857

Indeed, it is well accepted that for such spatial resolutions,many objects of interest are composed of a large numberpixels with high heterogeneity, and that pixel-based and evenneighborhood-based image analysis techniques fail to capturethis richness due to their limited perception of the spatialcontext of each pixel. Therefore, techniques such as object-based image analysis [1], object-based image classification,spatial reasoning [2], [3], or geospatial analysis have beenextensively studied during the last years and proven of greatinterest for the analysis of VHR optical images, for instance.Most of these methods perform some kind of segmentation,either as a preprocessing step or within the processing itself.

However, for VHR optical images delivered by sensors citedearlier, as well as for VHR SAR delivered by TerraSAR orCosmoSkymed sensors, and even for images from the upcom-ing Sentinel2 sensor, with lower resolution but larger swath,the traditional data processing paradigm hardly applies. Indeed,loading the whole data into memory, processing it, and retriev-ing the results from memory can turn impossible because theamount of memory needed to consume the input data volumeexceeds the available resources. The processing of such largedata or large images can be achieved by dividing the input datainto pieces such as tiles (a tile being a rectangular image sub-set), processing each tile sequentially [4], and gathering the fi-nal output. For some kind of image operations, such as pixel- orneighborhood-wise filtering, this methodology can be appliedwith guaranteed identical results with respect to processing thewhole image at once. However, most segmentation algorithmsdo not cope well with piece- or tile-wise computation becausethey consider long-range interactions between pixels. As aresult, the aforementioned advanced image analysis techniqueshave mostly been investigated at the scale of small extracts andcan hardly apply to real-life remote sensing applications withreal data. A few techniques have been proposed to tackle thisissue and are reviewed in Section II. These techniques do notprovide any guarantee regarding the exactitude of the result.In particular, they might spawn artifacts in the premises oftiles border because of the convergence of the segmentationalgorithm toward incompatible solution on each side of thisborder.

The work presented in this paper consists in two maincontributions. First, we define a new property of segmentationalgorithms called stability, as well as an empirical method tomeasure the adequacy of a segmentation algorithm with thisproperty. We also show that stable segmentation algorithmscan be used for tile-wise segmentation with guaranteed exact

0196-2892 © 2014 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

MICHEL et al.: MEAN-SHIFT ALGORITHM AND ITS APPLICATION TO LARGE REMOTE SENSING IMAGE SEGMENTATION 953

results, following our proposed methodology. Second, we pro-pose a stable segmentation algorithm derived from the mean-shift algorithm [5], which allows combining the accuracy of themean-shift algorithm with the benefit brought by the stabilityproperty, particularly regarding large images tile-wise process-ing. It is noteworthy that the execution of this methodology ina parallel or distributed environment is straightforward sinceeach step involves independent operations on image tiles thatare clearly isolated. Ultimately, this methodology provides anartifact-free segmentation result that can be used to assessobject-based methods at the scale of full VHR data.

The remaining of this paper is organized as follows. InSection II, we present an overview of existing work. Section IIIwill introduce the concept of segmentation stability. It will alsodescribe a methodology to measure this stability and present themeasured stability of four different segmentation algorithms.Section IV will investigate the instability sources of the mean-shift segmentation algorithm and propose a stabilized version.In Section V, we then describe how the guaranteed stabilityof a segmentation algorithm leads to a trivial solution for tile-wise segmentation, yielding identical results to those obtainedby processing the whole image at once. Finally, Section VI willpresent some segmentation results on Pleiades images.

II. PREVIOUS WORK

Despite its real interest for operational applications, the useof a tiling scheme for the segmentation of very large images hasbeen merely investigated in the literature.

In [6], the authors propose to combine the full lambdaschedule algorithm (FLSA) with such a tiling scheme. Theyapply the algorithm on each tile and consider segments entirelyincluded in an overlapping zone between tiles as not reliable.Pixels belonging to these segments are reinitialized to one-pixel segments and considered in the segmentation of the nexttile. Although applied to the FLSA, the authors advertise thatthis method can apply to any segmentation algorithm based onsegment merging. If this method is able to process correctlysmall segments within the tile overlap zone, larger segmentsnot entirely included in this zone are not considered and maytherefore exhibit artificial borders induced by the tiling scheme.

In [7], the authors propose a method for the detection ofroad networks on satellite images, based on segmentation. Thenormalized cuts algorithm of Shi and Malik is first applied toobtain an oversegmentation of the image, and segments are thenmerged into bigger segments, from which roads are detected.In their experiment, the image is divided into 200 × 200 pixeltiles, and the segmentation is applied to each tile independently.The merging step is applied afterward to all segments from alltiles, with no specific rule for segments on the tile border, whichleads to some artificial borders in the final result.

In his work on a topological and hierarchical model for thesegmentation of large images [8], Goffe separates the partsof segment borders that are artificially created by tiling orsegmentation errors from the parts of segment borders thatcorrespond to real image content. The criterion to detect theartificial borders shall denote the similitude of the neighboringsegments they separate. The author proposes a criterion based

on colorimetric analysis. However, it cannot be ensured that alltiles border are properly identified as artificial borders.

Some methods have been proposed to specifically process theartifacts created by tiled segmentation. In [9], a framework forthe segmentation of large images is described. Segmentationis performed independently on non-overlapping tiles, and theFLSA algorithm is used to merge segments having borders incommon with tiles, processing vertical tile borders first, andthen horizontal ones. Merging stops when a minimal cost isreached. In [10], Happ et al. proposes a method for parallelsegmentation of large images, with a segmentation methodalso relying on iterative segment merging. One iteration of thealgorithm is performed independently on each tile, possibly inparallel, without processing segments that touch the tile border.Once this iteration is done for all tiles, a single segmentationtask processes segments on borders. This scheme iterates untilthere is no possible merging left. Neither methods can guaran-tee exact results with respect to the same processing appliedwithout tiling.

In [11], the authors propose a parallel implementation ofa segmentation method based on minimum spanning trees bydividing the image into non-overlapping tiles. The authorsadvertise that the results are consistent whatever the numberof parallel threads is, but no details are given on how thisconsistency is ensured.

In [12], instead of using rectangular tiles, the tiles are adaptedto the image contour, with the hope that the tile border will coin-cide with actual segments and will therefore not create any arti-ficial segment shape. However, it cannot be guaranteed that thetile borders adapted with respect to the strength of the local gra-dient will match the decision of a given segmentation algorithm.

In [13], the author proposes a modification of the recur-sive hierarchical segmentation algorithm, in which pixels orsegments that have been inappropriately merged during thesegmentation process are identified, split, and re-merged with abetter candidate segment. According to the author, this modifiedalgorithm provides hierarchical segmentations of images thatare free of processing window artifacts.

In a previous work, Michel et al. proposed a generic toolfor tiled segmentation, vectorization, and stitching of segmentson the tile border based on a simple topological criterion[14]. While theoretically compatible with any segmentationalgorithms, the stitching criterion is very coarse, and manyartifacts remain in the final result.

III. SEGMENTATION ALGORITHMS STABILITY

A. Terms and Notations

In this paper, I denotes an image, and S : I → S(I) rep-resents a segmentation algorithm. S(I) forms a partition ofimage I , i.e.,

S(I) = {Ri, i ∈ [1, n]} ,

⎧⎨⎩

I =⋃n

i=1Ri

i �= j ⇒ Ri ∩Rj = ∅(1)

where R denotes an element of S(I) and will be referred as asegment throughout this paper. Whenever needed, a collectionof segments will be indexed by a subscript Ri.


I ′ ⊂ I denotes an image subset and will be called likewisein this paper. Note that when dealing with implementation,the term image tile will also be used. It denotes a rectangularand axis-aligned image subset of I . A segment from an imagesubset I ′ is represented by R′ ∈ S(I ′).

Finally, for a segment R ∈ S(I) and an image subset I ′ ⊂ I ,we define SR(I

′) as follows:

SR(I′) = {R′ ∈ S(I ′) \R′ ⊆ R} . (2)

SR(I′) is the set of segments R′ ∈ S(I ′) that are fully

included in segment R ∈ S(I).

B. Definition of Stability

With those notations, we can give a formal definition of thestability property.

Definition 1: Algorithm S is said to be stable if ∀R ∈ S(I)and ∀ I ′ ⊂ I , the following properties hold:

R ⊂ I ′ ⇒ ∃R′ ∈ S(I ′) \R′ = R (3)

R ∩ I ′ �= ∅ ⇒ R ∩ I ′ =⋃

R′∈SR(I′)

R′. (4)

We call property (3) the inner property. It states that anysegment of S(I) inner to I ′ is also a segment of S(I ′).Property (4) is called the cover property. It states that the re-striction to image subset I ′ of any segment of S(I) overlappingI ′ must decompose into a set of segments from S(I ′).

More intuitively, the stability of a segmentation algorithmimplies the following.

• Given an image, given an object of interest within thisimage, a segmentation algorithm is considered stable ifany image subset containing this object spawns the exactsame segmentation result of this object, or alternatively,

• Given an image, given two overlapping subsets of thisimage, a segmentation algorithm is considered stable if thesegmentation results in the intersection of these two imagesubsets match exactly.

C. Benefits of Stability

Stability is a much desirable property for remote sensingimage segmentation. It is obvious that using unstable segmen-tation with tiling will lead to different results depending on thetiling scheme, and Section V will show how guaranteed stabilityleads to an exact solution for tiled segmentation. However,even without considering tiling, identical objects with identicalbackgrounds located at different places in an image might besegmented very differently, apart from any consideration oflightning, sampling, noise, or small changes. For instance, usingan unstable algorithm may be sufficient to fail segmentingcoherently a whole set of identical buildings in an image(cf. Fig. 1).

While there have been several studies comparing segmen-tation algorithms and software with various criteria such asaccuracy with respect to ground truth, it is surprising that onlylittle attention has been given to this stability property. In [15], a

Fig. 1. Segmentation of two identical houses in an image: The initial imageis created by merging a 54 × 55 pixels image of a house and its horizontallyflipped version. After segmenting this image with the Edison mean-shift algo-rithm and the stabilized mean-shift algorithm, the left part of the segmentationis flipped back and subtracted to the segmentation of the right part. Unlike thestabilized mean-shift algorithm, differences appear with the Edison mean-shiftalgorithm.

comparison of segmentation software is presented, and amonga lot of other properties, the reproducibility when image size ismodified is evaluated, which is somehow similar to the notionof stability. However, among the seven software inquired, onlyeCognition is considered reproducible starting version 3.0, andthere is no detail on how this reproducibility has been evaluated,which is not the focus of this paper.

D. Measuring Stability

1) Comparing Segmentation Results: Evaluating quantita-tively the stability of a segmentation algorithm requires to beable to compare different segmentation results of the same im-age. For this purpose, there is a number of quantitative metricsavailable in the literature. A review of unsupervised metricsis available in [16], whereas supervised metrics relying on aground truth or reference segmentation, which are more suitablefor our work, have been compared in [17] and [18]. Among thedifferent proposed methods, we chose to use the Hoover et al.method [19] and its quantitative extension proposed byOrtiz et al. [20]. In addition to being based on the same settheory concepts than our definition of stability, this methodprovides both quantitative measures and segment-based clas-sification of the type of error. Moreover, it uses thresholds thatallow tuning the tolerance of the metric and error classificationwith respect to the aim of the experiment. Finally, even ifsometimes criticized, it is cited and used as a reference metricin most papers proposing new segmentation evaluation metrics.Note that the sensitivity to the distortion issue raised in [18] isnot relevant in our work since we analyze the quantitative scoresof each segment and not only the count of correctly detectedsegments.


The Hoover method matches one or several segments fromthe ground truth segmentation with one or several segmentsfrom the tested segmentation, and each match is called a Hooverinstance. Given two partitions of the same image S1(I) andS2(I), Hoover instances are based on the confusion matrix O,which is defined as

Oij =∣∣R1

i ∩R2j

∣∣ for(R1

i , R2j

)∈ S1(I)× S2(I). (5)

Let t denote an overlap threshold 0.5 < t ≤ 1. The Hoovermethod classifies segments from S1(I) and S2(I) into fourcategories. A pair of (R1

i , R2j ) ∈ S1(I)× S2(I) is considered

as a match if {Oij ≥ t×

∣∣R1i

∣∣Oij ≥ t×

∣∣R2j

∣∣ . (6)

In this case, the RC metric from Ortiz measures the matchperformance as

RC(R1

i , R2j , t

)= min

{Oij

|R1i |,Oij∣∣R2

j

∣∣}. (7)

A segment R1i ∈ S1(I) is considered as fragmented by a set

R2� of m ≥ 2 segments R2

� = {R2jk

∈ S2(I), k ∈ [1,m]} if{∀ k ∈ [1,m], Oijk ≥ t×

∣∣R2jk

∣∣∑mk=1 Oijk ≥ t×

∣∣R1i

∣∣. (8)

In this case, the RF metric from Ortiz measures the amountof fragmentation as

RF(R1

i , R2�, t

)= 1−

∑mk=1 (Oijk (Oijk − 1))

|R1i | × (|R1

i | − 1). (9)

In a similar way, a segment R2j ∈ S2(I) is considered as

fragmented by a set R1� of m′ ≥ 2 segments R1

� = {Rik ∈S1(I), k ∈ [1,m′]} if{

∀ k ∈ [1,m′], Oikj ≥ t×∣∣R1

ik

∣∣∑m′

k=1 Oikj ≥ t×∣∣R2

j

∣∣. (10)

In this case, the RA metric from Ortiz measures the amountof grouping as

RA(R1

�, R2j , t

)= 1−

∑m′

k=1 Oikj (Oikj − 1)∣∣∪m′k=1R

1ik

∣∣× (∣∣∪m′k=1R

1ik

∣∣− 1) . (11)

Segments from S1(I) or S2(I) that do not belong to any ofthe three previous categories are considered as not detected. Inthis case, Ortiz defines the missed measure RM(t) = 1.

Global metrics can then be defined for the whole image asthe mean of metrics values weighted by segment sizes.

In the remaining of this paper, we adopt the following colorconventions to display Hoover instances images. Given a scorethreshold ts

• if RC ≥ ts, the segment is green (correct detection);• if RC < ts and RF > 0, the segment is yellow

(oversegmentation);

Fig. 2. Initial image and subset.

• if RC < ts and RA > 0, the segment is purple(undersegmentation);

• if 0 < RC < ts, RF = 0, RA = 0 or if RM > 0, thesegment is red (missed detection);

• background is black (if applicable).

In all experiments, the threshold t to build Hoover instancesis set to 0.75, whereas the threshold ts to display the instancesis adapted to each experiment.

2) Protocol for Stability Evaluation: In order to verify thestability of the segmentation algorithms, the influence of thetiling is simulated by reducing or changing the image subsetobserved by these algorithms. The general protocol is thefollowing one.

First, an image is segmented, and the resulting segmentationis considered as the reference segmentation.

Second, a subset of the image is selected to create thecompared segmentation (cf. Fig. 2). Finally, the two segmen-tations are restricted by set intersection to the image subset andcompared with the Hoover method (cf. Section III-D1).

3) Results: The protocol for stability evaluation has beenapplied to the mean-shift segmentation algorithm, the water-shed algorithm applied to the image gradient magnitude, andthe connected-component algorithm [21] using a threshold onthe spectral distance between neighboring pixels as the con-nection criterion. The watershed algorithm implementation isprovided by the ITK software [22], which performs a top-down gradient descent toward local minima, as opposed tothe bottom-up watershed strategy starting from seeds at localminima of the height function. Two implementations of themean-shift algorithm have been considered: the original imple-mentation from Comaniciu et al. [5] distributed as the Edisonsoftware, and the multithreaded implementation from the OrfeoToolBox [23].

Please note that in this experiment, the intrinsic performancesare not compared: the aim is only the stability evaluation.Hence, the parameters used for these experiments were arbi-trarily chosen and kept identical for both the entire image andthe subset segmentations. Although we cannot guarantee thatresults will not change with a different parameter setup, we didnot observe such changes during our experiments.

Fig. 3 shows the comparison between the reference seg-mentation created with a 300 × 300 pixels extract from aPleiades scene over Paris and the compared segmentationscreated with the same extract where the 1-pixel-wide externalcrown of pixels is removed. The threshold for the display of the


Fig. 3. Comparison between the reference segmentation and the segmentationof a Pleiades scene extract over Paris, France, in the case where the 1-pixel-wideborder is deleted. Because of the cover property, algorithms with good stabilitywill exhibit both green (RC) and yellow (RF) segments. (a) Mean shift (Edison,case R ⊂ I′). (b) Mean shift (Edison, case R ∩ I′ �= ∅). (c) Mean shift (OTB,case R ⊂ I′). (d) Mean shift (OTB, case R ∩ I′ �= ∅). (e) Watershed (caseR ⊂ I′). (f) Watershed (case R ∩ I′ �= ∅). (g) Connected components (caseR ⊂ I′). (h) Connected components (case R ∩ I′ �= ∅). (i) Initial image.

Hoover instance has been set to ts = 1 to ensure that the greensegment will only correspond to exact matches. The figure alsodistinguishes between the R ⊂ I ′ case, where only segments

Fig. 4. Ortiz scores according to image subset shrinking (case R ⊂ I′).

Fig. 5. Ortiz scores according to image subset shrinking (case R∩I′ �=∅).

that do not touch the image borders are considered, and theR ∩ I ′ �= ∅ case, where all segments are considered.

We can see that only the connected-component algorithmseems to comply with the stability criteria: in the R ⊂ I ′ case,all segments are correctly detected (in green), whereas in theR ∩ I ′ �= ∅ case, all segments are either correctly detected oroversegmented (in yellow). In contrast, in the case of the mean-shift algorithms and the watershed algorithm, some segmentsare undersegmented (in purple) or even not detected (in red),even in the R ⊂ I ′ case.

Figs. 4 and 5 show the Ortiz scores, computed between 0(if no segment belongs to this kind of instance) and 1, withrespect to the size of the segmented image subset. In Fig. 4,segments touching the edges are excluded, whereas in Fig. 5,all segments are considered.

We can see that the instability of the mean-shift algorithmsand the watershed algorithm is confirmed: both mean-shiftimplementations show an erratic behavior, and if the watershedalgorithm curves are smoother, they exhibit instability nonethe-less. Regardless of the edges, the processed subset shrinkinginduces variations of the correctly detected metric (RC).

Moreover, the stability of the connected-component algo-rithm is confirmed: when segments touching the edges areexcluded, all segments are correctly detected (RC = 1), whichshows that the algorithm complies with the inner property (3).When edge segments are included, the correct detection


decreases in favor of the oversegmentation, which shows thatthe cover property (4) is respected. Thus, this algorithm re-spects the stability definition.

IV. STABILIZING THE MEAN-SHIFT ALGORITHM

A. Why the Mean-Shift Algorithm?

A wide range of segmentation algorithms has been appliedto remote sensing images in the literature. A complete reviewof those algorithms is beyond the scope of this paper. Someelements can be found in [24].

The mean-shift algorithm has received a lot of attentionfrom the remote sensing community [25]–[27]. Variants suchas variable-bandwidth mean shift [28], medoid shift, and quickshift [29] have also been investigated in this context. It hasbeen found to perform well with various remote sensing dataranging from medium to very high resolution and in variousapplications. Its multivariate nature, as the simplicity of thefiltering step, and the availability of various implementations,among which one is from the authors themselves [30], are someof the keys to this popularity. We have already been usingthis algorithm for various works [31]–[33], and the need toresolve its instabilities and to allow for scalability to real remotesensing data has been driven by our applications.

B. Overview of the Algorithm

The mean-shift algorithm is not really a segmentation algo-rithm by itself. It is a nonparametric method first introducedby Fukunaga and Hostetler [34] in 1975 for the estimation ofmodes in a multivariate density of probability function. In 2002,Commaniciu [5] proposed a spatial extension of the algorithmby applying the mode estimation to the joint spatial and spectraldomain.

Here, is the outline of the algorithm. Let xi and zi, i = 1,. . . , n, denote pixels from the input and output joint-domainimage. For each pixel, the following steps are computed:

1) initialize j = 1 and yi,1 = xi;2) while j < jmax and ‖yi,j − yi,j+1‖ > t do: yi,j+1 =

(∑

xk∈N(yi,j)K(xk−yi,j)xk)/(

∑xk∈N(yi,j)

K(xk−yi,j));3) set zi = (xs

i,j ,yri,j)

where yi,j is the current mode estimation for pixel xi atstep j, K(x) is a kernel function, N(x) is the set of pixelswithin the spatial range hs and spectral range hr of x, su-perscripts s and r denote the spatial and spectral componentsof the joint-domain image pixels, jmax denotes the maximumnumber of iterations, and t denotes the convergence thresh-old. Although the Edison implementation allows using bothGaussian and uniform kernels, the latter is usually chosen inmost applications.

At the end of the process, each pixel is assigned the estimatedspectral signature and spatial location of the local mode of theprobability density function it belongs to. This can be useful forimage denoising, for instance, but is not yet a segmentation ofthe image. In the remaining of this paper, we will refer to thisstep as the filtering step and refer to it as Fhr,hs,jmax

(I).

In Commaniciu’s paper, segmentation is obtained by group-ing together neighboring pixels that converged toward the samespectral and spatial mode. However, because the spatial kernelbandwidth is limited by hs, and because of the stopping crite-rion, each pixel converges toward its own estimate of the mode,which may slightly differ even for neighboring pixels belongingto the same object. Therefore, according to this paper, clustersare formed by grouping together all pixels that converged tospatial modes that are closer than hs and spectral modes thatare closer than hr, which is known as biconnected componentsin graph theory. In this paper, this step will be referred as thegrouping step.

This initial clustering may lead to an important number ofvery small segments that do not correspond to any meaningfulobject of the scene. In Comaniciu’s paper, segments smallerthan a given number of pixels are simply removed from theresulting segmentation, but in the Edison implementation [30],these small segments are merged with the neighboring segmentwith the closest spectral mode. This merging process is iterateduntil there are no small segments left or until a maximumnumber of iterations have been reached. In this paper, these twooperations will be referred to as the small segments filtering step.

C. Sources of Instability

In order to propose a stable version of the mean-shift algo-rithm, we first need to determine the sources of the instability,which can be classified into two categories: algorithmic issuesand implementation issues.

1) Algorithmic Issues: We found three main causes prevent-ing the algorithm from being stable: the first occurs during thefiltering step, second comes from the grouping step, and the lastoccurs during small segments processing.

As stated in Section IV-B, during the filtering step, thefiltered pixel iteratively moves toward the spatial position ofits mode. When filtering an extract of a given image, it maytherefore happen that pixels from an image subset converges toa spatial position outside of this subset. In this case, at somepoint, the algorithm will be lacking pixels within the spatialradius because they do not belong to the extract. As a result,the mode associated with the current pixel will be different thanthe one that would have been affected with the algorithm seeingthe whole image.

The grouping step is also unstable by nature: as stated bygraph theory, a pixel might belong to several biconnected com-ponents of pixels satisfying the close mode predicate, whereasin image segmentation, a pixel is only associated with a singlesegment. For this reason, finding clusters of pixels verifyingthe close mode predicate is an ill-posed problem for whichseveral solutions exist. Fig. 6(a) gives a simple illustration ofthis issue. The grouping step algorithm used during mean-shiftsegmentation therefore performs a greedy search that dependson initial conditions and on the order in which pixels areprocessed. As shown in Fig. 6(b), it is very easy to exhibita toy example of a group of pixels clustered together into abiconnected component that will be split into several segmentsif adding more pixel neighbors, which proves that this part ofthe algorithm does not satisfy the cover property (4).


Fig. 6. Illustration of the instabilities yielded by the close mode predicateduring the clustering step. (a) Simple example of graph with two possibleclustering. Green nodes can group either in component 1 with red nodes orin component 2 with blue nodes. (b) Simple example of clustering split byadditional nodes. Green nodes are grouped together in component 1, but whenadding red and blue nodes, component 1 is split and left green nodes are mergedin component 2 while right green nodes go to component 3.

Finally, one more stability issue comes with the processingof small segments, for two reasons. First, small segments lyingon the border of the segmentation of an extract of the imagecannot be considered as small segments, because they mightactually belong to larger segments that have been split bythe image subset. Any merging decision for these segments istherefore potentially wrong, but the worst comes when iteratingthe merging process: those errors on border segments willpropagate toward the image center and may cause large errors tohappen very far from the image subset border, where we wouldassume the result to be free from side effects.

2) Implementation Issues: Some implementations of the al-gorithm come with some optimizations that have a great impacton stability. A common optimization is to stop iterating thealgorithm for a given pixel if its spatial trajectory reachesthe vicinity of another pixel whose mode has already beenestimated, in which case the current pixel is assigned the samemode. This allows running the algorithm faster and also per-forms some kind of grouping that can be reused for the groupingstep, but it is very unstable by nature: the result depends on theorder in which pixels are processed, and the modes associatedwith pixels for which the optimization operates might slightlydiffer from the modes that would have been estimated for thesame pixels without the optimization.

The second instability caused by the implementation is nu-merical instabilities. It can be seen from Section IV-B that thealgorithm implies iterative computation of weighted means.Stability implies that any pixel from any image subset willconverge to the exact same mode than with the full imageduring the filtering step. It is therefore very important forthe computation of these weighted means to be numericallyinvariant to changes of a given pixel spatial position, whichwill be different in all extracts. It is also very important to avoidaccumulating numerical errors during the weighted sums and toavoid multiplying a weighted sum with numerical imprecisionby the spatial radius, for instance, since this may lead toamplifying a very small error to a significant one.

D. Proposed Stable Version

Now that we carefully reviewed all sources of instability ofthe mean-shift algorithm, we propose here to resolve all of themby slightly modifying the algorithm.

To resolve instabilities related to the algorithm itself, whichare presented in Section IV-C1, we apply the following im-provements. For a given pixel, the spatial position of the es-timated mode will not be found farther from this pixel thanthe product of the number of iterations by the spatial radius.We therefore define a margin of m = jmax ∗ hs + 1 from anyimage subset border, outside of which we can guarantee thatthe filtering step will give the exact same results than withthe entire image. Equivalently, to guarantee the stability of thefiltering step on a given subset, it is sufficient to add a marginof m pixels to this extract. Please note that the pixels from themargin do not need to be processed by the algorithm to gettheir estimated modes: they only need to be reachable by thealgorithm. Second, we relax the biconnected component searchof the grouping step into a simple connected-component search,where only neighboring pixels have to satisfy the predicate onspatial and spectral mode distance. Because this relaxed con-straint will lead to larger segments by nature, we propose to setthe thresholds on spatial and spectral mode distances to valueslower than the filtering step bandwidths. Finally, regarding theissue of the small segments filtering step, we simply decide notto do the merging or pruning of small segments, leaving it tofurther processing according to the user needs.

Regarding the implementation issues, we deactivated anyoptimization proposed by the implementation to get the closestimplementation possible to the mathematical expression of thealgorithm in Section IV-B.

With these modifications, we are able to define a segmen-tation algorithm with guaranteed stability that combines thestabilized filtering mean-shift algorithm and the proven stableconnected-component algorithm:

1) Filter image I by the modified mean-shift filtering stepwith parameter hr, hs, jmax, t with margin m = jmax ∗hs + 1 (except for image borders), to get filtered imageFhr,hs,jmax

(I)2) Segment filtered image Fhr,hs,jmax

(I) into segmentationSh′

s,h′r(Fhr,hs,jmax

(I)) with the connected-componentalgorithm using a predicate ensuring both spatial modescloser than threshold h′

s ≤ hs and spectral modes closerthan threshold h′

r ≤ hr.

Fig. 1 shows the comparison between the original mean-shiftimplementation and the stabilized version. We can observe thatnone of the segmentation errors from the original algorithm arefound with the stabilized version.

V. STABILITY AND SEGMENTATION OF LARGE IMAGES

In this section, we first prove the theoretical basis that allowsus to run a stable segmentation algorithm piecewise whileensuring identical results with respect to the same run on thefull data at once. We then describe and illustrate all steps of thesegmentation methodology.


A. Theory

The stability property given in definition (1) leads to thefollowing theorems.

Theorem 1: Let T (I) = {I ′i ⊂ I, i ∈ [1, N ]} denote a set ofsubset of image I such that

⋃i∈[1,N ] I

′i = I . Let S be a stable

segmentation algorithm, for which property (1) holds, i.e.,

∀R ∈ S(I), R =⋃

i∈[1,N ]

⎛⎜⎝ ⋃

R′∈SR(I′i)

R′

⎞⎟⎠ . (12)

Proof: Provided that⋃

i∈[1,N ] I′i = I , it is trivial that

R =⋃

i∈[1,N ]

R ∩ I ′i.

In addition, from the cover property (4), we know that∀ I ′ ∈T (I)

R ∩ I ′ =⋃

R′∈SR(I′)

R′.

Thus

R =⋃

i∈[1,N ]

⎛⎜⎝ ⋃

R′∈SR(I′i)

R′

⎞⎟⎠. �

Theorem 1 states that a segment R exactly decomposes intoa set of segments from each segmentation of image subsets I ′ ∈T (I), which can be a tiling scheme, for instance. Note that T (I)does not need to form a partition of I , only

⋃i∈[1,N ] I

′i = I is

needed. As a result, the different subsets I ′i may overlap, whichwill be useful in the following.

Theorem 2: Let I ′ ⊂ I denote a subset of image I and S bea stable segmentation algorithm, for which the cover property(4) holds. ∀R ∈ S(I), ∀R′ ∈ S(I ′)

R′ ∩R = ∅ ⇔ R′ � R. (13)

Proof: First, let R ∈ S(I) and R′ ∈ S(I ′) such that R′ ∩R = ∅. From the cover property (4), we know that

R ∩ I ′ =⋃

R′′∈SR(I′)

R′′.

However, since R′ ⊆ I ′

R ∩R′ = R ∩ I ′ ∩R′ = ∅.

Thus ⋃R′′∈SR(I′)

R′′ ∩R′ = ∅

which means that R′ �∈ SR(I′), and therefore, R′ � R.

Second, let R ∈ S(I) and R′ ∈ S(I ′) such that R′ � R.According to (2), if R′ � R, then R′ �∈ SR(I

′). From the coverproperty (4), we know that

R ∩ I ′ =⋃

R′′∈SR(I′)

R′′.

Since R′ ⊆ I ′

R ∩ I ′ ∩R′ = R ∩R′ =⋃

R′∈SR(I′)

R′′ ∩R′.

We can also note that SR(I′) ⊆ S(I ′). According to (1), S(I ′)

is a partition of I ′, and

R′ �∈ SR(I′) ⇒ R′′ ∩R′ = ∅ ∀R′′ ∈ SR(I

′).

Thus

R ∩R′ = ∅.

�Theorem 3: Let I ′1 ⊂ I and I ′2 ⊂ I such that I ′1 ∩ I ′2 �= ∅.

Let S be a stable segmentation algorithm, for which property(1) holds. Let R ∈ S(I), R′

1 ∈ S(I ′1), and R′2 ∈ S(I ′2), i.e.,

R′1 ∩R′

2 �= ∅ ⇒ ∃R ∈ S(I) \ (R′1 ⊆ R and R′

2 ⊆ R) . (14)

Proof: Let R′ = R′1 ∩R′

2. It is trivial that

∃R ∈ S(I) \R ∩R′ �= ∅.

Therefore

R ∩R′1 �= ∅ and R ∩R′

2 �= ∅.

Thus, from Theorem 2, we have

R′1 ⊆ R andR′

2 ⊆ R. �

From Theorem 3, we learn how to set together segments fromthe segmentations S(I ′1) and S(I ′2) of overlapping subsets I ′1and I ′2: if those segments overlap, they belong to the same finalsegment R in segmentation S(I).

Those three theorems will be used in the following section tobuild an algorithm for the piecewise segmentation of an arbi-trarily large image on top of the stabilized mean-shift algorithmproposed in Section IV-D.

It is very interesting to note that both proofs rely on thecover property (4) and not on the inner property (3). This meansthat an algorithm that only respects the cover property can becomputed piecewise at large scale with our method. No artifactswill be found on boundaries, but in this case, there will be noguarantee that the result will exactly match the segmentation ofthe whole image at once.

B. Algorithm for Large-Scale Segmentation

We propose the following methodology based on the theoret-ical proofs given in Section V-A. For a given image I , spatialradius hs, range radius hr, maximum number of iteration jmax,and tile size (tx, ty) (note that if the proposed methodology isvalid for any shape of image subset, for the sake of implemen-tation, rectangular tiles are used):

1) extract and filter each tile;2) segment each filtered tile with connected components;3) relabel segmented tiles;4) process small segments.Fig. 7 illustrates each of the steps above. In particular, we can

see how the relabelization step merges exactly segments acrossmultiple tiles.


Fig. 7. Illustration of the main steps of the proposed large-scale segmentationmethod on a pan-sharpened Pleiades extract over Saint Denis, France. (c), (e),and (g) show the labeled images, whereas (d), (f), and (h) show a colorizedversion of the labeled images maximizing color differences between adjacentsegments, for better visual interpretation. (a) Input image. (b) Filtered image.(c) Tile segmentation. (d) Tile segmentation (colorized). (e) Tile relabeling.(f) Tile relabeling (colorized). (g) Small segments processing. (h) Small seg-ments processing (colorized).

1) Filtering Step: First, the stabilized mean-shift filteringalgorithm is applied to each tile of size (tx +m, ty +m), withmargin m = jmax ∗ hs + 1, which leads to overlapping tiles,as shown in Fig. 8. As mentioned in Section IV-D, pixelsbelonging to the margin do not need to be filtered, but they must

Fig. 8. Overlapping tiling used for the filtering step.

Fig. 9. Overlapping tiling used for the segmentation step.

be reachable by the filtering algorithm. The processing of eachtile is completely independent and is therefore compatible withparallel or distributed execution. Once all tiles with margin havebeen processed, they can be stitched together to form exactlythe result of the filtering of the whole image.

2) Segmentation Step: Once the filtering step is achieved,we extract tiles of size (tx + 1, ty + 1) from the filtered image,as shown in Fig. 9. Note that one can also use a tile sizecompletely different from the filtering step here, as long asthe 1-pixel overlap is respected. This overlap will be usedduring the relabeling step. Each tile is independently pro-cessed by the connected-component algorithm, as explained inSection IV-D, which is again compatible with parallel or dis-tributed execution.

Since we proved in Section IV-D that this combination ofthe mean-shift filtering and the connected-component algorithmmakes a stable segmentation algorithm, the inner propertydefined in (3) already ensures us that any segment from thesegmentation of the whole image I that is fully contained ina given tile will be exactly recovered by the segmentation ofthis tile.

3) Relabeling Step: After the segmentation step, providedthat we ensure unique labels by shifting their values from onetile to another, we can already recover a full segmentation ofthe input image I . However, in this segmentation, we still havesegments split across multiple tiles, and even split into severalpieces within each tile, in the case of concave segments.

However, since our segmentation algorithm is stable, weknow that Theorem 1 applies. Therefore, we know that ifwe identify which segments of the segmentations of each tilebelong to the same segment of the final segmentation, thenwe can reconstruct this segment of the final segmentation bya simple union of the segments from each tile segmentations.


Fig. 10. Overlap of 1 pixel used to build the equivalence table.

The solution to identify which segments from the segmenta-tions of each tile belong to the same final segment comes fromTheorem 3. It states that for a pair of overlapping tiles, any pairof overlapping segments belongs to the same final segment. Weonly need a small overlap to determine which pairs of segmentsshould be merged; this is why we chose the minimal overlap of1 pixel presented in Figs. 9 and 10.

To recover the final segmentation, we explore all overlapsbetween all neighboring tiles to fill a label equivalence tableand then relabel and merge all tiles at once.

4) Small Segments Pruning: In Section IV-D, we explicitlyremoved the small segments filtering step from our algorithmas a major source of instability. However, in most cases, thosesmall segments are merely segmentation noise and can decreasethe performances of downstream processing methods that as-sume a certain level of segmentation quality. In our large-scalesegmentation method, we included a last step to filter thosesegments, with two possibilities: removing them or mergingthem. This step cannot be performed earlier in the processbecause we cannot ensure that small segments on the borderof a tile are not actually belonging to a larger segment.

The removing of small segments is straightforward: one lastpass on the final segmentation image to evaluate the numberof pixels in each segment and build a table to relabel all smallsegments as background. The merging strategy is a bit morecomplex. In the original mean-shift implementation, segmentswith size below a user-defined threshold are merged to the clos-est neighboring segment in terms of radiometry. This process isiterated until no small segments are left or a maximum numberof iterations have been reached. We choose a slightly differentstrategy: we merge segments by increasing sizes, starting withsegments of size 1, until segments of the minimal acceptablesize. This strategy is more conservative because it ensures thatthe noisier (i.e., smaller) segments will be merged first. Fig. 11shows the results of these different strategies.

VI. RESULTS AND PERFORMANCE ASSESSMENT

A. Experimental Verification of Stability

This first experiment aims at demonstrating the stability ofour method with respect to the tiling scheme. To do so, a1000 × 1000 pixels extract of a pan-sharpened Pleiades imageis segmented with our method and with increasing numberof tiles. The segmentation results are compared using Ortizmetrics to a reference segmentation obtained by processing theimage with a single tile. We use a range bandwidth of 50, aspatial bandwidth of 10, a maximum number of iterations of 10,and a convergence threshold of 0.1. The connected-componentpredicate uses a threshold of half the spatial and spectralbandwidths, and segments smaller than 50 pixels are processedby merging. With number of tiles ranging from 2 × 2 (i.e.,

Fig. 11. Examples of the different strategies to process small segments: noprocessing in (b), pruning in (c), and merging in (d). Only the colorized versionof the labeled images maximizing color differences between adjacent segmentsis shown, for better visual interpretation. (a) Input image. (b) Segmented image.(c) Removing small segments. (d) Merging small segments.

Fig. 12. Extract of segmentation results of full Rennes image (9091 ×12 707 pixels). The image has been divided into 20× 20 tiles of 455× 604 pixelsfor processing. In the extract, tile borders are represented by the dashed bluelines (8 of them are fully visible in the figure), while segments are outlined in red.

500 × 500 pixel tiles) to 10 × 10 (i.e., 100 × 100 pixel tiles),the same exact result is obtained: RC = 1 and other metricsare null. This supports our theoretical demonstration that ourmethod performs an exact reconstruction of the segmentationresult from the segmentation of individual tiles.

B. Large-Scale Performances

In this section, we apply our segmentation method to aPleiades pan-sharpened scene of Rennes city in France. Thisextract contains 9000 × 12 000 pixels with four spectral bands.For the filtering step, we used a spatial radius of 5, a rangeradius of 50, a maximum number of iterations of 4, and aconvergence threshold of 0.1. During the segmentation step, weused a distance threshold of 30 for the connected componentsand a minimum size of 0 pixels. A shadow mask is also usedto avoid considering shadowed pixels during the segmentationstep. Finally, we performed the merging of small segments up


TABLE IEVOLUTION OF PROCESSING TIME AND FILE SIZES WITH RESPECT TO THE INPUT IMAGE SIZE IN PIXELS

Fig. 13. Distribution of processing time among the different steps of the method.

to a segment size of 200 pixels. In each step, we used a tilingscheme of 20 × 20 tiles. While it is not possible to displaysuch large results in this paper, Fig. 12 shows a detailed viewof an extract of those results. This experiment confirms that noartifacts related to tiling can be observed in the segmentationresult, as stated by theory.

All experiments were conducted on an Intel Xeon(R) quadri-core CPU at 2.67 GHz with 8 Gb of RAM. For the sake ofcompactness of the information and also compatibility with GISsoftware, we convert our raster segmentation to a vector layerof polygons. Table I shows some figures denoting the perfor-mances of the method for different input image sizes. We cansee that the full scene is processed in 5 h and 15 min and con-tains around 170 000 segments. While the input image file sizewas 1.7 Go, the vector containing one polygon per segment,as well as statistics on image radiometry and additional shapedescriptors, only weighs 500 Mo. Fig. 13 shows the distributionof the processing time among the different steps. We can seethat the filtering step and the processing of small segmentsaccount for the majority of it. Note that as our work focuseson overcoming memory limitations by performing controlledtile-wise segmentation, we did nothing particular regardingprocessing time. Much speed up can be excepted from anoptimized implementation or port to the graphics processingunit (GPU) of the filtering or processing of small segmentssteps, for instance, as long as the stability constraint is met.Moreover, Steps 1–3 of our large-scale segmentation methodcan be applied independently to each tile, and this method istherefore compatible with cluster or grid parallelism.

C. Comparison With the Original Mean-Shift Algorithm

As described in Section IV, our method uses a modifiedversion of the mean-shift algorithm. In this section, we there-fore try to assess how close our segmentation results are to theoriginal algorithm. As a reminder, the two main changes thatmay affect the results are as follows: 1) the relaxed connectedcomponents with range and spatial thresholds on mode, withrespect to the original grouping step strategy; and 2) the pro-cessing of small segments steps.

The first main change adds two new parameters to thealgorithm: the range and spatial thresholds used during theconnected-component step. To evaluate the influence of those

Fig. 14. Evolution of Ortiz scores for both the small segments processingstrategy with respect to the percentage of range and spatial radius used as theconnected-component threshold. (a) Removal strategy. (b) Merging strategy.

parameters on the segmentation quality with respect to the orig-inal mean shift, the following experiment has been conducted.A pan-sharpened Pleiades image has been segmented with boththe original and stable algorithms. For both algorithms, weused a spatial radius of 10, a range radius of 50, a maximumnumber of iteration of 10, a convergence threshold of 0.1, and aminimum segment size of 50. We compute the Ortiz scores withrespect to the original mean-shift result while varying the addi-tional range threshold and spatial threshold parameters of thestable mean shift from 10% of range and spatial radii to 100%of range and spatial radii. We also evaluate both small segmentsprocessing strategies. Fig. 14 shows the variation of the Ortizscores with respect to the percentage of radii used as thresholds.We can see that for both small segments processing strategies,there is an optimal around 50%, where we reach an RC score of0.7 for the removal strategy and 0.6 for the merging strategy.We can also observe that these threshold parameters allowsetting the tradeoff between oversegmentation and underseg-mentation with respect to the original algorithm results. Othercombination with other properties can be obtained by usingdifferent ratios for spatial and range thresholds. For instance,when trying to segment elongated homogeneous objects, settinghigher spatial thresholds may lead to a better delineation.


Fig. 15. Hoover scores between results from the original mean shift, connected components, and stable mean shift for both merging strategies, for range andspatial threshold set to 50% of range and spatial radius (in the case of connected components, only range threshold is used and set to range radius). Note that thethreshold to display Hoover instances has been decreased to ts = 0.8 (typical value for segmentation evaluation). Only the colorized version of the labeled imagesmaximizing color differences between adjacent segments is shown, for better visual interpretation. (a) Original mean shift, removal. (b) Hoover scores (a)–(c).(c) Stable mean shift, removal. (d) Hoover scores (c)–(e). (e) Connected components, removal. (f) Input image. (h) Original mean shift, merging. (i) Hooverscores (h)–(j). (j) Stable mean shift, merging. (k) Hoover scores (j)–(l). (l) Connected components, merging.

Fig. 15 shows the Ortiz scores between original and stablemean-shift results when using a threshold of 50% of radii forboth strategies, which seems optimal according to Fig. 14. Itshows that both segmentation results are very similar, even ifsome oversegmentation occurs mainly because of the relaxedconstraint induced by the connected-component algorithm. Inthe same figure, we also compare the results from our methodto those from the connected-component algorithm alone, with adistance threshold of 50% of the range radius. We can observethat the results of our method are far more close to the originalmean shift than to the connected components. Using a distancethreshold of 100% yields even more different results and poorersegmentation. This highlights the strength of our method, whichcombines the stability brought by connected components andthe accuracy of the original mean-shift algorithm.

VII. CONCLUSION

In this paper, we have formally defined a stability propertyfor segmentation algorithms and proposed a method for mea-suring the actual stability of different segmentation algorithms.We experimentally showed that the mean-shift algorithm thatis widely used in remote sensing applications is not stableaccording to our definition and proposed a modified versionof the algorithm with guaranteed stability. We then provedthat the stability property allows deriving a simple solution forpiecewise processing of the segmentation results and appliedthis principle to the proposed stable mean shift, leading to asegmentation methodology scalable to real-world data. We then

experimentally demonstrated the capabilities of this new seg-mentation methodology by showing that results are independentof the tiling scheme, and we showed an example of applyingthe method to large imagery. While this method works aroundthe scientific locks of segmenting data at large scale, it doesnot, of course, resolve the well-known issues of robustness withrespect to scene variability and parameter tuning. However,allowing for real and larger data processing might allow for abetter knowledge of these issues in the future. Additional workmay also include applying this methodology to other segmen-tation algorithms, and better sketching the intrinsic propertiesthose algorithms must exhibit to be compatible large-scale pro-cessing. Further work to reduce processing time by optimizingor porting to GPU some steps of our methodology might also beneeded to meet expectations of time-constrained applications.Please note that the complete source code and ready-to-use bi-naries are available in the Orfeo ToolBox open-source software3.20 or later. The Orfeo ToolBox cookbook also provides a step-by-step guide to large-scale segmentation using these tools [35].

ACKNOWLEDGMENT

The authors thank A. Jaen and S. Harasse from CS-SI fortheir help with Section II, J. Nabucet from Costel for providingthe pan-sharpened image of Rennes used in Section VI-B, andJ. Inglada and A. Giros from the Centre National d’ÉtudesSpatiales (French Space Agency) for carefully reviewing thispaper before submission. All Pleiades images are copyright ofCNES (2012 and 2013), distribution Airbus DS/SpotImage.


REFERENCES

[1] T. Blaschke, “Object based image analysis for remote sensing,” ISPRS J.Photogramm. Remote Sens., vol. 65, no. 1, pp. 2–16, Jan. 2010.

[2] J. Inglada and J. Michel, “Qualitative spatial reasoning for high-resolutionremote sensing image analysis,” IEEE Trans. Geosci. Remote Sens.,vol. 47, no. 2, pp. 599–612, Feb. 2009.

[3] M. Vanegas, I. Bloch, and J. Inglada, “Alignment and parallelism forthe description of high-resolution remote sensing images,” IEEE Trans.Geosci. Remote Sens., vol. 51, no. 6, pp. 3542–3557, Jun. 2013.

[4] E. Christophe, J. Michel, and J. Inglada, “Remote sensing processing:From multicore to gpu,” IEEE J. Sel. Topics Appl. Earth Observ. RemoteSens., vol. 4, no. 3, pp. 643–652, Sep. 2011.

[5] D. Commaniciu, “Mean shift: A robust approach toward feature spaceanalysis,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 24, no. 5, pp. 603–619, May 2002.

[6] D. J. Crisp, P. Perry, and N. J. Redding, “Fast segmentation oflarge images,” in Proc. 26th Austral. Comput. Sci. Conf., 2003, vol. 16,pp. 87–93.

[7] A. Grote and C. Heipke, “Road extraction for the update of road databasesin suburban areas,” in Proc. Int. Arch. Photogramm., Remote Sens. SpatialInf. Sci., 2008, pp. 563–568.

[8] R. Goffe, “Pyramides irrégulières descendantes pour la segmentation degrandes images histologiques,” Ph.D. dissertation, Univ. Poitiers, Poitiers,France, 2011.

[9] Z. Yang, “Tiling and merging framework for segmenting large images,”U.S. Patent 8 086 037, Dec. 27, 2011.

[10] P. Happ, R. Ferreira, C. Bentes, G. Costa, and R. Feitosa, “Multiresolutionsegmentation: A parallel approach for high resolution image segmentationin multicore architectures,” in Proc. Int. Arch. Photogramm., Remote Sens.Spatial Inf. Sci., 2010, vol. 38, p. 4.

[11] J. Wassenberg, W. Middelmann, and P. Sanders, “An efficient parallelalgorithm for graph-based image segmentation,” in Proc. Comput. Anal.Images Patterns, 2009, pp. 1003–1010.

[12] T. S. Korting, E. F. Castejon, and L. M. G. Fonseca, “Divide andsegment—An alternative for parallel segmentation,” in Proc. GeoInfo,2011, pp. 97–104.

[13] J. C. Tilton, “Split-remerge method for eliminating processing windowartifacts in recursive hierarchical segmentation,” U.S. Patent 7 697 759,Apr. 13, 2010.

[14] J. Michel et al., “Open tools and methods for large scale segmentation ofvery high resolution satellite images,” in Proc. OGRS, 2012, pp. 179–184.

[15] G. Meinel and M. Neubert, “A comparison of segmentation programs forhigh resolution remote sensing data,” in Proc. Int. Arch. Photogramm.Remote Sens., 2004, vol. 35, pp. 1097–1105.

[16] H. Zhang, J. E. Fritts, and S. A. Goldman, “Image segmentation evalu-ation: A survey of unsupervised methods,” Comput. Vis. Image Under-standing, vol. 110, no. 2, pp. 260–280, May 2008.

[17] J. S. Cardoso and L. Corte-Real, “Toward a generic evaluation of imagesegmentation,” IEEE Trans. Image Process., vol. 14, no. 11, pp. 1773–1782, Nov. 2005.

[18] X. Jiang, C. Marti, C. Irniger, and H. Bunke, “Distance measures forimage segmentation evaluation,” EURASIP J. Appl. Signal Process.,vol. 2006, pp. 209–209, Jan. 2006.

[19] A. Hoover et al., “An experimental comparison of range image segmenta-tion algorithms,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 18, no. 7,pp. 673–689, Jul. 1996.

[20] A. Ortiz and G. Oliver, “On the use of the overlapping area matrix for im-age segmentation evaluation: A survey and new performance measures,”Pattern Recogn. Lett., vol. 27, no. 16, pp. 1916–1926, Dec. 2006.

[21] L. Di Stefano and A. Bulgarelli, “A simple and efficient connected compo-nents labeling algorithm,” in Proc. Int. Conf. Image Anal. Process., 1999,pp. 322–327.

[22] H. J. Johnson, M. McCormick, and L. Ibanez, “The ITK software guide,”in Updated for ITK 4.5, 3rd ed. Chapel Hill, NJ, USA: Insight SoftwareConsortium, 2013.

[23] E. Christophe and J. Inglada, “Open source remote sensing: Increasingthe usability of cutting-edge algorithms,” IEEE Geosci. Remote Sens.Newslett., pp. 9–15, Mar. 2009.

[24] T. Blaschke, S. Lang, E. Lorup, J. Strobl, and P. Zeil, “Object-orientedimage processing in an integrated gis/remote sensing environment andperspectives for environmental applications,” in Environmental Informa-tion Planning, Politics and the Public, 3rd ed, vol. 2. Berlin, Germany:Metropolis Verlag, 2000, pp. 555–570.

[25] N. Jiang, J. Zhang, H. Li, and X. Lin, “Semi-automatic building extrac-tion from high resolution imagery based on segmentation,” in Proc. Int.Workshop EORSA, 2008, pp. 1–5.

[26] X. Huang and L. Zhang, “An adaptive mean-shift analysis approach forobject extraction and classification from urban hyperspectral imagery,”IEEE Trans. Geosci. Remote Sens., vol. 46, no. 12, pp. 4173–4185,Dec. 2008.

[27] D. Ming et al., “Semivariogram-based spatial bandwidth selection forremote sensing image segmentation with mean-shift algorithm,” IEEEGeosci. Remote Sens. Lett., vol. 9, no. 5, pp. 813–817, Sep. 2012.

[28] D. Comaniciu, V. Ramesh, and P. Meer, “The variable bandwidth meanshift and data-driven scale selection,” in Proc. IEEE 8th ICCV., 2001,vol. 1, pp. 438–445.

[29] A. Vedaldi and S. Soatto, “Quick shift and kernel methods for modeseeking,” in Proc. Comput. Vis.-ECCV , 2008, pp. 705–718.

[30] Edison: Code for the Edge Detection and Image Segmentation. [Online].Available: http://www.caip.rutgers.edu/riul/research/code/EDISON

[31] E. Christophe and J. Inglada, “Object counting in high resolution remotesensing images with otb,” in Proc. IEEE IGARSS, 2009, vol. 4, p. IV-737.

[32] J. Michel, J. Malik, and J. Inglada, “Lazy yet efficient land-covermap generation for hr optical images,” in Proc. IEEE IGARSS, 2010,pp. 1863–1866.

[33] J. Michel, M. Grizonnet, and O. Canevet, “Supervised re-segmentationfor very high-resolution satellite images,” in Proc. IEEE IGARSS, 2012,pp. 68–71.

[34] K. Fukunaga and L. Hostetler, “The estimation of the gradient of a den-sity function, with applications in pattern recognition,” IEEE Trans. Inf.Theory, vol. IT-21, no. 1, pp. 32–40, Jan. 1975.

[35] The Orfeo ToolBox Cookbook, a Guide for Non-Developers Updated forOTB-3.20, OTB Development Team CNES, Paris, France, 2013.

Julien Michel (M’14) received the Telecommuni-cations Engineer degree from the École NationaleSupérieure des Télécommunications de Bretagne,Brest, France, in 2006.

From 2006 to 2010, he was with Communicationset Systèmes, Toulouse, France, where he workedon studies and developments in the field of remotesensing image processing. He is currently with theCentre National d’Études Spatiales (French SpaceAgency), Toulouse, where he is in charge of researchand development in image processing algorithms for

remote sensing images.

David Youssefi was born in Toulouse, France, in1990. He received the Electrical Engineering degreefrom the École Nationale Supérieure d’Electronique,Informatique, Télécommunications, Mathématiqueet Mécanique, Bordeaux, France.

He is currently a Study Engineer at Communi-cations et Systèmes, Toulouse, where he works onremote sensing image processing.

Manuel Grizonnet received the MathematicalModeling, Vision, Graphics, and Simulation Engi-neer degree from the École Nationale Supérieured’Informatique et Mathématiques Appliquées,Grenoble, France, in 2007.

From 2007 to 2009, he was with BRGM (FrenchGeological Survey), Niamey, Niger, as a Systems andGeological Information System (GIS) Engineer inthe frame of the SYSMIN project, which aim to givethe Niger ways to promote its mining potential byestablishing a GIS. He is currently with the Centre

National d’Études Spatiales (French Space Agency), Toulouse, France, wherehe is developing image processing algorithms and software for the exploitationof Earth observation images.

stable mean-shift algorithm and its application to the...

Documents