accelerated patch-based planar clustering of noisy range images in indoor environments for robot...

Robotics and Autonomous Systems 60 (2012) 584–598

Contents lists available at SciVerse ScienceDirect

Robotics and Autonomous Systems

journal homepage: www.elsevier.com/locate/robot

Accelerated patch-based planar clustering of noisy range images in indoorenvironments for robot mappingRavi Kaushik a, Jizhong Xiao b,a,∗

a Department of Computer Science, The Graduate Center, City University of New York, United Statesb Department of Electrical Engineering, The City College of New York, United States

a r t i c l e i n f o

Article history:Received 23 October 2010Received in revised form8 November 2011Accepted 5 December 2011Available online 16 December 2011

Keywords:ClusteringSegmentationPolygonRange imageCompressionPlane-fittingFeature extraction

a b s t r a c t

This paper introduces a methodology to cluster noisy range images into planar regions acquired in indoorenvironments. The noisy range images are segmented based on a Gaussian similarity metric, whichcompares the geometric attributes that satisfy the coplanarity conditions. The algorithm is designed tocluster coplanar noisy range data by means of patch-based sampling from range images. We discuss theadvantages of patch-based clustering over point-based clustering of noisy range images that eliminatescomputational redundancy to accelerate the clustering process while keeping the segmentation error toa minimum. The final output of the algorithm is a set of polygons, where each polygon is defined by aset of boundary points that replaces large number of coplanar data points in a given planar region. The3D range image is acquired by a rotating 2D range scanner and stored in a 2D array. Each element in thearray is explicitly stored as the range distance; the indices of the array implicitly retain neighborhood andangular information. The array is grouped into mutually-exclusive patches of size (k× k) and the Hessianplane parameters are computed for each patch. We propose a graph-search algorithm that compares theplane parameters of neighboring patches by searching breadth-wise and clusters the coplanar patchesinto respective planes. We compare the proposed Patch-based Plane Clustering (PPC) algorithm with thepoint-based Region Growing (RG) algorithm and the RANSAC plane segmentation method to analyze theperformance of each of the algorithms in terms of speed and accuracy. Experimental results indicate thatthe PPC algorithm shows a significant improvement in computational speed when compared with thestate-of-the-art segmentation algorithms while maintaining a high accuracy in segmenting noisy rangeimages.

© 2011 Elsevier B.V. All rights reserved.

1. Introduction

1.1. Motivation

3D range scanners acquire range images, which essentially out-puts range distance measurements between the sensor and thesurrounding environment. This paper deals with real-time pro-cessing of range images acquired in indoor environments, whichconsists of mostly planar surfaces. A range image represented inthe spherical coordinate system can be converted to a 3D pointcloud representation in the Cartesian coordinate system and vice-versa. This conversion is even applicable to multiple overlapping3D point clouds, which are an outcome of 3D registration [1].For robotic applications, it is desirable to have a more compact

∗ Correspondence to: Steinman Hall, Room ST-548, Grove School of Engineering,The City College of New York, 140th Street and Convent Avenue, New York,NY 10031, United States. Tel.: +1 212 650 6197.

E-mail addresses: [email protected] (R. Kaushik), [email protected](J. Xiao).

0921-8890/$ – see front matter© 2011 Elsevier B.V. All rights reserved.doi:10.1016/j.robot.2011.12.001

representation of range images, which can be achieved by extract-ing polygon features defined by a set of boundary points leading tocompression of range maps. Polygon extraction from noisy rangeimages is a challenging task and the polygonal model of the envi-ronment is useful in robotic applications such as 3D mapping androbot navigation in indoor environments. The current commercial3D range scanners used in robotic applications are capable of ac-quiring large number of data points (N > 104) in a short periodof time (<30 s). This data acquisition is achieved by spinning the2D range scanner on a rotating platform. The angular resolution ofthe range image is limited by the resolution of the motors spin-ning the range scanner. Most conventional range scanners used inrobotics are inexpensive but noisy. Their range measurement ac-curacy is dependent on a number of factors including range sen-sor limitations, embedded processors in the scanning equipment,step accuracy of internal and external motors driving the rangesensor and reflection/refraction of the laser beam on different sur-faces. The high-resolution range scanners are accurate but they arenormally bulky and expensive. Our research makes use of less ex-pensive light-weight range scanners, namely the Hokuyo R⃝ URG-LX and the UTM-30LX 2D range scanners that can be mounted on

http://dx.doi.org/10.1016/j.robot.2011.12.001

http://www.elsevier.com/locate/robot

http://www.elsevier.com/locate/robot

mailto:[email protected]

mailto:[email protected]

http://dx.doi.org/10.1016/j.robot.2011.12.001

R. Kaushik, J. Xiao / Robotics and Autonomous Systems 60 (2012) 584–598 585

wall climbing robots [2] andminiature Quadraptor helicopters [3],which have constraints in payload and on-board computationalpower.

The range data acquired by 3D range scanners consistsof large amounts of digital information that needs memorystorage space and should be further processed in real-time fora number of robotic applications such as 3D scan registrationwith polygons [4,5], digitization of indoor environments [6],3D map construction [7] and robotic navigation. Feature-basedrepresentation such as polygon representation of range imagesis advantageous over point-based representations in a number ofways. The polygonal maps occupy less storage space comparedto 3D point clouds/range images. Efficient memory storage isachieved by replacing large numbers of data points in the rangeimage with the polygons (defined by boundary vertices) thatrepresent planar regions, which results in high data compression.In addition, polygons extracted from range image can be renderedin computer graphics tools faster than 3D point clouds with the aidof Graphical Processing Units (GPUs). These two aspects becomeprominent while rendering large number of fused range maps.The polygons are extracted as features as a preliminary step tobuild 3D range maps. Most recently, a number of researchers haveworked on registration of multiple range images with polygonsas features [4,8,9]. These approaches have been experimentallyshown to bemuch faster than iterative point-based 3D registrationtechniques [1,10].

1.2. Related work

Planar segmentation is a widely studied topic in computergraphics, photogrammetry and robotics catering to numerousapplications. Many algorithms have been developed in the field ofcomputer graphics to obtain polygonal models of 3D point cloudsfor graphical rendering/visualization [11,12]. These algorithms aredesigned to model the objects with complex surfaces resultingin large number of polygons. The polygonal mesh can befurther decimated or simplified by approximation based on therequirements to satisfy a certain level of detail for visualization asin [13,14]. We can identify two major steps in these approaches:mesh formation/triangulation and mesh approximation. The bestknown time complexity of mesh formation using Delaunaytriangulation isO(n·log n) for an unstructured 3Dpoint cloud usingdivide-and-conquer strategy [15]. Further, [14] present an iterativeprocess for mesh optimization and is based on minimization of acost function. [13] employs constructing an error quadric metricthat involves computing the plane parameters for each trianglein the mesh. These algorithms focus more on visualization withpolygonalmodels and do not address real-time processing of rangeimages.

The requirements for 3D mapping in robotics community aredifferent from a computer graphics perspective. The latter focuseson building complex and accurate models of architectural objectsat the expense of speed of the algorithms. The range scannersused in the field of computer graphics produce dense and highlyaccurate range images. The former focuses on building simpler 3Dmodels using less accurate range scanners while they minimizethe computation time of the feature extraction algorithm to satisfythe real-time applications such as Simultaneous Localization andMapping (SLAM). A number of methodologies developed forrobotic applications are available to extract simplified polygonmodels from 3D range images [16–21]. The complexity of theplanar model is simple yet efficient to be useful in ground robotnavigation [5,22] and aerial robot navigation [23].

We compare our algorithm with two popular plane segmenta-tion algorithms, one is point-wise Region Growing (RG) algorithmand the other is RANSAC-based plane segmentation algorithm.

An example of a RG algorithm to extract polygons from range datais presented in [17]. The planar segments are extracted from therange image in two steps. First, nearest neighbors are searched toidentify coplanar data points belonging to individual planes. Sec-ond, the boundary points of each cluster are extracted to formpoly-gons. The RG algorithm has a time complexity of O(n2), where n isthe number of coordinate points in the range image. [20] is an ex-tension of the work carried out in [17], where the authors of [20]modified the RG algorithm and reformulated the underlyingmath-ematics to present an incremental version of planar segmentation.This improved the efficiency of the RG algorithm, which is de-scribed as follows. Initially, a random point (seed) and its nearestpoint is chosen as part of the set (Π )/region. More nearest pointsare added to the set (Π ) in increasing distance from the set. Thisregion is expanded by adding a nearest point p′ if the point lieswithin a certain distance and satisfies the optimal planarity con-ditions defined in [20]. In addition, the authors explain the effi-cient implementation of the RG algorithm, which uses a priorityqueue and grid-based range images that implicitly store nearestneighbor information. Another optimization is presented for opti-mal plane calculation, which performs incremental update of at-tributes while storing previous calculations. This efficient versionof the RG algorithm has a time complexity of O(n · log n) [20]. An-other example of plane segmentation algorithm that uses point-wise RGmethodology is presented in [24]. The algorithm segmentsthe range images into planar, smooth non-planar and non-smoothsurfaces. Thenormal vectors are computed for eachdata point froma set of data points belonging to the local neighborhood. The RG al-gorithms in [17,20,24] are empirically fine-tuned with a set of pa-rameters to segment planar regions within a tolerance limit. TheRGmethodology is not robust to noisy range images, which makesit difficult to fine-tune the parameters to segment range imageswith higher noise levels. Plane segmentation of noisy range im-ageswith RG algorithmmay lead to incorrect segmentation, whereone plane cluster fills partly into another plane cluster. The for-mer plane cluster will blend into its neighboring plane leading toover-segmentation (additional points are added to the formerplane segment that in real belongs to the latter plane segment)and the latter plane will undergo under-segmentation. The majordrawback of the RG algorithm is that it is computationally slow.The normal vector is repeatedly computed for the set (Π ∪ p′)while each data point (p′) is tested for coplanarity with the op-timal plane, which is an evaluation of data points to be includedin a cluster. The normal vector computation for each data point isredundant and performed repeatedly on the covariance matrix ofthe data points. A well-known approach of extracting the normalvectors for a given set of data points is by solving the Eigen prob-lem [25] for the covariancematrix of data points belong to the clus-ter, with a lower bound of Ω(n3), where (n = 3) is the size of thematrix. This repeated normal vector calculation is the reason whythe RG algorithm is computationally slow in nature.

In addition, we compare our algorithm with another popularform of planar segmentation, which belongs to the class of modelfitting using RANdom SAmple Consensus (RANSAC) [26]. Somerecent efforts on planar segmentation from range data usingRANSAC is presented in [27–29]. In general, a small number of datapoints are randomly chosen to establish the ‘‘hypothesis’’ set. Theparameters of the planar model are computed from this minimalset of random points. Further, the ‘‘verification’’ step involvestesting the remaining data points to see if it fits the model definedby the parameters computed earlier. If the error function evaluatesto a value below a set threshold, then the data points are addedto the ‘‘consensus set’’. The RANSAC plane segmentation algorithmstops looking for a better planar model once the probabilityof finding a better ranked consensus set drops below a certainthreshold. The number of trials can be set heuristically as an

586 R. Kaushik, J. Xiao / Robotics and Autonomous Systems 60 (2012) 584–598

algorithm input or evaluated based on the probability. If thereis more than one plane in the range image, the above steps arerepeated after removing previously segmented data points untilthe remaining points are reduced to a small percent of initial value.The time complexity of a general RANSAC plane segmentationalgorithm is given by O(TiterChyp(k) + NCver(k)), where Titer is thenumber of iterations determined by the probability of findingthe best ranked consensus set, Chyp(k) is the cost of hypothesisor computing the parameters of a planar model with minimumnumber of data points (k), Cver(k) is the cost to verify if the datapoint belongs to a particular consensus set and N is the numberof data points in the 3D data set. The RANSAC is slow for severalreasons. The randomized approach in choosing the data pointsfor establishing the hypothesis sometimes leads to picking anoutlier, which is imminent while establishing the hypothesis set.In addition, the number of iterations is related to the noise levelsin the range image. ‘‘While the RANSAC algorithm is effective in thepresence of noise and outliers, it has two significant disadvantages,namely, its efficiency and the fact that the plane detected by RANSACmay not necessarily belong to the same object surface; that is, spurioussurfaces may appear, especially in the case of parallel–gradualplanar surfaces such as stairs’’ [30]. Alternatively, [30] presents amodification of the RANSAC plane segmentation algorithm knownas the Seq-NV-RANSAC. This algorithm checks the Normal Vector(NV) of each data point (computed from a local neighborhoodat any given point) and that of the hypothesized RANSAC plane.This process is shown experimentally to improve the accuracyof RANSAC planar segmentation. However, it will also lead toadditional computation time in determining the normal vectorsfrom the 3D point cloud and performs slower than the traditionalRANSAC plane segmentation algorithm.

There are a few other plane segmentation approaches thatemerged recently. A probabilistic approach presented in [19] ex-tract planes from range images based on Expectation Maximiza-tion (EM). The disadvantage of this algorithm is that it needs to beinitialized with the number of planes for a given range image andtheir ‘‘main’’ directions, prior to the Least Square (LS) minimiza-tion. This number is usually unknown and a heuristic approach isused to determine the number of planes during the clustering pro-cess. This approach follows patch-based range image segmentationand effectively clusters points based on co-planarity conditions us-ing aprobabilistic approach. There are fewother algorithms [31,32]that employ clustering techniques to range image plane segmenta-tion. However, these type of algorithms have not been proved to bepractical when dealing with large 3D point clouds in the presenceof noise and outliers and are computationally expensive [30].

1.3. Contribution

Our plane segmentation algorithm aims to extract plane clus-ters from large noisy range images (N > 104) in real-time (up to10 frames per second) that forms a pre-cursor step to the 3D poly-gon registration [7] and robot mapping. The plane segmentationalgorithms [16,20,33–36] improve the performance over their pre-decessors in terms of speed and accuracy. However, they fail toeliminate the redundancy in evaluating the planarity of data pointsand do not run in real-time when processing large amount (N >104) of 3D range data. We present a novel algorithm that performsPatch-based Plane Clustering (PPC) of range images in real-timeand compare its performance in terms of speed and accuracy withtwo state-of-the-art algorithms that include the RG algorithm [20]and RANSAC plane segmentation algorithm implemented in [37].Our approach to plane extraction from range images is mainly fo-cused on obtaining a simple polygonal model while improving theextraction time [4]. The PPC algorithm is also robust to noisy rangeimage data and outliers. The algorithm clusters range image data

points into respective planar clusters and extracts polygon bound-ary points from each cluster. As a preliminary step, the noisy rangeimages are processed to eliminate any outliers. Further, the rangeimage in its native form is grouped intomutually exclusive patches(grids) of a fixed size. The size is empirically set depending on thenoise levels in the range image. Then, the plane parameters (nor-mal vector, orthogonal distance between the plane and the origin,mean of the data points in the patch) are computed for each patch.The patches are initially classified as ‘‘planar’’ and ‘‘non-planar’’patches, which is dependent on whether the data points in thepatch all lie on the same plane or not. The clustering algorithmprocess the patch information in two sub-stages. In the first stage,the algorithm searches planar clusters patch-wise based on certainplane segmentation criteria. The search for coplanar patches willend when all planar patches are assigned to a cluster. In the sec-ond stage, the remaining unassigned data points from non-planarpatches are clustered into respective planar bins based on a simi-lar segmentation criteria used in the first stage. Finally, the bound-ary data points are extracted for each cluster that represents thevertices of a polygon. We discuss the benefits of patch-based clus-tering over point-based approaches. We show experimentally thatthe algorithm does not need to recompute the plane parametersafter adding new data points to the cluster. The plane parameterscomputed for each patch in the preliminary stage is adequate tocomplete the clustering process using a Gaussian similarity met-ric. The contribution mainly lies in improving the speed of the al-gorithm without much loss of accuracy in the clustering process.We use inexpensive and less accurate range scanners designed forrobotic applications and limit our research to range image cluster-ing acquired in indoor environments, comprised mostly of planarsurfaces similar to approaches in [19,20].

2. Range image acquisition and pre-processing data

We obtain the 3D range image by rotating a commercial 2Dlaser scanner (Hokuyo R⃝ URG-LX) shown in Fig. 2.1(a) on a platformusing an external motor. The 2D laser scanner acquires a 2D scanalong the sensing plane (270°) as shown in Fig. 2.1(b). The 2Drange scanner is rotated by 180° (pan angle) on the platform toobtain a complete 3D scan. A pioneer robot and a wall climbingrobot (Fig. 2.2) were used in the experiments to acquire 3D rangeimages. The step of the external motor is given by 1θ ∼= 0.81°,considered as the ‘‘azimuth’’ angle. The 2D range scanner has aninternal mirror that is rotated in steps by 1ϕ ∼= 0.36° to acquirea flat 2D scan, considered as the ‘‘elevation’’ angle. The rangedistance acquired by the laser scanner at

θi, ϕj

is stored in a 2D

array indexed by (i, j), where θi = (1θ ∗ i) , i ∈ 1, 2, . . . , 220and ϕj = (1ϕ ∗ j) , j ∈ 1, 2, . . . , 680. Each element in thearray points to a (r, θi, ϕj), which together form the sphericalcoordinates. The indices of the array provide neighborhoodinformation for each element. A time-consuming nearest neighborsearch of elements can be avoided by comparing only indexedneighboring elements of the array. A 3D range image can beeasily transformed into a 3D point cloud represented in Cartesiancoordinate system and vice-versa [37] in O(n) time. We dealtwith multiple overlapping 3D point clouds in [7], resulting ina 3D polygonal map. Multiple overlapping 3D point clouds arehighly redundant since large percentage of data points overlapseach other. The multiple overlapping 3D point clouds can alsobe transformed into a range image by quantizing the 3D datapoints in spherical coordinate system within a limited radiusfrom the center of the coordinate system. A 3D range image is acompact representation of surrounding environment in sphericalcoordinate system and stores fixed amount of data points. We canensure that every data point in the 3D point cloud is accounted forby quantizing the spherical coordinate system with small azimuth


a b

Fig. 2.1. (a) 3D range scanner used in the experiment—Hokuyo R⃝ URG-LX 2D laser scanner mounted on a rotating platform to acquire a full 3D scan. (b) The range scanneracquires 2D range data along the sensing plane (270°). A complete 3D range image is obtained by rotating the 2D scanner by 180°.

a b

Fig. 2.2. Pioneer R⃝ robot (a) and a wall climbing robot (b) equipped with a light weight rotating 2D range scanner for acquiring 3D range images.

and elevation step angles. Another advantage of range imagerepresentation is that we can access the surrounding neighbors ofa given data point in constant time of O(1) and eliminates the needfor nearest-neighbor search algorithms, which have a lower boundtime complexity ofΩ(n· log n) to access the neighboring elements.

2.1. Pre-processing range images

The range image data obtained by spinning the commercial2D range scanners as in our experiments are fairly noisy. Therange measurements acquired by the 3D range scanner can bedescribed by an approximative physical model as presented in [38]and is described as a mixture of several kinds of distributions.The model is considered as a Gaussian distribution definedby N (0, σ ) centered around the ‘‘true’’ range between the 2Drange scanner and the object being measured. In addition, therange measurement model in dynamic environment can bedescribed by an exponential distribution, defined by an intrinsicparameter (λshort) of the measurement model. We do not considerexponential distribution as our environment is assumed to bestatic and we can ignore the effects cause by moving objects whilescanning the surrounding region. The range scanner occasionallyrecords ‘‘phantom’’ readings or outliers. This range measurementis modeled as uniform distribution with a range of [0, Zmax]. Thiskind of outliers needs to be eliminated as they can significantlyaffect the final outcome of segmentation. We present a simpleyet effective method to remove the outliers with a Gaussian filter

function (Eq. (2.1)), that describes the distancemeasure between apoint and its neighbor.

f (X, Xi) = e−(X−Xi)

2

2σ2 . (2.1)

The Gaussian filter function is unit-less and yields a valuebetween 0 and 1, where 1 stands for closest proximity and 0stands for the farthest distance of the point from its neighbor. TheGaussian filter function is the distance measure between a pointand its ith neighbor. If the Gaussian filter function yields an outputbetween the given element and all its neighbors and is less thana certain threshold (0 < λ < 1), the given element is consideredas an outlier and discarded from the range image leaving behinda hole in the 2D array. This threshold can be set heuristicallyand is dependent on the noise levels of the range scanner. In ourexperiments, we arrived at a preset threshold (σ = 0.01 m, λ =

0.7) to eliminate all outliers while we retained the rest of the rangemeasurements. This filtering process does not modify the rangedata, but successfully eliminates all ‘‘phantom’’measurements. Theholes left behind by removing the ‘‘phantom’’ readings in the 2Darray are filled by applying a median interpolator that computesthe values based on its horizontal and vertical neighbors.

After noise removal and interpolation, the range image can begrouped into mutually exclusive patches in two ways; one way isto break it down into rectangular grids of a fixed size (k × k), k =

2, . . . ,N , anotherway is circular extension froma given data point,where k = (2 ∗ r + 1). r is the radius in pixels from the data pointand extends into its neighboring pixels. If the patch radius is r = 1,


Fig. 2.3. Index of neighboring cells of given data point (i, j), which forms the patchof size (3 × 3). Each cell in a 2D range image contains the range data (r).

then the patch includes 8 surrounding neighbors along with theelement at (i, j) as shown in Fig. 2.3. If the patch radius is r = 2,then the patch includes a total of 24 neighboring data points andso on. Although both methods lead to the same outcome, but weused the latter method in experiments.

2.2. Orthogonal Distance Plane Regression

After the range image is grouped into patches of fixed size, eachpatch is fitwith a planemodel and resulting outcome is theHessianplane parameters: the normal vector

n(3×1)

and the distance (d)

from the patch to the origin. The normal vector is computed fromthe covariance matrix of all the data points (Xi, i = 1, . . . ,N) ina given patch. This process is known as Orthogonal Distance PlaneRegression (ODPR). Hessian normal form is a convenient way torepresent plane equation using normal vectors. The general planeequation is given byax + by + cz + d = 0. (2.2)

The normal vector n = (nx, ny , nz) and the scalar constantwhich defines the distance from the plane to the origin (p) can becomputed using the co-efficients of the plane equation as

nx =a

√a2 + b2 + c2

(2.3)

ny =b

√a2 + b2 + c2

(2.4)

nz =c

√a2 + b2 + c2

(2.5)

p =d

√a2 + b2 + c2

. (2.6)

The Hessian normal form is then given by the dot product asfollows:n · (Xi) = −p. (2.7)

In addition, the algorithm computes the mean (X) of all thedata points in the patch, which is an additional attribute thatdescribes the patch position relative to the coordinate frame ofthe range scanner. The range data points are affected by noise.We perform plane regression by minimizing the least squareorthogonal distance (β) between the points Xi, i = 1, . . . , n andthe plane model, which is defined by the normal vector (n) andthe mean point of the patch (X =

1n

ni=1 Xi). The error function

E(X, n) to be minimized is the sum of the squared point-planedistance for (n) data points that satisfy the hyper-plane equation

EX, n

=

ni=1

β2=

ni=1

n ·Xi − X

2. (2.8)

Prior to obtaining the normal vector (n) from the set of datapoints, the (3 × 3) symmetric covariance matrix of all data pointsis computed, which is given by,

MXi, X

=

ni=o

Xi − X

Xi − X

T. (2.9)

The eigenvalues and eigenvectors of the matrix MXi, X

are

obtained as described in [39]. The normal vector of the best-fitted plane to the points Xi for a given patch is the eigenvectorcorresponding to the minimum eigenvalue of the covariancematrixM

Xi, X

. We can assume that thematrixM is a set of three

linear equations, then the three eigenvalues form the characteristicsolution to the set of equations. The three eigenvectors correspondto each of the eigenvalues and are orthogonal to each other. Twoof the three eigenvalues correspond to eigenvectors that lie on theplane and is greater than the third eigenvalue. The third smallesteigenvalue corresponds to the normal vector which is orthogonalto the plane determined by the other two eigenvectors. The otherHessian parameter is the scalar constant, which is the distancefrom the planar patch to the origin of the coordinate system givenby,

p= −n · X . (2.10)

2.3. Distinction of patches and patch size

We distinguish between a planar patch and non-planar patchbased on the criterion (2.11) when grouping the data points intopatches. The data points in the planar patch (patches marked inred and blue, Fig. 2.4) lie on the same plane defined by the Hessianparameters (n, p). To accommodate the range measurement error,we define an orthogonal distance measure, which intuitivelyrepresents the spread of the data points of a given patch alongthe normal vector of the plane. If the distance measure is less thanthe preset distance threshold (ST ), then the patch is considered asplanar. This criterion is given by the dot product of the diagonal ofcovariance matrix, diag(M(Xi, X)) computed for Xi data points ofthe patch and the normal vector n of the patch.

diagMXi, X

· n < ST . (2.11)

Non-planar patches do not satisfy the above criterion. Theyusually lie on the boundary of two planar segments or containdata points that do not form a plane (patches marked in green,Fig. 2.4(b)).

The Patch-based Plane Clustering (PPC) algorithm computesplane parameters of each patch as a preliminary step. A singleplanar patch is selected as the seed for planemodel. The rest of theplanar patches are added to the cluster depending onwhether theysatisfy the plane segmentation criteria described in Section 3. Theplane model is not recomputed when new data points are addedto the cluster as in the region-growing algorithm. In this section,we study the effects of patch size on the plane segmentation.A simulation study was conducted to evaluate the accuracy ofnormal vectors computed for patches of a simulated range image.The study indicates that the plane parameters computed for eachpatch is accurate enough to complete the clustering process ifappropriate patch size is selected. Fig. 2.5 shows five planarsegmented regions (4wall surfaces and the floor plane representedby different colors) in a simulated environment as the groundtruth. The noise level in the range image is set at 1% of the rangedistance measured as is the case for the range scanner used inour simulations. The Hokuyo R⃝ Range scanner URG-LX has an errorof ±0.01 m for a range measurement between 0.2 and 1 m and±1% for the range measurement between 1 and 4 m [URG-04LX


a b

Fig. 2.4. Computation of normal vectors after grouping the range image data points into patches. (a) Shows accurate grouping of planar patches (marked in red and blue)that fall into two separate clusters. (b) Shows a possible inaccurate grouping of patches leading to boundary patches not assigned to either of the clusters. Boundary patchesform the non-planar patches (marked in green). (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

Fig. 2.5. Planar segments of the simulated environment consisting of a floor surfaceand four walls. Different planar segments are displayed in several colors that matchthe colors of the normal vectors plotted in Fig. 2.6.

Specifications, 2005]. In this simulated experiment, we computedthe normal vectors of planar patches for each plane segment andrepresented its distribution with different colors corresponding tothe color of the plane. From the results, we can assess the accuracyof the normal vectors computed for planar patches by varying thepatch size. Three plots of normal vectors extracted from the samenoisy range image for different patch sizes (2×2), (3×3) and (5×

5) are shown in Fig. 2.6(a)–(c) respectively. Different colormarkersare used in Figs. 2.5 and 2.6 to distinguish between different planesegments and their respective normal vectors.We can observe thatdata points marked in red are floor points in Fig. 2.5. It can also beobserved that normal vectors marked in red in Fig. 2.6 computedfor the planar patches belonging to the floor plane are inaccurateand spread randomly over the normal vector space for the patchsize set at (2×2). As the patch size is increased, the normal vectorsappear to be more densely populated and close to the real planemodel. The improvement in accuracy can be noted with the otherplane segments marked in different colors.

In Fig. 2.7, we can observe that the error rate of normal vectorsdrops to negligible amount with increase in patch size. Error ratewas computed by taking the dot product of the normal vector of thepatches and the actual planemodel. Increasing the patch size leadsto less number of patches. Further, the time required to computethe normal vectors is reduced. Although the accuracy of the normal

vectors can be improved by choosing a large patch size, there is acaveat in choosing a very large patch size. The patch size is onlylimited by the nature of surfaces in the indoor environment, wherethe range images are acquired by the sensor. If the environmentconsists of large planar surfaces, itwould beuseful to choose a largepatch size. However, if the indoor environment is highly clutteredand consists of non-planar surfaces, then a small patch size ispreferred which will ensure that the algorithm finds adequateplanar patches.

3. Patch-based Plane Clustering algorithm

In this section, we propose a clustering algorithm that searchesfor coplanar data points of the range image in two stages. In thefirst stage, the algorithm starts with a seed patch and performsa breadth-wise search among all the planar patches for coplanarpatches. Depending on whether the planar patches satisfy theplane segmentation criteria, the data points of the coplanar patchare added to the cluster. The algorithm continues until all planarpatches are assigned to a cluster. In the second stage, data points ofthe non-planar patches are individually added to the clusters afterevaluating the segmentation criteria.

In the first stage, the algorithm starts by assigning the firstplanar patch with index [1, 1] as the root of the graph, labeled ascluster-1 (Fig. 3.1). The neighboring planar patches are assignedone-by-one as children to the root (parent) of the graph if and onlyif they meet the following plane segmentation criteria:

1. Gaussian similarity measure between the normal vectors of theparent patch and its neighboring planar patch is greater thanthe preset threshold (λ)1

fnA, nB,

= exp

−

nA − nB

22σ 2

> λ.

2. The orthogonal distance between the parent patch and itsneighboring patch must be less than a preset threshold (δ)measured in meters. The orthogonal distance between patchA(nA, pA) and patch B(nB, pB) is given by, dist (A, B) =nA · XB

+ pA.

1 The parameters σ and λ are unit-less as the Gaussian similarity metriccompares normalized vectors. The range of both these parameters is [0, 1]. σ is thesame for all three axes of the normal vector.


a

b

c

Fig. 2.6. Plot of normal vectors extracted from noisy range image with varyingpatch sizes (a) 2 × 2 (b) 3 × 3 (c) 5 × 5. The markers with five different colors inthe normal vector space indicate normal vectors of patches belonging to differentplane segments.

If the above two criteria are met, the neighboring patchesare pushed onto a First-In-First-Out (FIFO) queue. After all theneighboring elements of the patch with index [1, 1] are evaluated,

Fig. 2.7. Plot of error in computing normal vectors with varying patch sizesmarked in red. In addition, the computation time and number of normal vectorsextracted for a single planar region can be seen in blue and green respectively. (Forinterpretation of the references to colour in this figure legend, the reader is referredto the web version of this article.)

the first element in the FIFO queue is popped and its neighboringpatches are evaluated by coplanarity test. If the neighboringpatches are coplanar to its parent patch, then they are assignedwith the same cluster label and added to the FIFO queue. If theyare not coplanar, the neighboring patches are unassigned. Thewhole operation is repeated until all the elements in the FIFOqueue are popped out, which means that all the planar patchesbelonging to the same cluster are found. In addition, the graphsearch algorithm maintains continuity of the plane by comparingonly neighboring patches. A search for the new cluster beginswhen the FIFO queue is empty. The FIFO queue executes insidea double-for loop that searches for a new cluster row-wise. Thealgorithm searches for a new seed patch and skips all the planarpatches that were already assigned with a cluster label. The wholeprocess is repeated until all the planar patches are assigned withcluster labels and the algorithm moves to the next stage. In thesecond stage, the unassigned data points from non-planar patchesare evaluated to place them in appropriate clusters. The processof adding individual data points to different clusters uses thesame graph search method as described earlier. The normal vectorof each data point is extracted from the local patch of radius(r) centered at that data point. This ensures that the normalvectors are accurate when dealing with noisy range images. Twoclusters are merged in the process if the algorithm finds that theneighboring data points belonging to the two different clustersmeet the coplanarity conditions. Data points that do not meet thesegmentation criteria are put to a cluster andmarked as non-planarcluster. The flow chart of the algorithm is shown in Fig. 3.2.

3.1. Discussion on computational complexity

Most plane segmentation algorithms spend bulk of its process-ing time in computing the plane parameters or fitting the datapoints to a specific plane model. Our proposed PPC algorithm ex-ploits the redundancy of data points that lie on the same plane.The planar parameters are computed patch-wise and a distinctionis made been the planar patches and non-planar patches. If we as-sume that the range image containing (n) data points was grouped


Fig. 3.1. Graph search algorithm indicates the tree structure and the assignment of the patches to a particular cluster after evaluation of the plane segmentation criteria.Orange nodes belong to cluster ‘‘1’’ indicated inside the circle and element in the blue nodes are still not assigned to any cluster, denoted by ‘‘U ’’. (For interpretation of thereferences to colour in this figure legend, the reader is referred to the web version of this article.)

perfectly into respective planar patches of size, N = (k× k). In ad-dition, the clustering algorithm uses a FIFO queue that maintainsa list of neighboring patches belonging to the specific cluster. Thecomputation time is reduced by a factor of patch size (N), as thepatches process data points in batches, leading to the time com-plexity of O(( n

N ) · log( nN )). Once planar patch points are added in

batches, the unassigned data points from the non-planar patchesthat lie on the border of two or more patches are considered oneat a time. The number of unassigned data points is dependent onthe nature of surfaces in the indoor environment. In few occasions,grouping of the range image into patches may result in a non-planar patch which includes data points from two or more planes.These patches are excluded in the first stage of clustering process.Later, the data points are individually added to the clusters afterevaluating the segmentation criteria. If the environment is highlycluttered, the PPC algorithmbehaves similar to the RG algorithm asit can find only few planar patches and the remaining data pointsbelong to the non-planar patches.

3.2. Boundary detection of clusters

The data points in the range image are assigned to theirrespective clusters as described in the previous section. In thissection, we describe the extraction process of the boundary pointsor the convex hull of each cluster. The extracted boundary pointswill eventually replace the data points in the range image as theplanar features. This feature extraction facilitates real-time 3Drobot mapping and form a pre-cursor step to feature-based 3Dregistration, which use planar features as input. We use the labelsof each of the clustered points in the range image (2D array) toidentify the boundary points of a given cluster. The boundary datapoints of each cluster are extracted by following the outline of a

cluster in the 2D range image array. This functionality is providedby a boundary detection program in image processing toolboxof MATLAB 7.6.0 to obtain the boundary points. In addition, theprogram is useful in identifying the boundary of holes in a givenplanar cluster. After extracting the boundary points, we fit themwith as many 3D lines as possible by sequentially following theboundary points of the polygon contour. If the collinear pointssatisfy orthogonal distance threshold (0.01 m was used in theexperiment) and obtain the edge points that best fits the boundaryof the planar region. If the data points do not satisfy the threshold,then they are retained as boundary points without fitting them toa line (e.g. data points lying on an edge of a plane segment with acurved boundary). The line is replaced by its two end points. Theextracted boundary points together form an irregular polygon. Wenote that boundary points cannot be fit directly with 3D lines on aplanar surface, since data points are affected by noise resulting innon-coplanar points. As a result, the boundary points are projectedto fit them on a single plane along the major axis so that theboundary points are coplanar. The major axis is defined as thenormal vector component that has the maximum value along thex, y or z axis. If nx is the major axis of the normal vector of theplanar cluster i.e. nx > ny & nx > nz , then the x-component of eachboundary point is recomputed as xi =

−ny · yi − nz · zi + p

/nx,

where i = 1, . . . ,m boundary points of a given cluster.

4. Experimental results

In this section, the performance of the PPC algorithm wasevaluated in a number of experimental scenarios. In the firstscenario, the range image was obtained through simulation toproduce a complete planar environment as the ground truth andto compare the performance of the PPC algorithmwith the RG and


Fig. 3.2. Flow chart showing the graph search algorithm for clustering point cloudto planar surfaces.

RANSAC plane segmentation algorithms at different noise levelsof range measurement. Then, the PPC algorithm was evaluated inreal-time using real range images acquired by a robot in a corridorconsisting of largely planar surfaces. The third experiment wasdesigned to test the robustness of the PPC algorithm in a highlycluttered environment. The range images that are acquired by the

range scanners in both the simulated and real environments arestored in a 2D array in the spherical coordinate system.

4.1. Quantitative analysis in a simulated environment

The 3D range images are simulated in a planar environmentconsisting of eight planes, which includes walls, a floor and aceiling. The range data was obtained in form of a (360 × 360)range image stored in a 2D array using ray tracing and ray-polygonintersection algorithm. The sensor coordinate frame is assumedto be at a certain pose in the middle of the room. The rangeis calculated along the ray that originates at the center of thecoordinate frame and intersects a planar segment along a certaindirection. The indices of the array establish connectivity of eachelement to its neighbors. The direction of the ray is computedusing unique elevation (θ ) and azimuth (ϕ) angles. The range datawas simulated with noise added and measured as a percentage ofthe range distance. This noise is an approximative model of thereal range image described in the laser range scanner specificationsheet. We varied the noise in the range image up to 1% of therange distance and evaluated the performance of the PPC, RG andRANSAC plane segmentation algorithms at different noise levels.The three parameters (σ , λ, δ) of the PPC algorithm were fine-tuned empirically to achieve the best clustering results for rangeimages simulated at different noise levels. The clustering resultsfor two Range Images (RI) with noise variance of (σ 2

RI = 0.5%) and(σ 2

RI = 1%) are shown in Figs. 4.1 and 4.2 respectively. In thesefigures, we included all extracted planes for visualization exceptthe ceiling to avoid overlapping of extracted ceiling and floor datapoints. We can observe in Fig. 4.1 that the three algorithms incomparison showed high accuracy in segmentation under σ 2

RI =

0.5% noise level. However, the RG algorithm resulted in under-segmentation for one distant plane from the center under σ 2

RI = 1%noise level. This plane was split into two individual planes, whichcan be observed in Fig. 4.2(b).

Segmentation results of each algorithm can be fine-tuned by aset of parameters. The patch size in the PPC algorithm was chosenas (5 × 5) to obtain the optimal results. With a fixed patch size,the parameter (σ ) in the Gaussian similarity metric governs thetolerance of noise levels in the normal vectors. A large value ofσ indicates more tolerance for noise in the normal vectors of thepatches. It is set to lower values when the noise is higher than(σ 2

RI ≥ 1%) of the rangemeasurement. The parameter (λ) is the cut-off threshold for the Gaussian similaritymetric function to identifycoplanar patches. If the (λ) is set to lowvalues, coplanar tolerance ishigh.More patches that do not belong to the cluster are assigned toit resulting in over-segmentation. If the (λ) is set to higher values,coplanar tolerance is low and may result in under-segmentation.Both (σ , λ) are unit-less quantities and has a range of [0,1]. Theparameters can be varied depending on the noise levels of therange sensor. The parameter δ (measured inmeters) is ameasure ofthe orthogonal distance between the plane models of two patcheswhen evaluating the segmentation criteria. A description of theparameters of the RG algorithm can be found in [20] and theRANSAC segmentation algorithm in [27]. All parameters were setempirically to achieve the highest qualitative accuracy for each ofthe algorithms.

Table 4.1 lists the quantitative results of plane segmentationof noisy range images in a simulated environment by the threealgorithms, namely PPC, RG and RANSAC plane segmentation.The algorithms were compiled and executed on Intel PentiumCore 2 Duo 2.4 GHz processor and 2 GB RAM, Ubuntu 10.10OS, Robot Operating System (ROS) framework. The table includesthe actual cluster size of each planar segment (ground truth),accuracy of segmentation of each cluster in percentage, over-segmentation in percentage with respect to the actual cluster


a

b

c

Fig. 4.1. Outcome of PPC (a), RG (b) and RANSAC (c) plane segmentation algorithms for a noisy range image with noise levels of 0.5% of the range distance measured.

size and overall computation time for each algorithm. It canbe noted that the PPC algorithm was able to extract all eightplanes undergoing a small percentage of over-segmentation orunder-segmentation under noisy conditions. The computationtime in each case was under 0.08 s. The number of respectivecluster points varied slightly compared with the RG and RANSACplane segmentation algorithms, which depends on whether theedge points were included or excluded from the plane segments.

The RANSAC-based plane segmentation executed under 1.18 sand was able to accurately extract all eight plane segments forboth noisy range images. For the range image with 1.0% noise,the RANSAC-based plane segmentation algorithm extracted edgepoints as a separate plane (272 points). The RG algorithm wassuccessful in extracting eight planes for the range image with0.5% noise. However, one of the wall segments was under-segmented splitting them into two separate planes for the


a

b

c

Fig. 4.2. Outcome of PPC (a), RG (b) and RANSAC (c) plane segmentation algorithms for a noisy range image with noise levels of 1% of the range distance measured.

range image with 1% noise (Fig. 4.2(b)). Further fine-tuning ofparameters for RG algorithm resulted either in one plane segmentblending into another cluster or leading to under-segmentation.We can attribute the significant improvement in speed of thePPC algorithm to eliminating redundancy in data fitting. Theredundancy can be seen when the Hessian plane parameters andcovariance matrix of all data points for a given planar segmentare recomputed whenever a new point is added to a cluster by

the RG algorithm. In RANSAC plane segmentation algorithm, thedata points are repeatedly tested to verify if it fits a hypothesisplane model and sometimes tested for an incorrect hypothesisplane model. We avoid the redundant approaches and insteadcompute Hessian plane parameters of each patch at the beginningof the algorithm and hence incur an additional computation cost,which is seen in Table 4.1. Once the planar patches are includedin their respective clusters, the unassigned points are added to


Table 4.1Performance of PPC, RG and RANSAC plane segmentation algorithms.

Patch-based plane clustering

True values Noise levels—0.0 variance Noise levels—0.005 variance Noise levels—0.01 variancePlane no. Cluster size Accuracy (%) Over-seg (%) Accuracy (%) Over-seg (%) Accuracy (%) Over-seg (%)

P1 40708 99.98 1.27 99.89 1.08 99.57 0.58P2 39432 99.86 0.30 99.49 0.21 99.73 0.07P3 18022 99.59 0.90 99.68 0.75 99.68 1.08P4 13815 98.15 0.41 98.23 0.29 98.41 0.32P5 6293 98.92 0.41 99.22 0.84 99.19 2.07P6 4721 95.36 2.73 95.53 2.50 96.00 3.20P7 3564 94.22 1.96 94.44 4.43 94.47 2.33P8 2537 99.33 1.69 99.68 1.89 99.01 1.06Normal calc. time (ms) 13.857 – 15.399 – 15.524Clustering time (ms) 35.668 – 31.598 – 33.069

Refinement time (ms) 16.096 – 27.262 – 36.833

Total time (ms) 65.621 – 74.259 – 85.426

Parameters used

Sigma (σ ) 0.98 0.98 0.95Lamda (λ) 0.98 0.95 0.955Delta (δ) 0.012 0.015 0.028

Region growing plane segmentation

Ground truth Noise levels—0.0 variance Noise levels—0.005 variance Noise levels—0.01 variancePlane Cluster size Accuracy (%) Over-seg (%) Accuracy (%) Over-seg (%) Accuracy (%) Over-seg (%)

P1 40708 100.00 0.36 99.98 0.27 99.92 0.30P2 39432 99.98 0.13 99.95 2.35 99.78 2.29P3 18022 99.98 0.47 99.93 0.32 99.77 0.53P4 13815 100.00 0.66 99.96 0.58 99.56 0.77P5 6293 99.52 0.37 99.49 0.70 95.47 1.03P6 4721 96.08 0.49 99.94 0.57 99.60 1.42P7 3564 99.97 0.45 76.04 0.67 – –P8 2537 99.96 11.82 99.57 3.31 99.92 3.74Total time (ms) 19158.7 – 19710.5 – 18714.1

Parameters used

Delta (δ) 0.25 0.25 0.4Epsilon (ε) 0.1 0.1 0.1Gamma (γ ) 0.005 0.008 0.014

RANSAC plane segmentation

Ground truth Noise levels—0.0 variance Noise levels—0.005 variance Noise levels—0.01 variancePlane Cluster size Accuracy (%) Over-seg (%) Accuracy (%) Over-seg (%) Accuracy (%) Over-seg (%)

P1 40708 100.00 0.37 99.98 0.48 99.92 0.52P2 39432 100.00 0.40 99.98 0.46 99.90 0.41P3 18022 99.97 0.14 99.71 0.01 99.36 0.32P4 13815 99.98 0.49 99.67 0.32 98.67 0.14P5 6293 99.95 1.62 99.38 0.78 99.08 0.51P6 4721 100.00 0.17 99.81 0.32 99.70 1.78P7 3564 99.97 0.20 99.05 0.00 95.17 0.00P8 2537 99.92 0.20 99.68 0.04 99.37 0.00Total time (ms) 1120.32 – 1103.62 – 1184.06

Parameters used

Threshold (λ) 0.005 0.015 0.022

their respective cluster, which incurred an additional computationcost less than 0.02 s for the 360 × 360 range image. The overallprocessing time of the PPC algorithm is only less than 1/10th ofthe RANSAC-based plane segmentation algorithm [27] and muchless compared to the RG algorithm [20]. The accuracy of thePPC algorithm varies between 99% and 94% for different clustersizes. PPC algorithm under-went a small percentage of over-segmentation ranging from 0.07% to 3.2% for the range image with1% noise.

4.2. Performance of PPC algorithm in structured indoor environments

This section analyzes the performance of the PPC algorithmin a largely planar environment consisting of walls, ceiling, floor

and doors as shown in Fig. 4.3. The algorithm was able to clusterthe planar patches to a good accuracy in the indoor environment(Fig. 4.4). The Gaussian parameters (σ = 0.95, λ = 0.95) and thedistance threshold (d < 0.01 m) was set empirically while fine-tuning the PPC algorithm.

4.3. Performance of PPC algorithm in a cluttered environment

In this experiment, we acquired four range images at differentlocations in a highly cluttered laboratory environment (Fig. 4.5) toevaluate the robustness of the clustering algorithm. The qualitativeresults of the PPC algorithm are presented in the form of clustersand represented by different colors as shown in Fig. 4.6. Table 4.2presents the quantitative results of clustering four range images in


Fig. 4.3. Experimental setup shows a ground robot acquiring a 3D range image inan office corridor. A few additional objects can be seen.

Fig. 4.4. Shows clustered range image acquired in the office corridor. All the largeplanar clusters could be identified.

the cluttered environment. We achieved a high data compressionrate close to 96%. We define the data compression as the ratioof number of data points present in a given planar cluster to theboundary points that outline the planar cluster (and form irregularpolygons) [7]. Polygon maps are an efficient way of representing3D data and decreases stored memory by replacing large numbersof data points present in each cluster of the range image by aset of boundary points that represent planar regions. The PPCalgorithm was robust to sensor noise and able to cluster theplanar regions from all four range images. The number of clustersextracted in the process is dependent on the parameters, whichwere fine-tuned empirically to achieve highest accuracy for thePPC algorithm. The PPC algorithm was executed in Matlab 7.6.0,2 GB RAM, 2.4 GHz Intel R⃝ processor to extract polygons fromthe four range images. Since we do not have the ground truthmodel of the real environments, it is difficult to make meaningfulquantitative comparison with other algorithms such as the RG andRANSAC-based plane segmentation algorithms.

4.4. Limitations of the PPC algorithm

The PPC algorithm showed a significant improvement incomputation time without much loss of accuracy in comparison

Table 4.2Performance of PPC clustering algorithm in a cluttered environment.

Range image 1 2 3 4

No. of extracted planes 52 54 67 60No. of clustered points 91182 83497 78532 81813No. of boundary points 6123 6014 5673 5426Data compression (%) 96.81 95.90 96.14 96.31

PPC algorithm parameters (σ = 0.95, λ = 0.95, δ = 0.04 m).

with the state-of-the-art RANSAC and RG algorithms. The analysisin Section 2.1 indicates that normal vectors calculation of localpatches can be accurate enough for plane segmentation if thepatch size is properly selected. It is especially true in structuredindoor environments consisting of large planar surfaces, wherethe larger patch size leads to more computational efficiency.However, it is necessary to select small patch size in highlycluttered environments. In worst case scenario if the environmentis highly cluttered with no planar regions, the PPC algorithmbehaves invariably like a RG algorithm since it cannot find anyplanar patches to cluster. The sampling of range data into patchesleads to inaccurate inclusion of points around the border of thepolygons where some patches may include points from two ormore planes. A higher threshold for the Gaussian similarity metriceliminates the inclusion of this type of patch into neighboringplanar clusters. A lower threshold leads to false positive datapoints being included into the cluster. Thus, an additional step wasnecessary to handle border points and patches. The refinementprocess is an additional step we incorporated in our model wherethe points of border patches were tested for coplanarity condition.The test ensured that the data points along the border of eachpolygon were included into the correct cluster if they were foundto be a false positive result for a given cluster. The computationtime of this refinement process is a function of number of clustersextracted from a range image, which in turn is dependent onthe environment being scanned. However, the time taken by therefinement process is negligible and the overall computationaltime of the PPC algorithm is only less than 1/10 of that ofthe RANSAC-based plane segmentation algorithms and much lesscompared to the RG algorithm.

5. Conclusion

We introduced a novel and fast approach to cluster range datainto planes and extract polygons fromnoisy range images acquiredby 3D range scanners in indoor environments. The range data isinitially grouped into mutually-exclusive patches or grids insteadof considering each data point individually. We showed that thenormal vectors of the range data are accurate enough for clusteringwhen extracted from local patches. We described a clusteringalgorithm that can handle noisy range data in this process. Theclustering algorithm compares the Hessian plane parameters andmean positions of neighboring patches grouped from a rangeimage. We introduced a graph search method in the clusteringprocess that compares the neighboring patches and clusters theminto planar surfaces. The simulation and experimental resultsshowed that the proposed algorithm achieved a high level ofdata compression of range image acquired in structured indoorenvironments. The processing speed of the Patch-based PlaneClustering algorithm is significantly increased in comparison withthe state-of-the-art RG and RANSAC-based plane segmentationalgorithms. The extracted polygons from noisy range images canbe used as inputs to a class of feature-based scan registrationalgorithms and facilitate the robotic applications such as 3Dmapping and indoor navigation.


Fig. 4.5. The experimental setup showing a pioneer R⃝ robot equipped with a Hokuyo R⃝ range scanner mounted on a rotating platform to acquire a 3D range image in acluttered laboratory environment.

Range Image 1 Range Image 2

Range Image 3 Range Image 4

Fig. 4.6. Result of patch-based plane clustering of four range images acquired in a highly cluttered indoor environment at 4 different locations.

References

[1] P.J. Besl, N.D.McKay, Amethod of registration of 3-D shapes, IEEE Transactionson Pattern Analysis and Machine Intelligence 14 (1992) 239–256.

[2] J. Xiao, A. Sadegh, M. Elliot, A. Calle, A. Prasad, H.M. Chiu, Design of mobilerobots with wall climbing capability, in: IEEE/ASME International Conferenceon Advanced Intelligent Mechatronics, 2005, pp. 438–443.

[3] W. Morris, I. Dryanovski, J. Xiao, 3D indoor mapping for micro-UAVs usinghybrid range finders andmulti-volume occupancy grids, in: RGB-DWorkshopat Robotics; Robotics Science and Systems, Zaragoza, Spain, 2010.

[4] R. Kaushik, J. Xiao, S.L. Joseph,W.Morris, Polygon-based laser scan registrationby heterogeneous robots, in: IEEE International Conference on Robotics andBiomimetics, Tianjin, China, 14–18 Dec, 2010, pp. 1618–1623.

[5] K. Pathak, A. Birk, N. Vaskevicius, M. Pfingsthorn, S. Schwertfeger, J. Poppinga,


Online 3D SLAM by registration of large planar surface segments and closedform pose-graph relaxation, Journal of Field Robotics 27 (2010) 52–84.

[6] H. Surmann, A. Nüchter, J. Hertzberg, An autonomous mobile robot with a 3Dlaser range finder for 3D exploration and digitization of indoor environments,Robotics and Autonomous Systems 45 (2003) 181–198.

[7] R. Kaushik, S.L. Joseph, W. Morris, J. Xiao, Polygon-based 3D scan registrationwith dual-robots in structured indoor environments, International Journal ofRobotics and Automation 27 (2012).

[8] K. Pathak, N. Vaskevicius, J. Poppinga, M. Pfingsthorn, S. Schwertfeger,A. Birk, Fast 3D mapping by matching planes extracted from range sensorpoint-clouds, in: IEEE/RSJ International Conference on Intelligent Robots andSystems, St. Louis, MO, USA, 2009, pp. 1150–1155.

[9] V. Sequeira, K. Ng, E. Wolfart, J.G.M. Goncalves, D. Hogg, Automatedreconstruction of 3D models from real environments, ISPRS Journal ofPhotogrammetry and Remote Sensing 54 (1999) 1–22.

[10] E. Takeuchi, T. Tsubouchi, A 3-D scan matching using improved 3-D nor-mal distributions transform for mobile robotic mapping, in: IEEE/RSJ Inter-national Conference on Intelligent Robots and Systems, Beijing, China, 2006,pp. 3068–3073.

[11] G. Turk, M. Levoy, Zippered polygon meshes from range images, in: 21stInternational Conference on Computer Graphics and Interactive Techniques,1994, pp. 311–318.

[12] F. Bernardini, J. Mittleman, H. Rushmeier, C. Silva, G. Taubin, The ball-pivotingalgorithm for surface reconstruction, IEEE Transactions on Visualization andComputer Graphics 5 (1999) 349–359.

[13] M. Garland, P.S. Heckbert, Surface simplification using quadric error metrics,in: SIGGRAPH’97: Conference on Computer Graphics and Interactive Tech-niques, 1997, pp. 209–216.

[14] H. Hoppe, Progressive meshes, in: SIGGRAPH’96: Conference on ComputerGraphics and Interactive Techniques, 1996, pp. 99–108.

[15] P. Cignoni, C. Montani, R. Scopigno, DeWall: a fast divide & conquer Delaunaytriangulation algorithm in Eˆd, Computer-Aided Design 30 (1998) 333–341.

[16] A. Hoover, G. Jean-Baptiste, X. Jiang, P.J. Flynn, H. Bunke, D.B. Goldgof,K. Bowyer, D.W. Eggert, A. Flitzgibbon, R.B. Fisher, An experimentalcomparison of range image segmentation algorithms, IEEE Transactions onPattern Analysis and Machine Intelligence 18 (2002) 673–689.

[17] D. Hähnel, W. Burgard, S. Thrun, Learning compact 3D models of indoorand outdoor environments with a mobile robot, Robotics and AutonomousSystems 44 (2003) 15–27.

[18] J. Weingarten, G. Gruener, R. Siegwart, A fast and robust 3D feature extractionalgorithm for structured environment reconstruction, in: InternationalConference on Advanced Robotics, Coimbra, Portugal, 2003.

[19] R. Triebel,W. Burgard, F. Dellaert, Using hierarchical EM to extract planes from3D range scans, in: IEEE International Conference on Robotics andAutomation,Barcelona, Spain, 2005, pp. 4437–4442.

[20] J. Poppinga, N. Vaskevicius, A. Birk, K. Pathak, Fast plane detectionand polygonalization in noisy 3D range images, in: IEEE/RSJ Interna-tional Conference on Intelligent Robots and Systems, Nice, France, 2008,pp. 3378–3383.

[21] S. Kurogi, D. Wakeyama, H. Koya, S. Okada, S. Inoue, T. Nishida, Applicationof CAN2 to plane extraction from 3D range images, in: IEEE InternationalJoint Conference on Neural Networks, IEEEWorld Congress on ComputationalIntelligence, Hongkong, China, 2008, pp. 2327–2332.

[22] J. Weingarten, R. Siegwart, 3D SLAM using planar segments, in: IEEE/RSJInternational Conference on Intelligent Robots and Systems, 2006,pp. 3062–3067.

[23] S. Grzonka, G. Grisetti, W. Burgard, Towards a navigation system forautonomous indoor flying, in: IEEE International Conference on Robotics andAutomation, Kobe, Japan, 2009.

[24] C.C. Chen, I. Stamos, Range image segmentation for modeling and objectdetection in urban scenes, in: International Conference on 3-D Digital Imagingand Modeling, Montréal, Canada, 2007, pp. 185–192.

[25] I. Dyranovski, W. Morris, J. Xiao, Multi-volume occupancy grids: an efficientprobabilistic 3D mapping model for micro-aerial vehicles, in: IEEE/RSJInternational Conference on Intelligent Robots and Systems, Taipei, Taiwan,2010, pp. 1553–1559.

[26] M.A. Fischler, R.C. Bolles, Random sample consensus: a paradigm for modelfitting with applications to image analysis and automated cartography,Graphics and Image Processing 24 (1981) 381–395.

[27] R. Schnabel, R. Wahl, R. Klein, Efficient RANSAC for point-cloud shapedetection, Computer Graphics Forum 26 (2007) 214–226.

[28] F. Tarsha-Kurdi, T. Landes, P. Grussenmeyer, Hough transform and extendedRANSAC algorithms for automatic detection of 3D building roof planes fromLIDAR data, Science and Technology 36 (2007) 407–412.

[29] F. Tarsha-Kurdi, T. Landes, P. Grussenmeyer, Extended RANSAC algorithm forautomatic detection of building roof planes from LIDAR data, Photogrammet-ric Journal of Finland (2008).

[30] T.M. Awwad, Q. Zhu, Z. Du, Y. Zhang, An improved segmentation approachfor planar surfaces from unstructured 3D point clouds, The PhotogrammetricRecord 25 (2010) 5–23.

[31] A.D. Hofmann, H.-G. Maas, A. Streilein, Derivation of roof types by clusteranalysis in parameter spaces of airborne laserscanner point clouds, in: IAPRSInternational Archives of Photogrammetry and Remote Sensing and SpatialInformation Sciences, Dresden, Germany, 2003, pp. 112–117.

[32] J.M. Biosca, J.L. Lerma, Unsupervised robust planar segmentation of terrestriallaser scanner point clouds based on fuzzy clusteringmethods, ISPRS Journal ofPhotogrammetry and Remote Sensing 63 (2008) 84–98.

[33] D. Marshall, G. Lukacs, R. Martin, Robust segmentation of primitives fromrange data in the presence of geometric degeneracy, IEEE Transactions onPattern Analysis and Machine Intelligence 23 (2001) 304–314.

[34] S. Thrun, C. Martin, Y. Liu, D. Hähnel, R. Emery-Montemerlo, D. Chakrabarti,W. Burgard, A real time expectation-maximization algorithm for acquiringmultiplanar maps of indoor environments with mobile robots, IEEE Transac-tions on Robotics 20 (2004) 433–442.

[35] T. Rabbani, F.A. Van Den Heuvel, G. Vosselman, Segmentation of point cloudsusing smoothness constraint, in: ISPRS Commission V Symposium ‘ImageEngineering and Vision Metrology’, Dresden, 2006, pp. 248–253.

[36] G.M. Hegde, C. Ye, Extraction of planar features from swissranger SR-3000 range images by a clustering method using normalized cuts, in: IEEEInternational Conference on Intelligent Robots and Systems, St. Louis, USA,2009, pp. 4034–4039.

[37] R.B. Rusu, S. Cousins, 3D is here: point cloud library (PCL), in: IEEE InternationalConference on Robotics and Automation, Shanghai, China, 2011.

[38] S. Thrun,W. Burgard, D. Fox, Probabilistic Robotics, in: Intelligent Robotics andAutonomous Agents, The MIT Press, Cambridge, MA, 2005.

[39] V.Y. Pan, Z.Q. Chen, The complexity of the matrix eigenproblem, in: STOC’99Proceedings of the Thirty-First Annual ACM Symposium on Theory ofComputing 1999.

Ravi Kaushik received his Ph.D. degree (2011) in Com-puter Science from The Graduate Center, The City Univer-sity of New York (CUNY), NY, USA. He worked as ResearchAssociate from 2006 to 2011 and conducted research on3Dmappingwithmultiple robots at the CCNYRobotics Labat The City College of NewYork (CCNY), CUNY. He receivedthe B.E. degree (2003) in Instrumentation Technology fromM.S. Ramaiah Institute of Technology, Bangalore, India. Hereceived the M.E. degree (2006) in Electrical Engineeringfrom The City College of New York. His research interestsinclude 3D mapping with multi-robot systems, range and

vision sensor technologies, 3D geometric modeling and embedded systems.

Jizhong Xiao (SM’06) received the B.S. and M.S. degreesfrom the East China Institute of Technology, Nanjing,China, in 1990 and1993, respectively, theM.E. degree fromNanyang Technological University, Singapore, in 1999, andthe Ph.D. degree from the Michigan State University, EastLansing, in 2002. He is currently an Associate Professorof electrical engineering with The City College, CityUniversity of New York (CCNY/CUNY City College). Hestarted the Robotics Research Program at CCNY in 2002as the Founding Director of CCNY Robotics Laboratory. Hiscurrent research interests include robotics and control,

computer vision, sensor fusion, autonomous navigation, and multi-agent systems.Dr. Xiao received the US National Science Foundation CAREER Award in 2007 andthe CCNY Outstanding Mentoring Award in 2011.

accelerated patch-based planar clustering of noisy range images in indoor environments for robot...

Documents