[ieee 2009 ieee symposium on computational intelligence for image processing (ciip) - nashville, tn,...

7
3-D Reconstruction and Automatic Fusion of Edge Maps from Different Modalities of an Object Umesh C. Pati 1 , Aditya Modi 2 , Pranab K. Dutta 3 , and Alok Barua 3 , Member, IEEE 1 Department of Electronics and Communication Engineering, National Institute of Technology, Rourkela, Orissa, India 2 GE Infra Energy, Mumbai, Maharastra, India 3 Department of Electrical Engineering, Indian Institute of Technology, Kharagpur, West Bengal, India. Abstract - The paper presents reconstruction of a 3-D model as well as automatic fusion of edge maps extracted from the 3-D model and intensity image of an object. A data acquisition system has been developed with a laser source and camera. The 3-D model of the object is reconstructed by registration and integration of a set of range images obtained from scanning the object by laser beam. The intensity image of the object is captured under illumination light. Edge maps in both the cases are extracted by appropriate techniques. The corner points of various shapes in both the edge maps are obtained by implementing a technique using shape signatures. 3-D edge points are mapped to 2-D plane with the help of perspective transformation. A novel algorithm has been proposed for the establishment of automatic correspondence between the corner points in both the edge maps. The method for automatic fusion of edge maps using affine transformation followed by iterative closest point algorithm has been introduced. Range and intensity images are complementary in nature and provide a richness of description which is not possible with either source in isolation. I. INTRODUCTION ethods to digitize and reconstruct the shapes of complex three-dimensional (3-D) objects have evolved rapidly due to much attention from many industrial and research groups. There has been considerable interest in the construction of 3-D models for applications where the focus is more on visualization of the object by humans [1]. 3-D reconstruction of an object usually takes place in three main steps which are data acquisition, registration and integration [2]. Shape is acquired as a set of coordinates corresponding to points on the surface of the object. These coordinates measure the distance or depth of the point from a measuring device and are called range values. Optical triangulation is one of the most popular approaches for range finding. In this method, a light pattern, typically of laser, is projected onto an object and then a camera is used to observe how the pattern is illuminated on the surface of the object. Structured light scanners have been used in manifold applications over the last few years [3]-[7]. A single scan by a structured light scanner provides a range image that covers only part of an object. Therefore, multiple scans from different view points are necessary to capture the entire surface of the object. Registration is the process in which the multiple views and their associated coordinate frames are aligned into a single global coordinate frame. When aligning two range images, the translation and rotation that bring the points as close together as possible have to be found out. For rigid surfaces, the transformation consists of a rotation matrix R and a translation vector t [8]. The rigid transformation is defined as y Rx t = + (1) where x is the point in camera frame coordinates and y is the same point in world coordinates. Existing registration techniques can be mainly categorized into feature matching and surface matching. Iterative closest point (ICP) algorithm proposed by Besl et al. [9] is the most popular algorithm for accurately registering a set of range images. Successful registration aligns all the range images into a common coordinate system. However, the registered range images taken from adjacent viewpoints will typically contain overlapping surfaces with common features in the areas of overlap. The integration process eliminates the redundancies and generates a single connected surface model. Multisensor fusion refers to the synergistic combination of different sources of sensory information into a single representational format. Many of the advanced sensors produce images. Because each kind of image sensor can only focus on a given operating range and environmental conditions, it may not receive all the necessary information. A human observer cannot reliably combine visual information by viewing multiple images separately. Image fusion refers to the techniques that integrate images obtained using different imaging techniques or with different acquisition parameters from different image sensors so that the new composite image is more useful for the purpose of human visual perception and the computer processing [10]. Given a set of input images, different fused images may be created depending on the specific application using various fusion algorithms [11]-[13]. In a broad sense, image fusion is performed at three different processing levels according to the stage at which the fusion takes place. These are pixel level fusion, feature level fusion and decision level fusion [14]. While considerable work has been done at pixel level, less work has been done at feature level and decision level image fusion. Feature level fusion is a medium level image fusion. This level can be used as a means of creating additional composite features. Features correspond to characteristics extracted from the original images, which can be edges, corners, lines and texture features. It is not feasible M 978-1-4244-2760-4/09/$25.00 ©2009 IEEE

Upload: alok

Post on 04-Apr-2017

221 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: [IEEE 2009 IEEE Symposium on Computational Intelligence for Image Processing (CIIP) - Nashville, TN, USA (2009.03.30-2009.04.2)] 2009 IEEE Symposium on Computational Intelligence for

3-D Reconstruction and Automatic Fusion of Edge Maps from Different Modalities of an Object

Umesh C. Pati 1, Aditya Modi 2, Pranab K. Dutta 3, and Alok Barua 3, Member, IEEE 1 Department of Electronics and Communication Engineering,

National Institute of Technology, Rourkela, Orissa, India 2 GE Infra Energy, Mumbai, Maharastra, India

3 Department of Electrical Engineering, Indian Institute of Technology, Kharagpur, West Bengal, India.

Abstract - The paper presents reconstruction of a 3-D model

as well as automatic fusion of edge maps extracted from the 3-D model and intensity image of an object. A data acquisition system has been developed with a laser source and camera. The 3-D model of the object is reconstructed by registration and integration of a set of range images obtained from scanning the object by laser beam. The intensity image of the object is captured under illumination light. Edge maps in both the cases are extracted by appropriate techniques. The corner points of various shapes in both the edge maps are obtained by implementing a technique using shape signatures. 3-D edge points are mapped to 2-D plane with the help of perspective transformation. A novel algorithm has been proposed for the establishment of automatic correspondence between the corner points in both the edge maps. The method for automatic fusion of edge maps using affine transformation followed by iterative closest point algorithm has been introduced. Range and intensity images are complementary in nature and provide a richness of description which is not possible with either source in isolation.

I. INTRODUCTION ethods to digitize and reconstruct the shapes of complex three-dimensional (3-D) objects have

evolved rapidly due to much attention from many industrial and research groups. There has been considerable interest in the construction of 3-D models for applications where the focus is more on visualization of the object by humans [1]. 3-D reconstruction of an object usually takes place in three main steps which are data acquisition, registration and integration [2]. Shape is acquired as a set of coordinates corresponding to points on the surface of the object. These coordinates measure the distance or depth of the point from a measuring device and are called range values. Optical triangulation is one of the most popular approaches for range finding. In this method, a light pattern, typically of laser, is projected onto an object and then a camera is used to observe how the pattern is illuminated on the surface of the object. Structured light scanners have been used in manifold applications over the last few years [3]-[7]. A single scan by a structured light scanner provides a range image that covers only part of an object. Therefore, multiple scans from different view points are necessary to capture the entire surface of the object. Registration is the process in which the multiple views and their associated coordinate frames are aligned into a single global coordinate frame. When aligning

two range images, the translation and rotation that bring the points as close together as possible have to be found out. For rigid surfaces, the transformation consists of a rotation matrix R and a translation vector t [8]. The rigid transformation is defined as y Rx t= + (1) where x is the point in camera frame coordinates and y is the same point in world coordinates. Existing registration techniques can be mainly categorized into feature matching and surface matching. Iterative closest point (ICP) algorithm proposed by Besl et al. [9] is the most popular algorithm for accurately registering a set of range images. Successful registration aligns all the range images into a common coordinate system. However, the registered range images taken from adjacent viewpoints will typically contain overlapping surfaces with common features in the areas of overlap. The integration process eliminates the redundancies and generates a single connected surface model.

Multisensor fusion refers to the synergistic combination of different sources of sensory information into a single representational format. Many of the advanced sensors produce images. Because each kind of image sensor can only focus on a given operating range and environmental conditions, it may not receive all the necessary information. A human observer cannot reliably combine visual information by viewing multiple images separately. Image fusion refers to the techniques that integrate images obtained using different imaging techniques or with different acquisition parameters from different image sensors so that the new composite image is more useful for the purpose of human visual perception and the computer processing [10]. Given a set of input images, different fused images may be created depending on the specific application using various fusion algorithms [11]-[13]. In a broad sense, image fusion is performed at three different processing levels according to the stage at which the fusion takes place. These are pixel level fusion, feature level fusion and decision level fusion [14]. While considerable work has been done at pixel level, less work has been done at feature level and decision level image fusion. Feature level fusion is a medium level image fusion. This level can be used as a means of creating additional composite features. Features correspond to characteristics extracted from the original images, which can be edges, corners, lines and texture features. It is not feasible

M

978-1-4244-2760-4/09/$25.00 ©2009 IEEE

Page 2: [IEEE 2009 IEEE Symposium on Computational Intelligence for Image Processing (CIIP) - Nashville, TN, USA (2009.03.30-2009.04.2)] 2009 IEEE Symposium on Computational Intelligence for

to depend on any type of image to provide a complete edge map for the scene. Therefore, combination of different sources of information can help to locate most of the important discontinuities in the image and provide more reliable information about the scene or object.

In this work, the 3-D model of an object has been reconstructed from the images obtained by scanning the object by a laser beam. The steps involved for this purpose are image acquisition by the developed system, registration between different views of range images, integration of the registered views and generation of a 3-D model of the object. Extraction of edge maps from the 3-D model and intensity image has been made by suitable edge detectors. We have introduced the concept of automatic fusion of edge maps extracted from these two different modalities. Automatic fusion of edge maps has been performed by affine transformation followed by ICP algorithm.

II. RECONSTRUCTION OF 3-D MODEL

A. Data Acquisition

The object for the experimentation purposes is made of wood and its external dimensions are 12.2 cm × 12.5 cm × 3.5 cm. The surface of the object contains features of different shapes like triangle, notch, rectangle and circle. The object is placed on a horizontal and smooth platform. The method of optical triangulation is being used to acquire 3-D data of the object. A red line diode laser which spreads the laser beam into a sheet of light with a cylindrical lens is placed on a mount. The laser plane scans the object with the help of a manually operated mechanical arrangement on the mount. Part of the laser stripe falls on both sides of the object on the platform. A charge-coupled device (CCD) camera interfaced with Silicon Graphics machine acquires series of range images while scanning is performed. The experimental setup and the object are shown in Fig. 1. The process is repeated for the rest three different views by rotating the object manually by 90 degrees in anticlockwise direction each time so that all of the surface detail is captured.

Fig. 1. Experimental setup and the object.

B. Image Pre-processing

The captured RGB images are converted into grayscale and then binary images. The binary image is subjected to thinning (skeletonizing) algorithm so that a single pixel thin line representing the illuminated region is obtained. A laser scanned image and its skeletonized image are shown in Fig. 2 (a) and (b) respectively.

(a) (b)

Fig. 2 (a). Laser image (b) Skeletonized image

The coordinate values in 3-D space (X, Y, Z) for all the illuminated pixels have been calculated using the method of optical triangulation. The laser line on the platform is considered as the base line for the particular scan. Fig. 3 shows the relative position of object with respect to laser source and camera.

Fig. 3. Relative position of laser, object and camera

From Fig. 3, tan( ) tan( )

tan( ) tan( )yh θ β

θ β=

+ (2)

It is clear from (2) that height h of a particular illuminated point on the surface of the object is proportional to distance y between the laser line on the object and that on platform considering the fact that θ and β are constant for the particular point. The conversion from image coordinates to world coordinates is performed using Xcoeff and Ycoeff obtained experimentally. Xcoeff and Ycoeff are inter pixel distances along X and Y direction respectively. This process is repeated for all the images of the first view and then for the rest three views C. Registration

For successful registration of range images from different views, we have applied feature matching technique for coarse registration followed by ICP algorithm for fine registration. In the present work, the range images of other three views have been registered with that of the first view. For that purpose, the coordinate values of the range images of all the views except the first one have been rotated in clockwise direction by an angle that the object was rotated to take the images (900, 1800 and 2700 respectively). This is accomplished by the application of the rotational matrix Rθ

in (1) where θ is the corresponding angle of rotation. As

Page 3: [IEEE 2009 IEEE Symposium on Computational Intelligence for Image Processing (CIIP) - Nashville, TN, USA (2009.03.30-2009.04.2)] 2009 IEEE Symposium on Computational Intelligence for

there is no translation, translation vector t has been assumed to be zero. The rotation matrix Rθ is given by

cos sin 0 0sin cos 0 00 0 1 00 0 0 1

θ θθ θ

⎛ ⎞⎜ ⎟−⎜ ⎟=⎜ ⎟⎜ ⎟⎝ ⎠

(3)

The left bottom corner point on the surface of the object in the range images has been chosen as the feature for registration of different views. The steps for finding the left bottom corner point automatically are as follows:

Step 1: Initially, the first point encountered with Z coordinate above a threshold value has been picked as the left bottom corner.

Step 2: When another point above the threshold is encountered, it takes the place of earlier value if both the X and Y coordinates of the point are less than or equal to that of the earlier point.

Step 3: This process is repeated for all the points in the range image of a particular view and the resultant point is the left bottom corner on the surface of the object.

The left bottom corner points on the surface of the object in the range image of the first view and the rotated range images of the rest three views are identified. Then the range images are aligned so that the corner points overlap with respect to the origin.

For fine registration, iterative closest point algorithm has been implemented which performs process of registration iteratively. Besl et al. [9] have used an efficient solution proposed by Horn [15] to obtain R and t using unit quaternions. The same solution has been implemented in this work. The range images of the four different views after being subjected to feature based registration followed by ICP algorithm have been shown in Fig. 4.

Fig. 4. Registered range images

D. Integration

The registered images are a cloud of 3-D points. The laser points on the platform have been removed for the integration purpose as the points on the surface of the object are the desired points for 3-D reconstruction. The steps of integration algorithm include detection of overlapping points, merger of corresponding points and generation of a single connected surface model. Surface fitting has been accomplished using the griddata function of MATLAB. It uses triangle-based cubic interpolation method based on Delaunay triangulation of data for surface fitting. A median filter of window size 5 5× is used to create a smooth surface. The function isosurface of MATLAB has been used

to render the surface. The 3-D reconstructed model of the object has been shown in Fig. 5.

Fig. 5. Reconstructed 3-D model

III. FEATURE EXTRACTION AND REPRESENTATION A. Edge Detection in Range Image

Instead of applying segmentation process directly on the discrete point domain, the Laplacian of a Gaussian (LoG) edge detector has been applied on the interpolated depth coordinates of the mesh surface of the 3-D model. The edge features are extracted based on the discontinuity in depth. The detected edge points using LoG edge detection algorithm and its 3-D plot are shown in Fig. 6 (a) and (b) respectively.

(a) (b)

Fig. 6 (a). Edge map (b) 3-D plot B. Edge Detection in Intensity Image

For the intensity image, edge features are extracted based on discontinuity in intensity value at edges. Circular, rectangular and triangular shapes on the object surface are painted with black color to minimize the effect of shadows. The notch shape is not colored to observe the effect of shadow in the edge map. The object is placed on a horizontal and smooth platform. The illumination light is turned on and an intensity image is acquired with the same CCD camera interfaced with Silicon Graphics machine. The captured image is converted into gray scale image and Canny edge detection algorithm is applied to the converted image. The intensity image and the processed image have been shown in Fig. 7 (a) and (b) respectively

(a) (b) Fig. 7 (a). Intensity image (b) Edge map

Page 4: [IEEE 2009 IEEE Symposium on Computational Intelligence for Image Processing (CIIP) - Nashville, TN, USA (2009.03.30-2009.04.2)] 2009 IEEE Symposium on Computational Intelligence for

C. Shape Signatures In the present work, the primary focus is on external shape

characteristic of the image and its constituents. So we have chosen shape signatures for feature representation. The original signature has been constructed by measuring and plotting the distance from the centroid of the shape to the boundary at all discrete positions along the digitized boundary as a function of angle [16]. Signatures along with the peaks have been obtained for all segmented regions of both range and intensity images. From the shape signatures, ‘derived’ signatures are constructed on the basis of preserving important shape features emphasized by the original version of the signature. This is achieved by considering prominent peaks in the signature plot as important features [17]. Prominent peaks are obtained with values above the mean of the peaks. The segmented circle boundary, shape signature with peaks and prominent peaks of the range image have been shown in Fig. 8 (a), (b) and (c) respectively where as segmented rectangle boundary, its shape signature with peaks and prominent peaks of the intensity image have been shown in Fig. 9 (a), (b) and (c) respectively.

(a) (b) (c) Fig. 8 (a). Circle boundary (b) Shape signature (c) Prominent peaks

(a) (b) (c) Fig. 9 (a). Rectangle boundary (b) Shape signature (c) Prominent peaks D. Corner Point Detection

In the present case, those points are considered to be corner points which are at a significant distance from the centroid. Shape signatures have described the shape based on distance form the centroid. Corner points have been detected out of the peak values from the shape signature. Prominent peaks corresponding to corners of the various shapes have been obtained. Fig. 10 (a) and (b) shows the corner points detected from the prominent peak points of the circle boundary in range image and the rectangle boundary in intensity image respectively. The same process is repeated for all the segmented regions of range and intensity image. Fig. 11 (a) and (b) shows all the detected corner points in range image and intensity image respectively.

(a) (b) Fig. 10. Detected corner points in (a) circle boundary (b) rectangle boundary

(a) (b) Fig. 11. Corner points in (a) range image (b) intensity image IV. MAPPING BY PERSPECTIVE TRANSFORMATION

Perspective transformation plays a central role in image processing because they provide an approximation to the manner in which an image is formed by viewing a 3-D world [18]. For the fusion in 2-D space, 3-D edge points in range image have been projected onto 2-D plane by perspective transformation.

With the help of experimentally obtained scene-image coordinate pairs of the control points on the object, the perspective transformation matrix has been obtained as

1

2

3 3 1 3 2 3 3 3 4

4

1 0 0 3 5 .0 6 5 70 0 .8 6 6 0 0 .5 3 0 8 .8 1 6 0

0 0 .0 0 0 4 0 .0 0 4 1 0 .6 8 8 0 1

h

h

h

h

c Xc Yc a a a a Zc

−⎛ ⎞ ⎛ ⎞ ⎛ ⎞⎜ ⎟ ⎜ ⎟ ⎜ ⎟−⎜ ⎟ ⎜ ⎟ ⎜ ⎟=⎜ ⎟ ⎜ ⎟ ⎜ ⎟⎜ ⎟ ⎜ ⎟ ⎜ ⎟⎜ ⎟ −⎝ ⎠ ⎝ ⎠⎝ ⎠

(4)

where 1 2 3 4( , , , )h h h hc c c c are camera coordinates in homogeneous representation and ( , , )X Y Z are world

coordinates. The unknown coefficients 31a , 32a , 33a and

34a are not computed as these are related to z .The camera coordinates in Cartesian form are 1

4

h

h

cxc

= and 2

4

h

h

cyc

= (5)

For any world point ( , , )X Y Z on the object, the image coordinates ( , )x y can now be computed with the help of (4) and (5). The image coordinates are computed for all the points in the 3-D edge map of the object obtained using LoG edge detector and the projected 2-D edge map has been shown in Fig. 12 (a). The corner points detected using shape signatures have also been perspectively projected onto 2-D plane and the projected points have been shown in Fig. 12 (b).

Page 5: [IEEE 2009 IEEE Symposium on Computational Intelligence for Image Processing (CIIP) - Nashville, TN, USA (2009.03.30-2009.04.2)] 2009 IEEE Symposium on Computational Intelligence for

(a) (b) Fig. 12 (a). Projected 2-D edge map (b) Projected corner points

V. AUTOMATIC FUSION OF EDGE MAPS

The literature survey shows that only a few research works have been carried out in the area of fusion between range and intensity edge maps. Nadabar et al. [19] have proposed a scheme for detection and labeling of edges from a pair of registered intensity and range images in the Bayesian framework. The weakness associated with using Bayesian approach for fusion is that when considering new evidence as a proposition of old evidence, there is no symmetry between the bodies of evidence within a decision of a group. Another issue is ignorance, which is better known as the lack of priors. Abidi et al. [20] have developed an approach for fusion of range and intensity edge maps using fuzzy logic to extract a complete edge map of an arbitrary scene. The limitation of this approach is that fuzzification of real data, selection of the shape of membership functions and the generation of rule bases has no unique solutions. Chang et al. [21] have proposed a segmentation algorithm based on fusion of range and intensity images using robust trimmed methods. The algorithm takes more time to estimate surface parameters. Also, there is no information regarding shadow edges. In the present work, the concept of automatic fusion of edge maps from both the modalities has been introduced and accomplished using affine transformation followed by ICP algorithm. A. Fusion of Edge Maps using Affine Transformation

In general, the affine transformation matrix can be written as

11 12 13

21 22 23

31 32 331 1

x a a a xy a a a y

a a a

′⎛ ⎞ ⎛ ⎞⎛ ⎞⎜ ⎟ ⎜ ⎟⎜ ⎟′ =⎜ ⎟ ⎜ ⎟⎜ ⎟⎜ ⎟ ⎜ ⎟⎜ ⎟⎝ ⎠ ⎝ ⎠⎝ ⎠

where 11 12 13

21 22 23

31 32 33

a a aA a a a

a a a

⎛ ⎞⎜ ⎟= ⎜ ⎟⎜ ⎟⎝ ⎠

(6)

In order to fuse two data sets by affine transformation, a set of control points must be identified that can be located on both the maps. Since the general affine transformation is defined by six constants, at least three control points must be identified. Generally, more number of control points is identified for optimal result.

The perspective projected corner points from the range image edge map have been shown in Fig. 12 (b) where as the detected corner points in the intensity image edge map have been shown in Fig. 11 (b). The steps for establishing automatic correspondence between the corner points in both the edge maps as well as identifying the control points are as follows:

Step 1: For recovering correspondence between the two modalities, the calculation requires the selection of consistent origin in the image which has been defined as the centroid of the detected corner points. All the corner points have been shifted with respect to the centroids in both the images.

Step 2: Orientation of all the corner points in both the images have been determined with respect to the centroids in terms of Euclidean distance and angle by simple geometric measures.

Step 3: Differences between the Euclidean lengths and angles for every point pair in two images have been determined. This has resulted in matrices both for length and angle differences which are utilized for obtaining correspondence between corner points.

Step 4: A matching array of corresponding pairs has been determined from the matrices obtained above. For a particular corner point in the projected range image, few corner points in the intensity image are selected based on a threshold value for angle differences. Out of the selected points, the one with minimum of length difference has been considered as the corresponding point of the range image corner point.

Step 5: The above step has been repeated for all corner points in the projected range image. The corner points in the range image for which corresponding points are identified in the intensity image satisfying the angle and length criterion have been saved as the control points. The coordinates of the identified fifteen control points in both the images have been shown in Table 1.

A least-square technique has been used to determine the six affine transformation parameters from fifteen matched control points. For the fifteen number of control points, (6) can be written as

1 2 1511 12 13 1 2 15

21 22 23 1 2 15 1 2 15

31 32 33

............... .....

1 1 ..... 1 1 1 ..... 1

x x xa a a x x xa a a y y y y y ya a a

⎛ ⎞′ ′ ′⎛ ⎞⎛ ⎞ ⎜ ⎟⎜ ⎟⎜ ⎟ ⎜ ⎟′ ′ ′=⎜ ⎟⎜ ⎟ ⎜ ⎟⎜ ⎟⎜ ⎟ ⎜ ⎟⎝ ⎠⎝ ⎠

⎝ ⎠

or, AX X ′= (7) The elements of X (range image) and X ′ (intensity image) can be obtained from Table 1. Simple mathematical operations on (7) yields 1( )T TA X X X X −′= (8) The affine transformation matrix A has been computed by (8) using the coordinate values of the control points in Table 1 and is given by

1.4152 0.0363 31.96640.0046 1.6523 341.7584

0 0 1A

−⎛ ⎞⎜ ⎟= −⎜ ⎟⎜ ⎟⎝ ⎠

(9)

With the help of (9), the transformation equations are obtained as follows:

1 .4 1 5 2 0 .0 3 6 3 3 1 .9 6 6 4x x y′ = − + and 0 .0046 1.6523 341.7584y x y′ = + − (10)

Page 6: [IEEE 2009 IEEE Symposium on Computational Intelligence for Image Processing (CIIP) - Nashville, TN, USA (2009.03.30-2009.04.2)] 2009 IEEE Symposium on Computational Intelligence for

TABLE 1 LOCATION OF CONTROL POINTS

Control points

Range image (pixel)

Intensity image (pixel)

x

y

x′

y′

1

23.4248

341.5663

97.9193

186.1385

2

157.9498

236.1446

255.1909

48.0266

3

143.1508

232.7981

230.2870

47.0724

4

111.7903

308.1063

187.1643

180.9019

5

48.6617

323.3889

97.9193

186.1385

6

53.5870

327.1906

96.0503

202.1571

7

68.4678

328.9386

105.8636

213.9312

8

73.9684

313.9793

127.0363

180.0273

9

64.0207

308.0958

54.5811

206.4277

10

44.9787

272.6436

93.1221

121.1272

11

76.0858

273.0538

133.0545

123.9591

12

76.6065

250.7178

103.5513

45.1361

13

149.1444

244.1411

255.1909

48.0266

14

142.1814

339.2647

187.1643

180.9019

15

150.0590

339.7346

252.9114

246.0109

Equation (10) is applied to all the points in the projected range image edge map for fusion with the intensity image edge map. Both the edge maps after fusion have been shown in Fig. 13.

Fig. 13. Fusion of edge maps using affine transformation

The corner points and subsequently control points have been identified automatically from the two modalities in this method. Few of the control points chosen are not that close a match. For this reason, the results shown in Fig. 13 have deviated more from the desired result. B. Fusion of Edge Maps using ICP Algorithm

The coarsely fused edge points by affine transformation have been subjected to fine fusion using iterative closest point algorithm. The calculation of transformation parameters has been made using the singular-value decomposition (SVD) method [22]. Both the edge maps after fusion using ICP algorithm are shown in Fig. 14.

Fig. 14. Fusion of edge maps using ICP algorithm

The difficulties with manual fusion which are tedious,

labour-intensive and repetitive have been overcome by the automated algorithm for fusion. The 3-D information can be obtained from the range image edge map. It can be seen that there are no edges due to shadow in range image edge map where as edges by shadow exist in the intensity edge map. Fusion of both the edge maps can detect edges formed by shadows. Texture edges can also be obtained from the intensity edge map. Fusion of both the edge maps provides more reliable information about the important discontinuities of the object.

VI. CONCLUSION

In this paper, we have described the method of 3-D reconstruction as well as automatic fusion of edge maps from two different modalities of an object. The registration of range images takes advantages of both feature and surface matching techniques. The integration algorithm has eliminated the redundant data points considerably resulting in reduced memory and time requirement. A median filter of window size 5 5× is used to create a smooth surface of the object. Use of a larger size window for median filtering can give more smoothness but at the cost of reduced accuracy in dimensions.

In this work, feature extraction using LoG and Canny edge detectors require least human intervention. A technique involving shape signatures has been implemented for the detection of corner points in both the edge maps. We have

Page 7: [IEEE 2009 IEEE Symposium on Computational Intelligence for Image Processing (CIIP) - Nashville, TN, USA (2009.03.30-2009.04.2)] 2009 IEEE Symposium on Computational Intelligence for

proposed a novel method for the correspondence and detection of control points by finding the geometric orientation of corner points based on length and angle with respect to the centroid. The concept of automatic fusion of edge maps extracted from the 3-D model and intensity image of the object using affine transformation followed by ICP algorithm has been introduced. The proposed algorithm for automatic fusion has eliminated the problems associated with manual fusion. Fusion of the edge maps from both the modalities provides complementary information, increases the degree of confidence regarding the presence or absence of the edge feature and supplies information about shadows as well as intensity variations. This approach can find industrial application for the detection of defects in the final product. The affine mapping function is appropriate for data with flat topography. For data with nonlinear and local geometric distortions, we are investing further for the algorithms to produce better interpolation results.

REFERENCES [1] J. Park, and G. N. Desouza, “3-D modeling of real-world objects using

range and intensity images,” in Innovations in Machine Intelligence and Robot Perception, S. Pattnaik, L. C. Jain, G. Tzafestas, and V. Bannore, Eds. Springer-Verlag, pp. 205-266, 2005.

[2] Z. Zhang, X. Peng, W. Shi, and X. Hu, “A survey of surface reconstruction from multiple range images,” in Proc. 2nd Asia-Pacific Conf. Systems Integrity and Maintenance, Nanjing, 2000, pp. 519-523.

[3] C. Fruh, and A. Zakhor, “3-D model generation for cities using aerial photographs and ground level laser scans,” in Proc. 2001 IEEE Computer Society Conf. Computer Vision and Pattern Recognition, 2001, pp. II31-II38.

[4] A. H. Eid, S. S. Rashad, and A. A. Farag, “A general-purpose platform for 3-D reconstruction from sequence of images,” in Proc. 5th Int. Conf. Information Fusion, 2002, vol. 1, pp. 425-431.

[5] G. Tognola, M. Parazzini, P. Ravazzani, F. Grandori, and C. Svelto, “3-D acquisition and quantitative measurements of anatomical parts by optical scanning and image reconstruction from unorganized range data,” IEEE Trans. Instrumentation and Measurement, vol. 52, no. 5, pp. 1665-1673, Oct. 2003.

[6] I. Abuhadrous, F. Nashashibi, C. Laurgeau, and F. Goulette, “Onboard real-time system for 3-D urban environment reconstruction,” in Proc. IEEE Intelligent Vehicle Symposium, 2003, pp. 479-483.

[7] M. Demeyere, and C. Eugene, “Measurement of cylindrical objects by laser telemetry: a generalization to a randomly tilted cylinder,” IEEE Trans. Instrumentation and Measurement, vol. 53, no. 2, pp. 566-570, Apr. 2004.

[8] B. M. Planitz, A.J. Maeder, and J. A. Williams, “The correspondence framework for 3-D surface matching algorithms,” Computer Vision and Image Understanding, vol. 97, no. 3, pp. 347-383, Mar. 2005.

[9] P. J. Besl, and N. D. McKay, “A method for registration of 3-D shapes,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 14, no. 2, pp. 239-256, Feb. 1992.

[10] A. A. Goshtasby, and S. Nikolov, “Image fusion: advances in the state of the art,” Information Fusion, vol. 8, no. 2, pp. 114-118, Apr. 2007.

[11] Z. Liu, D. S. Forsyth, and R. Laganiere, “A feature-based metric for the quantitative evaluation of pixel-level image fusion,” Computer Vision and Image Understanding, vol. 109, no. 1, pp. 56-68, Jan. 2008.

[12] S. Zheng, W. Z. Shi, J. Liu, G. X. Zhu, and J. W. Tian, “Multisource image fusion method using support value transform,” IEEE Trans. Image Processing, vol. 16, no. 7, pp. 1831-1839, July 2007.

[13] F. Nencini, A. Garzelli, S. Baronti, and L. Alparone, “Remote sensing image fusion using the curvelet transform,” Information Fusion, vol. 8, no. 2, pp. 143-156, Apr. 2007.

[14] H. Wang, “A new multiwavelet-based approach to image fusion,” J. Math. Imaging and Vision, vol. 21, no. 2, pp. 177-191, 2004.

[15] B. K. P. Horn, “Closed-form solution of absolute orientation using unit quaternions,” J. Optical Society of America A, vol. 4, no. 4, pp. 629-642, Apr. 1987.

[16] R. C. Gonzalez, R. E. Woods, and S. L. Eddins, Digital Image Processing using MATLAB, Delhi: Pearson Education, 2004.

[17] J .Maeder, “Feature-based hierarchical shape matching,” in Proc. 2nd Australian and New Zealand Conf. Intelligent Information Systems, Brisbane, 1994, pp. 342-346.

[18] R. C. Gonzalez, and R. E. Woods, Digital Image Processing, Addison-Wesley Publishing Company, 1999.

[19] S. G. Nadabar, and A. K. Jain, “Fusion of range and intensity images on a connection machine (CM-2),” Pattern Recognition, vol. 28, no. 1, pp. 11-26, 1995.

[20] M. A. Abidi, M. Abdulghafour, and T. Chandra, “Fusion of visual and range features using fuzzy logic,” Control Engg. Practice, vol. 2, no. 5, pp. 833-847, Oct. 1994.

[21] I. S. Chang, and R. Park, “Segmentation based on fusion of range and intensity images using robust trimmed methods,” Pattern Recognition, vol. 34, pp. 1951-1962, 2001.

[22] R. Madhavan, and E. Messina, “Iterative registration of 3-D LADAR data for autonomous navigation,” in Proc. IEEE Intelligent Vehicles Symposium, Columbus, 2003, pp. 186-191.