Novel Approach for Recognizing Bridges over Water in Large Remote Sensing Images

Aijun Chen College of Mechanical and Electrical Engineering, Northeast Forestry University

Harbin 150040, China jschenaj@163.com

Abstract

There have been many approaches to recognize bridges, but most of them can not be effectively used on real-time occasions. In this paper, a novel approach is presented to rapidly recognize bridges over water in large remote sensing imagery. First, the rough river regions are quickly detected based on fuzzy pattern classification and clustering analysis. Next, with the knowledge of relationship between a bridge and the river under it, regions of interest are obtained. Then, in a ROI sub-image, several features of a candidate bridge and its relative water regions are extracted. At last, three rules are applied to judge whether the candidate bridge is a real one or not. Experiments on real images were conducted, and the results demonstrate that the approach is feasible and effective. 1. Introduction

Extraction of man-made objects such as bridges, buildings and airports, from remote sensing imagery is very important to update geographical databases and helpful to quickly assess the extent of damages in the case of natural disasters such as flooding or earthquake. Especially, bridge detection in remote sensing imagery has been a topic of considerable interest. Much work has been done in this area and a lot of algorithms have been presented.

Hou et al. [1] presented an algorithm for automatic segmentation and recognition of bridges in high-resolution SAR images according to the idea of detecting the targets along the edges of the image.

Cheng et al. [2] proposed an approach for detecting bridges by combined wavelet support machine (WSVM) with knowledge of bridges. The feature of river domain was extracted by analyzing bridges and their backgrounds in SAR imagery. Next, WSVM was used to make classification model by training example

data for segmenting river region. The last direction energy function was used as the rule for identifying abridge in the binary image of river class. In [3], a method is presented to recognize bridges based on region growing from high resolution optical images. Some other examples can be found in [4-7].

The above methods did gain some success, whereas they are time-consuming when used in large imagery. In order to save executive time and be used for real-time applications, a new method is proposed that rapidly recognized bridges over water in high resolution remote sensing imagery with large size and complex background in the next section. 2. The proposed method

A bridge over water is a structure spanning a river between two land masses. No river, no bridge over water. In this paper, the first step work is to detect whether there are rivers or not. If so, getting ROIs (region of interest) in which real bridges may locate is followed. Then the bridge detection and recognition are implemented in the ROI image, not in the original large image. The detailed work on bridge recognition is described as follows.

2.1. River region detection

In high resolution remote sensing imagery, a river

has got distinctly spectral and geometric features, namely, its gray is smaller and distributed more equably than that of its background, and it geometrically looks like a long strip. With the characteristics, a river can be easily detected.

In order to speed up the detection, a moving window is adopted to traverse the original image iteratively and thus the large image is divided into small sub-images. The size of the moving window is considered by experiments and the moving step length is equal to the width of the window. Each window in river regions

2008 International Symposium on Computational Intelligence and Design

978-0-7695-3311-7/08 $25.00 © 2008 IEEE

DOI 10.1109/ISCID.2008.82

47

2008 International Symposium on Computational Intelligence and Design

978-0-7695-3311-7/08 $25.00 © 2008 IEEE

DOI 10.1109/ISCID.2008.82

47

should be of small gray average and small gray variance according to the spectral character of a river. Therefore, a fuzzy classifier [8] is adopted to sort out the sub-images belonging to river regions. For a sub-image with the top-left pixel f(x, y), a feature vector u(x, y) can be erected with its gray average u1(x, y) and gray variance u2(x, y), namely

),(),( 21 uuyx =u (1) Some sub-images with the same size as that of the

moving window are manually obtained from the river regions and non-river regions in the original large image, respectively. They are called samples. Let a1 = (a11, a12) be the average feature vector of river region samples, a2 = (a21, a22) be that of non-river region samples, and u = (u1, u2) is the feature vector of a sub-image to be classified, then the membership functions A1(u) that the sub-image belongs to river regions and A2(u) that the sub-image belongs to non-river regions are respectively calculated by

DdA /),(1)( 111 auu −= (2) DdA /),(1)( 222 auu −= (3)

where ),(),( 2211 auau ddD += , di(u,ai) is the fuzzy Hamming distance between u and ai, defined as

∑=

−=2

121),(

jijjii auad u 2,1=i (4)

After the membership function of each sub-image is constructed, the sub-image can be classified to river areas or non-river areas by the rule of integrating maximum membership and threshold [8]. That is

⎩⎨⎧ ≥>

=else

AAAjig

0)(,)()(255

),( 121 λuuu (5)

1;1 −+≤≤−+≤≤ nyjymxix where g is the segmented large image and λ is a threshold which is determined through experiments.

All sub-images classified as river regions are grouped into regions by a nearest neighbor clustering algorithm [9]. However, the grouped regions might contain lakes, whose spectral features are similar to those of rivers. River regions should be distinguished from lake regions to improve the detection efficiency. The posture ratio c is introduced to achieve the aim.

LWc /= (6) where W and L denote the width and the length of the minimum enclosing rectangle of a grouped region, respectively. Because a river region is like a long strip in high resolution remote sensing imagery, if the posture ratio of a grouped region is larger than a given threshold Tr, it must not be a river and will not be considered in the algorithm. 2.2. Obtain ROIs

Due to the fact that bridges over water are usually above rivers and that a bridge locates between two neighbor river regions, itt is enough to detect and recognize bridges over water only in the near area of the gap of two neighbor river regions avoiding taking more time to operate in the whole large image. Therefore, some ROIs are obtained according to the detected river regions.

A ROI is obtained as illustrated in Figure 1. R1 and R2 are two neighbor river regions and P1, P2, P3 and P4 are their 4 end points. If there is a bridge between R1 and R2, it must locate in the rectangle area whose four side segments pass through P1, P2, P3 and P4, respectively. In order to make full use of the relationship between a bridge and its connected river regions, the area is extended outwards by a giver distance d to get the final ROI whose four vertexes are denoted by P'1, P'2, P'3 and P'4. The final ROI is used to get a ROI sub-image from the original large image f, and the following work will be done in each sub-image.

Figure 1. Illustration of window selection of ROI 2.3. Bridge recognition

The most distinct character for a bridge is that there are two parallel segments on its both sides. So there are parallel segments in a ROI sub-image, maybe there is a bridge in it. Owing to the fact that, in high resolution remote sensing imagery, the segments on both sides of a bridge are actually parts of contours of river regions locating at both sides of the bridge. In this paper, parallel segments are not directly detected by extracting lines in the whole ROI sub-image, but obtained through the following steps.

Step 1. Water region segmentation. The river regions in above paragraphs are rough. To

improve the precision, a refined segmentation is needed. A ROI sub-image is small and its background is simple that the scene is composed of the bridge, the land and the waters. In a ROI sub-image, the waters can be obviously distinguished from the bridge and the land. The binary image may be got by

P1

P4 P2

P3

P'1 P'2

P'3 P'4

R1

R2

d

d

d

d

4848

⎩⎨⎧

>≤

=TyxsTyxs

yxb),(255),(0

),( (7)

where s(x, y) represents the pixel gray of a original ROI sub-image, b(x, y) is the pixel gray of the binary image, and T is a threshold of image segmentation.

Owing to the effects of disturbances, in the original ROI sub-image, there may be some dark regions among the land area, whose gray are quite close to that of the waters, these regions may been wrongly marked as waters in the binary image. Fortunately, these regions are usually smaller than water regions, so they can be eliminated after an area threshold is set.

Step 2. Obtain edge points of a candidate bridge. A fast algorithm for extracting contours in a binary

image [10] is adopted to get contours in a binary ROI sub-image. As mentioned in the above paragraphs, if a bridge locates in the ROI sub-image, the side segments are parts of the contours of the waters. That is, in Figure 2, R1 and R2 are two water regions, then the pixels belonging to the contour of R1 between P1 and P2 are edge points of one segment of the candidate image and the pixels between P3 and P4 are edge points of another segment. The two groups of pixels are fitted into two segments by a least squares method and some parameters such as length, direction and position, of the two segments can be acquired. At the same time, other pixels from the contours are considered as bank edge points which are used to estimate the direction of the river locating in the ROI sub-image.

Figure 2. Contours of two river regions If such two segments are detected in a ROI sub-

image, there is a candidate bridge. However, it is arbitrary to claim that there must be a bridge. Several rules have to be used to verify whether the candidate bridge is a real one.

For the sake of illustration, suppose β is the direction of a river, and the detected segments are denoted by L1 and L2, their lengths are d1 and d2, 1α and 2α are their directions, respectively. Then, 3 rules are listed as follows.

Rule 1. A bridge should have a couple of parallel segments and their lengths are approximately equal. That is

dTdddd >

),max(),min(

21

21 (8)

121 θαα <− (9)

where Td is a distance ratio threshold small than 1, 1θ is a degree threshold.

Rule 2. The angle between a bridge and the river under it is relatively large. That can be defined as

2θβα >− (10)

where 2θ is a degree threshold. Rule 3. The real water regions should be two parts

that are divided by a bridge, and the grays of these two water regions are similar and they are lower than that of the bridge. That can be illustrated in Figure 3. Let L1 and L2 are the detected edge segments of the candidate bridge. Select a window area locating in the middle between the two segments and the average gray Val0 of pixels in the window from the original ROI sub-image is calculated. Then move the window respectively until it locates outside L1 and L2, thus two windows W1 and W2 are obtained and the average grays Val1 and Val2 can be calculated in the two windows. If the grays are satisfied with the inequations (11) and (12), the candidate bridge might be a real one.

Figure3. Illustration of Rule 3

0201 , ValValValVal << (11)

vTValValValVal >

),max(),min(

21

21 (12)

where Tv is a gray ratio threshold. Only when all of the extracted features including

gray and structure are satisfied with the inequations from (8) to (12), is the candidate bridge recognized as a real one. 3. Experimental results and analysis

To validate our approach, experiments are carried out on a 2.60GHz workstation with Windows XP operating system and 2G memory using Visual C++

R1

R2

P1

P4 P2

P3

L1

L2 W0

W1

W2

4949

6.0 program language and they are performed on a set of large (up to 10000×10000 pixels), gray-level, high resolution satellite images (Ikonos-2 images at 1m2 per pixel) containing bridges, rivers, roads, lands , forests and buildings.

One of the images is shown in Figure 4 including 13 bridges over water. The image with sub-images having been classified as river regions or non-river regions is demonstrated in Figure 5. Here the size of a moving window is 9×9. With the parameter Tr = 0.35, the river regions are detected, and 17 ROIs are obtained as shown in Figure 6. In Figure 7, 3 of 17 ROI sub-images from Figure 4 are illustrated which are of size 162×127, 171×136 and 180×118 pixels, respectively. Other parameters are set as follows: Td=0.90, 1θ =5°,

2θ =70°and Tv =0.93. At last, 13 real bridges over water are recognized in Figure 4, and Table 1 presents the parameters of three recognized bridges from the ROI sub-images in Figure 7.

In our experiments, 9 images are tested with the proposed approach and the results are illustrated in Table 2. In the 9 images, there are 77 bridges over water, and 69 of them are correctly recognized, however, 8 lost and 13 false alarms occurred. Thus, the correct recognition rate is 89.61%. Besides, the average executive time is no more than 7s.

Figure 7 Sub-images of 3 ROIs from Figure 6

Table 1 Parameters of recognized bridges Bridge

ID Location (pixel)

Length (pixel)

Width (pixel)

Orientation (degree)

1 (2744,3543) 84.91 32.81 71.37 2 (2916,3911) 82.27 45.12 67.19 3 (3097,4596) 111.50 34.18 89.54

Figure 4 A high resolution remote sensing image including bridges

Figure 5 Sub-images of river obtained by fuzzy classification

Figure 6 Extacted ROIs possibly including bridges

5050

Table 2 Test Result of bridge recognition

Image ID Image Size Number of correctly recognize bridges

Number of false alarms

Number of miss alarms Operation time(s)

1 9000×9000 4 3 1 4.156 2 10000×10000 7 1 1 4.891 3 10000×10000 22 0 2 4.719 4 10000×10000 3 4 0 12.734 5 10000×10000 1 2 0 6.250 6 9000×5650 13 0 0 6.984 7 10000×10000 3 0 0 4.719 8 10000×10000 9 0 2 5.828 9 10000×10000 7 3 2 10.640

4. Conclusions

An approach for rapidly recognizing bridge over water in large remote sensing imagery is presented. According to the gray feature and shape feature of rivers in high resolution remote sensing images, river regions are roughly segmented based on fuzzy theory and clustering analysis. Moreover, a method of selecting regions of interest (ROI) is proposed in terms of the spatial relationship between rivers and bridges. With the method, sub-images are automatically obtained, each of which represents a ROI. At last, image segmentation, contour tracking and line fitting are implemented to extract features of candidate bridges in each sub-image, and each candidate bridge is verified using priori knowledge of bridges. During the recognition, priori knowledge has been used all the time to speed up detecting and recognizing.

Some experiments on real images have shown the effectiveness of the proposed approach. However, if a river is dry or a bridge is not over a river, the proposed method will not work well for it is made to recognize bridges only over water. How to improve the proposed method to be widely used is the future research issue. 5. References [1] B. Hou, Y. Li, and L.C. Jiao, “Segmentation and

Recognition of Bridges in High Resolution SAR Images,” Proceedings of the CIE International Conference on Radar, Beijing, China, Oct. 15-18, 2001, pp. 479-482.

[2] H. Cheng, Q.Z. Yu, J.W. Tian, et al., “Detection of Bridges Based on WSVM Segmenting in SAR Image,”

Journal of Huazhong University of Science and Technology (Natural Science Edition), 2006, 34(4), pp. 52-55.

[3] G.Q. Li, D. Yin, and C.R. Xue, “Study of Road and Bridge Recognition Based on Region Growing,” Computer Engineering and Applications, 2007, 43(16), pp. 216-218.

[4] N. Lomenie, J. Barbeau, R. Trias-Sanz, et al., “Integrating Textural and Geometric Information for an Automatic Bridge Detection System,” Proceedings of IEEE International Conference on Geoscience and Remote Sensing Symposium (IGARSS'03), Toulouse, France, Jul. 21-25, 2003, Vol.6, pp. 3952-3954.

[5] F. Wu, C. Chao, H. Zhang, et al., “Knowledge-Based Bridge Recognition in High Resolution Optical Imagery,” Journal of Electronics & Information Technology, 2006, 28(4), pp. 587-591.

[6] J.J. Zhang, X.K. Yan, C.C. Shi, et al., “Recognition of Bridges over Water Based on Fracatal and Rough Sets Theory in Aerial Photography IR Image,” Proceedings of the 8th International Conference on Signal Processing (ICSP'06), Guilin, China, Nov. 16-20, 2006, Vol. 4.

[7] M. Zhu, “A New Way for the Detection and Recognition of Bridges,” Proceedings of the 8th International Conference on Signal Processing (ICSP'06), Guilin, China, Nov. 16-20, 2006, Vol. 2.

[8] S.L. Chen, J.G. Li, and X.G. Wang, Fuzzy Set Theory and Applications, Science Press, Beijing, 2005.

[9] Y.P. Xu, H.W. Deng, and Y.X. Li, “A New Nearest Neighbor Clustering Algorithm”, Journal of Southwest China Normal University (Natural Science), 2006, 31(6), pp: 114-116.

[10] B. He, T.Y. Ma, Y.J. Wang, et al, Digital image processing with Visual C++, Posts and Telecom Press, Beijing, 2001.

5151