edge linking by using causal neighborhood windo · edge element and (b) the concept of causal...

Pattern Recognition Letters 13 (1992)647-656 North-Holland

September 1992

Edge linking by using causal neighborhood window

Ming Xie

INRIA Sophia Antipolis. 2004. Route des Lucioles. 06561 Valbonne. France

Received ! July 1991 Revised 22 January 1992

Abstract

Xie, M., Edge linking by using causal neighborhood window, Pattern Recognition Letters 13 (1992) 647-656.

A novel edge linking algorithm will be presented in this paper. We shall make use of two concepts: (a) the concept of horizontal edge element and (b) the concept of causal ne,~hborhood window for realizing the edge linking. On the basis of these two concepts, the proposed algorithm will carry out (in one pass on the edge map) the edge linking by performing two operations which are the contour chain creation and the contour cha#! extension. For the illustration of the applicability of our algorithm, some experimental results with real images will be shown.

Kt:rwords. Edge linking.

I. Introduction

Computer vision aims at the recovery of visual objects from the visual sensors' data such as images. In general, an operational vision system leading to this goal must incorporate a set of hierarchical processings, i.e., the low-level processing for feature extraction and the high-level processing for image interpretation or understanding. It is well known that one of the important tasks in low-level processing is image segmentation. Up to now, many investigations have been devoted to either region-based or boundary-based segmentation techniques [1], [2]. Due to the fact that an edge element may belong to an object boundary and that a region may be charcterized by its sur- rounding boundary, we can say that edge detection and region growing are the two complementary

Correspondence to: Ming Xie, PASTIS Project, INRIA Sophia, 2004, Route des Lucioles, 06561 Valbonne, France.

aspects of image segmentation. Especially, different techniques exist in literature for converting a region-based representation of images into a boundary-based representation (see [7]).

In [2, p. 119], Ballard and Brown state:

An image of only disconnected edge elements is relatively featureless; additional processing must be done to group edge elements into struc- tures better suited to the process of interpretation.

Thus the segmentation of an image into edge pixels is just a necesarry but not a sufficient step prior to the description of an edge map. Another necessary step consists of the grouping of edge pixels into some coherent sets suitable for further treatment. For this reason, edge linking or contour tracing has become an important topic in shape analysis of visual patterns. The problem of edge linking is generally formulated as the grouping of edge pixels

0167-8655/92/$05.00 © 1992 -- Elsevier Science Publishers B.V. All rights reserved 647

Volume 13, Number 9 PATTERN RECOGNITION LETTERS September 1992

based on the criterion of proximity or similarity. In [3], an algorithm for converting region boundaries into chain-coded line segments was described. Assuming that the edges have single-pixel width and that there are no gaps on the boundaries, a 3 × 3 neighborhood window is used to group the connected edge pixels that form a straight line segment. The interesting aspect of this algorithm is that it identifies the junctions between chain-coded line segments. In [5], an algorithm of edge linking was presented, which also uses a 3 x 3 neighborhood window due to the same prerequisite assumption about single-pixel width of edges and no-gap boundaries. After defining a system of finite state automata, the grouping of edge pixels into a set of contour chains is carried out in one pass on the edge map. The defined system of automata takes into account all kinds of junctions between the chains generated. In order to minimize the number of chains, a merging and a thresholding of contour chains are finally performed. In [8], an edge linking algorithm based on the criterion of similarity between edge pixels was described. In that algorithm, an edge pixel is characterized by a feature vector with an associated probability distribution (i.e., covariance matrix). An ellipsoidal clustering technique is used to generate a decision tree that allows the grouping of edge pixels into a set of contour chains.

in this paper, we shall present an edge linking algorithm that uses a causal neighborhood window. Our main motivation for this investigation is to develop a general-purpose edge linking algorithm without requiring the usual assumption about single-pixel width of edges and no-gap boundaries, because in a large range of applications it is difficult to obtain edge maps which are always per- fect. For this reason, we shall develop a new edge linking strategy based on the use of two concepts. The first one is the concept of horizontal edge element (which is defined as a set of adjacent edge pixels aligned in the same image line) for re- organizing the edge data. So, an edge map will previously be transformed into a set of horizontal edge elements a~ ~ the task of edge linking becomes the grouping of ~orizontal edge elements into a set of contour chains. The use of horizontal edge elements in the edge linking process will allow the

edge to have different width without influencing the algorithm performance. The second concept is the one of causal neighborhood window for identi- fying the neighborhood relationship between the horizontal edge elements. Here, a neighborhood window being causal means that all the neighboring edge elements inside this window have already been treated and the contour chains they belong to are known. Since the size of our neighborhood windows is an input parameter which may take different values, we can actually deal with boundaries that have gaps. Based on these two concepts, our algorithm will carry out the edge linking by performing two operations which are contour chain creation and contour chain extension.

This paper is organized as follows. We first state the problem of edge linking (Section 2) and then describe the key concepts used by the algorithm (Section 3). The two operations of edge linking will be presented in Section 4. Finally, the experimental results (Section 5) and conclusions (Section 6) will be given.

2. Problem statement

From the point of view of data structure, an edge map is a matrix of edge pixels. According to a criterion of proximity, these edge pixels can be grouped into a set of so-called contour chains. For a given configuration of edge map, it is easy to notice that there are several possible definitions of the term 'contour chain' depending on the edge linking strategy. Figure 1 shows an example. If we ask how many contour chains are in this edge map, the answer will not be unique. One may reply that there are three contour chains {AC, BC, CD} which are separated by a junction point C. Another answer may be that there are two contour chains {ACD, BC}, the first being the longest one and the second being the shortst one in the edge map.

In the context of edge linking, we are of the opinion that defining the term 'contour chain' by referring to junctions is more consistent. The reason is that the contour chains defined in such a way correspond to the elementary contour chains, i.e., the combination of such contour chains may generate a new contour chain satisfying any other

648


i

Figure 1. In this example, two definitions of 'contour chain' are possible: (a) defining three contour chains {AC, BC, CD} or (b) defining two contour chains {ACD (longest), BC (shortest)}.

definitions. If we define the junctions between contour chains in the following way (the definition of the term 'edge element' used below will be given in the next section):

Definition 1. A junction consists of an edge element at which two or more contour chains inter- sect.

then we can give the definition of contour chain used by our edge linking algorithm as follows:

Definition 2. A contour chain is a set of linked edge elements, in which only its two end edge elements may constitute junctions.

Following the above definitions, C in Figure 1 is a junction, and {AC, BC, CD} are three contour chains.

Thus, so far, the problem of edge linking can be stated as the grouping of edge elements into a set of contour chains (Definition 2) separated by junctions (Definition 1).

3. Key concepts

An edge map is usually obtained by an edge detector such as the one described in [4]. The usual operations for edge extraction are image convolu- tion (or recursive filtering), no-maxima suppres- sing and edge thresholding by hysteresis. For a wide range of images, it is difficult for an edge detector to ensure that the edge maps obtained will

only contain the edges of single-pixel width and the no-gap boudaries. In order to tolerate the presence of any kinds of edges and of boundaries, our algorithm of edge linking will be implemented around two concepts: the horizontal edge element and the causal neighborhood window.

3.1. Horizontal edge element (HEE)

Like the algorithm presented in [5], we will carry out the edge linking itself in one pass on the edge map, scanning line by line from top to bottom. However, since it will be more elegant and faster to link edge elements instead of edge pixels, we will precede that step with the grouping of the adjacent edge pixels in each image line into edge elements. In this way, the computation time of edge linking will be reduced since there will be fewer edge elements than edge pixels to be linked. Moreover, the performance of the linking of such edge elements will not depend on the edges' width. For the above reasons, the first concept used by our algorithm will be the horizontal edge element which is defined as follows:

Definition 3. A horizontal edge element is a set of adjacent edge pixels aligned in the same image line.

Figure 2 illustrates an example of three horizontal edge elements which correspond to fourteen edge pixels. The linking of three horizontal edge elements can obviously be done more quickly than that of the fourteen edge pixels.

Now, considering a two-dimensional coordinate system OXY in the image plane, if we denote (Xleft 0, Y0) the left endpoint of a horizontal edge element and (Xright0,Y0) its right endpoint, then a horizontal edge element will be characterized by

(Xleft 0, Xright 0, Y0)'

3.2. Causal neighborhood window (CNW)

We want to use a proximity criterion for grouping the horizontal edge elements into a set of contour chains. And we want to do it in one pass on the edge map, i.e., in scanning the edge map line by line from top to bottom and left to right. In such a context, it is possible to have three kinds of

649


I 2 I ! I

y rill I l 211 31111 4ii I I sll I I 61111 ~II I I

I I I

I l l [ ~[ ][ ~NN I l l m l l l i l l m t l l n n l l n l l l I l l l l U [ ~[ ~[ i n m m n lmmmmmmc :c ;[ :[ :[ :[ )[ l l l [ l i l l ( [ l l l l l m m m m u B i m m m m m m m

mmmm mmmmmmmmmmmmmmm

Figure .~. A horizontal edge element is defined as a set of edge pixels aligned in the same image line. In this example, the fourteen edge pixels are grouped into three horizontal edge elements.

neih~,borhood windows which are: (a) the causal neighborhood window; (b) ihe semi-casual neighborhood window and (c) the non-causal neighborhood window. A similar kind of classification of neighborhood

windows has also been considered in image processing (image representation, image compression, etc.) (see [6]). In our case, a neighborhood window being causal means that all the neighboring horizontal edge elements inside it have already been treated and the contour chains they belong to are known. This property will not be true for semi. causal neighborhood windows and non-causal neighborhood windows. It is interesting to note that the use of causal neighborhood windows will simplify the edge linking operation since we need not take into account the future situations of grouping during the edge linking process. As previously mentioned, the boundaries in an edge map may have gaps; for overcoming this diffi- culty, we just need to choose appropriate values for the height and the width of the neighborhood window.

In practice and without loss of generality, if we denote Xgap the maximum gap value in X direction of boundaries and .Vgap the maximum gap value in Y direction, we will characterize a causal neighborhood window by (Xs,p, ygap ). Given a causal neighborhood window (Xgap, Ygap) positioned at a horizontal edge element (xleft 0,Xright 0, Y0), its corresponding window region in O X Y will be determined by:

650

f Xleft0--Xgap-- | ~X~Xright 0 i f Y=Yo,

Xleft 0 -- Xgap -- I ~ X ~< Xright 0 -I- Xgap -I- 1 (1)

if Y0 -Ygap - 1 <~Y<~Yo - 1.

So, the neighborhood window width is (2Xgap+ Xright 0-Xleft 0 + 3) and its height is ()'gap + 2). Figure 3 illustrates the dimensions of a causal neighborhood window.

4. Edge linking

In this section, we will present our strategy for the linking of horizontal edge elements (HEE) into a set of contour chains, based on the use of causal neighborhood windows (CNW). The principle of our algorithm can be summarized as scanning an edge map line by line (i.e., from top to bottom and left to right) during which, if a HEE is met, a CNW will first be created at it and then the linking operations will be carried out. In order to decide upon the linking operations, we first need to know how to identify the neighboring HEEs and the neighboring contour chains inside a CNW.

4.1. Neighboring HEEs and neighboring contour chains

In our algorithm, a neighbouring HEE inside a CNW is defined as follows:

Definition 4. Supposing that a causal neighbor-


~x

I l | | • . | I I l I

A Causal Neighbourhood Window

I I I I I I / I I I I I I I I l_1 ,1 , /1 I I I I I I

y~ap+l

xgap+l x.~.teftO xrigh..tO _x_~ap+l

l l l l l ! _ - I i i i i ~ .

I A horizontal edge element

Figure 3. The definition of a causal neighborhood window positioned at a horizontal edge element. The dimensions of the neighborhood window are entirely determined by (XleftO, XrightO, Xgap, ygap). For this example, Xgap = 2 and ygap= 3.

hood window (Xgap, Ygap ) has been created at a HEEj : (Xleftj,Yteftj,Yj), if any other HEEi: (Xlefti,Yleft i ,Yi) verifies the following relations:

Yj - ) ' g a p - 1 ~ Yi ~ Yj - 11 , Xleft i ~ Xright j -k Xgap -b 1 (2)

Xright i >I Xleft j - Xgap -- l

or

I Yi =Yj,

Xright i ~ Xleft j , (3)

Xright i ~> Xleft j -- Xgap -- 1,

then the H E E i is called the neighboring HEE of 8 Ej.

According to the above definition, the identification of neighboring HEEs inside a CNW positioned at the H E E j can be done by verifying all the HEEs from the image line Yj-Ygap-l to yj. If a HEE satisfies the relation (2) or (3), then it will be registered as a neighboring HEE of the HEEj.

Due to the causality of a CNW, we immediately know the contour chains that the neighboring HEEs belong to. Such contour chains that the neighboring HEEs belong to will be called the neighboring contour chains inside a CNW.

4.2. Configurations of neighboring contour chains

After the identification of the neighboring HEEs and the neighboring contour chains inside a CNW, now, we must know what are the possible con-

figurations of neighboring HEEs or neighboring contour chains inside this CNW. Depending on the possible configurations, the linking operation re- quired will be different. Since the size of a CNW may be relatively great, some of the neighboring HEEs may belong to an identical contour chain. For this reason, it will be more efficient to consider the configurations of neighboring contour chains inside a CNW. As shown by Figure 4, four typical configurations in a CNW of neighboring contour chains are possible, which are:

I. Case A: There are no contour chains inside the CNW. In this configuration, it is obvious that we must create a new contour chain which takes the current HEE scanned as its end edge element (head HEE).

2, Case B: There is only one contour chain that has its head HEE inside the CNW. In this configuration, the neighboring contour chain will be allowed to extend, i.e., taking the current HEE scanned as its new head HEE.

3. Case C: There are two (or more) contour chains with their head HEEs inside the CNW. In this configuration, a junction of contour chains is detected. Consequently, the neighboring contour chains will finish their extension and a new contour chain will be created at the current HEE scanned.

4. Case D: There is one (or more) contour chain with its head HEE outside the CNW. This configuration is more difficult to treat since it is very hard to determine if a junction is detected or not. Depending on the application context where the

651

Volume 13. Number 9 PATTERN RECOGNITION LETTERS September 1992

~ X

y

Figure 4. Four typical configurations of neighboring contour chains in a causal neighborhood window of (Xgap = i, ))gap = 1): (a) Case A where there are no contour chains inside the neighborhood window; (b) Case B where there is only one contour chain with its head HEE inside the neighborhood window; (c) Case C where there are two (or more) contour chains with their head HEEs inside the neighboorhood window; (d) Case D where there is one (or more) contour chain with its head HEE outside the neighborhood window.

junction detection may be more or less important, two solutions to this problem are possible. One consists of treating it like case C, by arbitrarily stating that a junction is detected. The second solution consists of treating it like case A. In our imple- mentation, we have adopted the second solution. So, in this case, a new contour chain will be created at the current HEE scanned.

The above cases are the four basic configurations of the neighboring contour chains inside a CNW. But combinations of them are possible. For example, the combination of case B with case D (respectively the combination of case C with case D) may exist. In practice, it is not hard to treat these combinations since we consider case D like case A. Therefore the combination of case B (resp. case C) with case D is equivalent to case B (resp. case C).

4.3. Homogeneity o f HEEs" size in a contour chain

The goal of our algorithm consists of linking the horizontal edge elements (Definition 3) to form a set of contour chains (Definition 2). The main advantage of using horizontal edge elements lies in the fact that we do not need the assumption about the single-pixel width of edges anymore. On the other hand, since a contour chain is a set of hori-

zontal edge elements (HEEs), another advantage of using HEEs in the linking process is that we can from now on control the homogeneity of the sizes of HEEs belonging to a same contour chain. It means that if needed we can force the HEEs of a contour chain to have similar size. For example, by controlling the homogeneity of HEEs' size in a contour chain, we can prevent a horizontal line segment (a long HEE) from being associated to a vertical or sloped contour chain (a set of HEEs of similar size).

In our algorithm, the contol of homogeneity of HEEs' size will be done at the moment when a contour chain should be extended to the current HEE scanned (case B). At this stage, we calculate the ratio between the current HEEs' size and the size of the head HEE of the contour chain to be extended. If this ratio is inferior to a threshold, saying ro, then this contour chain will be allowed to extend to the current HEE; otherwise, this contour chain will be forbidden to extend and a new contour chain will be created at the current HEE. An interesting aspect of the control of the homogeneity of HEEs' size is that it allows to quickly identify the horizontal line segments in an edge map.

4. 4. Edge linking algorithm

We have seen that the two operations of the edge

652


linking are the creation of contour chain and the extension of contour chain. However, for the configurations of case C and case B, we must manage the permission to extend a contour chain. In order to do this efficiently, we define two states of contour chain, which are the state OPEN and the state CLOSE. Once a contour chain is forbidden to extend, then it will be in the state CLOSE. All the contour chains of the state CLOSE will not be taken into account anymore during the edge linking process.

So far, our algorithm of edge linking can be summarized in the following way:

Step 1. Transform the matrix of edge pixels into a matrix of horizontal edge elements (Definition 3).

Step 2. Scan the matrix of horizontal edge elements (HEEs) line by line (from top to bottom and left to right). If no more HEE are met, then stop. Otherwise, create a causal neighborhood window (CNW) at the current HEE scanned.

Step 3. Identify the neighboring HEEs (Definition 4) and then the corresponding neighboring contour chains of the state OPEN.

Step 4. Carry out the edge linking operation (i.e., the contour chain creation or the contour chain extension) depending on the configurations of the neighboring contour chains and the control of the homogeneity of I-IEEs' size. Then go to Step 2 and continue the scanning process.

5. Experimental results

We have tested our algorithm with various kinds of real images. Here, we want to show two examples. In our algorithm, there ale three working parameters, which are (Xgap, y gap, r0). For the two examples shown below, we have fixed these parameters to constant values as follows:

t ~'Xgap -" l ,

)'gap = 1, (4)

, / 'o=4"1.

Our algorithm has been implemented in C++ on a Sun4 workstation. For the purpose of showing the

usefulness of our algorithm, we would like to specifically present the 2D line segments estimated from the contour chains obtained by our algorithm. The estimation of 2D line segments is based on the operation of contour chain splitting which uses the criterion of distance to line (see [9]).

Experiment 1. Figure 5 gives the first example. Figure 5a shows the real image of a girl's face picture. We have used the edge detection algorithm presented in [4] to extract the edge pixels. Figure 5b shows the edge map extracted, in which there are 19663 edge pixels that have been transformed into 13218 horizontal edge elements. Figure 5c illustrates the 685 contour chains obtained by our algorithm. The computation time is about 27.93 s. The small circle 'o' in the figure indicates the end HEEs of the contour chains. In Figure 5d, we display the 2D line segment, s estimated from the contour chains obtained by our algorithm (there are 697 line segments).

Experiment 2. Figure 6 gives the second example. In Figure 6a, we show the real i~aage of an indoor scene. The edge map extracted is presented in Figure 6b, in which there are 13864 edge pixels that have been transformed into 7066 horizontal edge elements. The 312 contour chains obtained by our algorithm are illustrated in Figure 6c, in which the small circle 'o' indicates the end HEEs of the contour chains. The computation time for this example is about 7.75 s. In Figure 6d, we show the 2D line segments estimated from the contour chains obtained by our algorithm (there are 383 line segments).

Table 1 displays the data concerning the above experimentations.

Table 1 The data and experimental results

Example 1 Example 2

Number of edge pixels 19663 13864 Number of HEEs 13218 7066 Number of contour chains 685 312 Number of line segments 697 383 CPU time (s) 27.93 7.75

653


I l l I

/

I

// /.. 74~,?:~

', \\ /

)7 I, 'jv, ,""

Figure 5. The first example: (a) the image of a girl's face picture of the size of 512 x 512; (b) the edge map; (c) the set of contour chains obtained, in which the small circle ,o, indicates the end HEEs of the contour chains; (d) the set of 2D line segments estimated from

the contour chains.

6. Conclusion

in this paper, we have presented a novel algorithm of edge linking. We have proposed the use of two concepts which are the horizontal edge element and the causal neighborhood window, to develop a general-purpose algorithm of edge linking

without requiring any assumptions about the quality of edge maps. On the basis of the two concepts, a strategy of edge linking has been de- veloped. This strategy involves two operations (i.e., the contour chain creation and the contour chain extension) and allows the detection of junctions between contour chains. Moreover, due to

654


' .m / t <. -

I

: ."": i ' ]

@ . i /-r. I

\ . , , , . , \ } \ .. : t % I I - " , - - 7 / " 7 ".

I I I I

Figure 6. The second example: (a) the image of an indoor scene of the size of 512 x 512; (b) the edge map; (c) the set of contour chains obtained, in which the small circle 'o' indicates the end HEEs of the contour chains; (d) the set of 2D line segments estimated from

the contour chains.

the use of horizontal edge elements in the edge

linking process, we can from now on control the

homogeneity of the sizes of HEEs belonging to a

same contour chain. In our current implementa-

tion of the algorithm, we have used fixed values

(Xgap, Ygap ) of a causal neighborhood window.

Theoretically, these two parameters could be

chosen dynamically during the edge linking pro-

cess. This may be an interesting issue for dealing

with edge maps having different kinds of gaps on

boundaries.

655


References

[1] Aggarwal, J.K., R.O. Duda and A. Rosenfeld (1977). Com- puter Methods in Image Analysis. IEEE Press, New York.

[2] Ballard, D.H. and C. Brown (1982). Computer Vision. Prentice-Hall, Englewood Cliffs, NJ.

[3] Chakravarty, i. (1981). A single-pass, chain generating algorithm for region boundaries. Computer Graphics and Image Processing 15, 182-193.

[4] Deriche, R. (1987). Using Canny's criteria to derive a recur- sively implemented optimal edge detector. Int. J. Computer Vision.

[5] Giraudon, G. (1987). A real time parallel edge following in single pass. IEEE Workshop on Computer Vision, 228-230.

[6] Jain, A.K. (1989). Fundamentals of Digital Image Proces- sing. Prentice-Hall, Englewood Cliffs, NJ.

[7] Liow, Y.T. (1991). A contour tracing algorithm that pre- serves common boundaries between regions. CVGIP: Image Understanding 53(3), 313-321.

[8] Ramesh, V. and R.M. Haralick (1989). Edge linking by ellipsoidal clustering. SPIE Intelligent Robots and Com- puter Vision VIII: Algorithms and Techniques ! 192, 147- 158.

[9] Xie, M. and P. Rives (1988). From edges to line segments: a robust algorithm. Research Report No. 429, IRISA (in French).

656

edge linking by using causal neighborhood windo · edge element and (b) the concept of causal...

Documents