adaptive mathematical morphology for edge linking

Information Sciences 167 (2004) 9–21

www.elsevier.com/locate/ins

Adaptive mathematical morphologyfor edge linking

Frank Y. Shih *, Shouxian Cheng

Computer Vision Laboratory, College of Computing Sciences, New Jersey Institute of Technology,

Newark, NJ 07102, USA

Received 8 October 2002; received in revised form 8 May 2003; accepted 4 July 2003

Abstract

In this paper, adaptive mathematical morphology and its application to edge linking

are presented to fill in the gaps between edge segments. Broken edges are extended along

their slope directions by using the adaptive dilation operation with the suitable sized

elliptical structuring elements. The size and orientation of the structuring element are

adjusted according to the local properties, such as slope and curvature. Post-processes

of thinning and pruning are also applied. The edge-linking operation is performed in an

iterative manner, so that the broken edges can be linked up gradually and smoothly

while the details of the object shape are preserved.

� 2003 Elsevier Inc. All rights reserved.

Keywords: Mathematical morphology; Image processing; Edge linking; Adaptive

morphology; Thinning; Pruning

1. Introduction

In the different approaches for edge or line linking, two assumptions are

usually made [1–4]: (1) True edge and line points in a scene follow somecontinuity patterns, whereas the noisy pixels do not follow any such continuity.

(2) The strengths of the true edge or line pixels are greater than those of the

* Corresponding author. Tel.: +1-973-5965654; fax: +1-973-5965777.

E-mail address: [email protected] (F.Y. Shih).

0020-0255/$ - see front matter � 2003 Elsevier Inc. All rights reserved.

doi:10.1016/j.ins.2003.07.020

mail to: [email protected]

10 F.Y. Shih, S. Cheng / Information Sciences 167 (2004) 9–21

noisy pixels. The ‘‘strength’’ of pixels is defined differently for different appli-

cations.

Nevatia [5] presented an algorithm to link edge points by fitting straight

lines. Linking direction is based on the direction of edge elements within adefined angular interval. However, portions of small curved edge segments may

be neglected. Nalwa and Pauchon [6] presented an approach based upon local

information. The defined edgels (i.e., short, linear edge elements, each char-

acterized by a direction and a position) as well as curved segments are linked

based solely on proximity and relative orientation. No global factor has been

taken into consideration, and there is little concern about noise reduction. Liu

et al. [7] proposed an edge-linking algorithm to fill gaps between edge segments.

The filling operation is performed in an iterative manner rather than a singlestep. They first connect so-called tip ends (i.e., the set of edge pixels which are

very likely to be the knots of some contour containing the one being consid-

ered) of two segments with a line segment and then try to modify the resulting

segment by straight-line fitting. In each iteration, they define the dynamic

threshold and noises are removed gradually. For this method, it is difficult to

get accurate tip ends. Another method [8] locates all of the end points of

broken edges and uses a relaxation method to link them up such that line

direction is maintained. Lines are not allowed to cross, and closer points arematched first. However, this suffers the problems if unmatched end points or

noises are present.

Mathematical morphology [9–11] has been extensively applied to image

processing and analysis. Many morphological algorithms have been proposed

in the areas of image enhancement, feature extraction, shape analysis, etc. A

recursive soft morphological filter by Shih and Puttagunta [12] has been used to

reduce noises while details are preserved. A noisy edge filtering algorithm using

the so-called space-varying opening is referred to [14]. A thinning algorithmusing mathematical morphology is developed by Jang and Chin [15]. The

algorithm is an iterative process based on the hit/miss operation. A simple

approach to edge linking is a morphological dilation of endpoints by a circle

followed by OR of the boundary image with the resulting dilated circles, and

skeletonization is finally applied [8]. This method, however, has the problems

that some of the points may be too far apart for the circles to touch, while

oppositely the circles may obscure details by touching several existing lines. An

adaptive morphology for edge linking by using circular arcs to measure cur-vature can be found in [13].

Conventional morphological methods perform operations by using a fixed

structuring element on all the image pixels, although the size and shape of the

structuring element can be arbitrarily designed. This presents problems since

the local properties of input image pixels may not be identical everywhere and

a fixed structuring element may not meet our purposes. In this paper, we

present an adaptive morphological edge-linking algorithm. Adaptive dilation

F.Y. Shih, S. Cheng / Information Sciences 167 (2004) 9–21 11

operation is applied at each endpoint with an adaptive elliptical structuring

element. The size and orientation of the structuring element are adjusted

according to local properties, such as slope and curvature. Post-processes of

thinning and pruning are also applied.This paper is organized as follows. Section 2 introduces the adaptive

mathematical morphology. Section 3 presents the proposed adaptive mor-

phological edge-linking algorithm. Section 4 shows the experimental results.

Conclusions are made in Section 5.

2. The adaptive mathematical morphology

2.1. Traditional mathematical morphology

Mathematical morphology provides an effective tool for image processing

and analysis. It has been used for extracting image components that are useful

in the representation and description of shape, such as boundaries, skeletons,

and the convex hull [4]. The language of mathematical morphology is set

theory. Sets in mathematical morphology represent objects in an image. The

morphological operators deal with two images. The image being processed isreferred to as the active image, and the other image being a kernel is referred as

the structuring element. Two basic morphological operations, dilation and

erosion, are defined below:

Definition 1. Let A and B be subsets of N -dimensional Euclidean space EN . The

dilation of A by B is defined by

A� B ¼ fc 2 EN jc ¼ aþ b for some a 2 A and b 2 Bg:

Definition 2. Let A and B be subsets of EN . The erosion of A by B is defined by

A� B ¼ fx 2 EN jxþ b 2 A for every b 2 Bg:

Another important operation, translation, is defined below:

Definition 3. Let A be a subset of EN and x 2 EN . The translation of A by x isdefined by

ðAÞx ¼ fcjc ¼ aþ x for some a 2 Ag:

In binary image processing, the dilation of pattern A by structuring element

B is to grow a translated version of B centered at each point of A and union all

of the translated versions. The erosion of pattern A by B is to check the pointsof A where the translated version of B is completely contained in A. Dila-

tion and erosion are fundamental to morphological processing. Many other


operations are defined based on the combination of these two operations, such

as opening, closing, hit-or-miss transform, thinning, pruning [9,10].

2.2. Adaptive mathematical morphology

Traditional morphological operators process an image by using a struc-

turing element of fixed shape and size over the entire image pixels. In adaptivemorphology, rotation and scaling factors are incorporated. The structuring

elements are adjusted according to the local properties of an image. In order

to introduce the adaptive morphology, we first describe several terminologies

[13].

Assume that the sets in consideration are always connected and bounded.

Let the boundary oB of a set B be the set of points, all of whose neighborhoods

intersect both B and its complement Bc. If a set B is connected and has no holes,

it is called simply connected. If it is connected but has holes, it is called multiply

connected. In this paper, we refer the concept of oB as the continuous boundary

of a structuring element in Euclidean plane and only the simply connected

structuring element is considered. Therefore, the transformations performed on

the structuring element B are replaced by the transformations on its boundary

oB in Euclidean plane and followed by a positive filling operator. The positive

filling ½oB�þ of a set B is defined as the set of points that are inside oB.The adaptive morphological dilation and erosion can be defined by slightly

modifying Definitions 1 and 2 of the traditional operations.

Definition 4. Let A and B be subsets of EN . The adaptive morphological dilationof A by a structuring element B is defined by

A c� B ¼ fc 2 EN jc ¼ aþ b̂ for some a 2 A and b̂ 2 ½RðaÞSðaÞoB�þg;

where S and R are the scaling and rotation matrices respectively.

Definition 5. Let A and B be subsets of EN . The adaptive morphological erosionof A by a structuring element B is defined by

A bHB ¼ fc 2 EN jcþ b̂ 2 A for some a 2 A and every b̂ 2 ½RðaÞSðaÞoB�þg:

The broken edge segments can be linked gradually by using the adaptive

morphological dilation as shown in Fig. 1. Fig. 1(a) is the input signal with

gaps. Fig. 1(b) shows an adaptive dilation at the endpoints. The reason for

choosing the elliptical structuring element is that by using appropriate majorand minor axes, all kinds of curves can be linked smoothly. Fig. 1(c) shows the

result of an adaptive dilation operation.

Fig. 1. (a) Input signal with gaps, (b) adaptive dilation using elliptical structuring elements, and

(c) the result of an adaptive dilation operation.


3. The adaptive morphological edge-linking algorithm

Removing noisy edge segments, checking endpoints, adaptive dilation,

thinning, and pruning constitute a complete edge-linking algorithm. If large

gaps occur, this algorithm need be applied several times until no more gap

exists or a predefined number of iterations has been reached.Step 1: Removing noisy edge segments

If the length of one edge segment is shorter than a threshold value, then this

segment is removed. We use a value 3 in our program.

Step 2: Detecting all the endpoints

We define endpoint as any pixel having only one 8-connected neighbor. All

the edge points are checked to extract the entire endpoint set.

Step 3: Applying adaptive dilation operation at each endpoint

From each endpoint, we choose a range of pixels along the edge. From theproperties of this set of pixels, we obtain the rotation angle and size of the


elliptical structuring element. In our program, we use increasingly sized range

of pixels for each iteration. It is referred to as an adjustable parameter sdenoting the size of the range of pixels from the endpoint. We take two pixels

from this range of pixels, the left-most side p1 ðx1; y1Þ and the right-most side p2ðx2; y2Þ. We use the equation slope¼ðy2 � y1Þ=ðx2 � x1Þ to compute the slope.

The rotation angle of the elliptical structuring element can be obtained by using

the equation h ¼ tan�1(slope). The size of the elliptical structuring element is

adjusted according to the number of the pixels in the set, the rotation angle h,and the distance between p1 and p2. In our program, we use the elliptical

structuring element with fixed b ¼ 3 and a changing from 5 to 7. These values

are based on our experimental results. For a big gap, by using a ¼ 7 the gap

can be linked gradually in several iterations. For a small gap, using a ¼ 5 isbetter than using 7. If a ¼ 3 or 4 is used for a small gap, the structuring element

will be almost a circle, and the thinning algorithm does not work well.

As we know, an ellipse can be represented as

x2=a2 þ y2=b2 ¼ 1;

where a and b denote, respectively, the major and minor axes. If the center of

the ellipse is shifted from the origin to ðx0; y0Þ, the equation becomes

ðx� x0Þ2=a2 þ ðy � y0Þ2=b2 ¼ 1:

If the center is at (x0; y0) and the rotation angle is h (�p=26 h6 p=2), theequation becomes: ððx� x0Þ � cos h� ðy � y0Þ � sin hÞ2=a2 þ ððx� x0Þ � sin h�ðy � y0Þ � cos hÞ2=b2 ¼ 1. Because the image is discrete, the rounded values are

used in defining the elliptical structuring element. At each endpoint, we per-

form an adaptive dilation operation by using the elliptical structuring element.

Therefore, the broken edge segment will be extended along the slope direction

by the shape of ellipse.

Step 4: Thinning

After applying the adaptive dilation at each endpoint, the edge segments are

extended in the direction of local slope. Because the elliptical structuring ele-ment is used, the edge segments grow a little fat. Morphological thinning is

used to obtain the edge with one pixel of width.

Step 5: Branch pruning

The adaptively dilated edge segments after thinning may contain noisy short

branches. These short branches must be pruned away. The resulting skeletons

after thinning have width of one pixel. We define a root point as a pixel having

at least three pixels in 8-connected neighbors. From each endpoint we trace

back along the existing edge. If the length of this branch is shorter than athreshold value after it reaches a root point, the branch is pruned.


Step 6: Decision

The program terminates when no endpoint exists or when a predefined

number of iterations has been reached. We use 8 iterations as the limit in our

program.

4. Experimental results

Fig. 2(a) shows an original elliptical edge and Fig. 2(b) shows its randomly

discontinuous edge. The edge-linking algorithm is experimented on Fig. 2(b).

1. Using circular structuring elements

Fig. 3 shows the results of using circular structuring elements with r ¼ 3 and

r ¼ 5, respectively, in 5 iterations. Compared with the original ellipse in Fig.

2(a), we know if the gap is larger than the radius of the structuring element, it is

difficult to link smoothly. However, if a very big circular structuring element is

used, the edge will look hollow and protuberant. Besides, it can obscure thedetails of the edge.

2. Using a fixed sized elliptical structuring element and a fixed range of pixels

to measure the slope

In this experiment, we use an elliptical structuring element with a fixed

minor axis b ¼ 3 and a fixed major axis a ¼ 5, 7, 9, respectively. We use a fixed

range of pixels to measure the local slope for each endpoint. That is, we use the

Fig. 2. (a) Original elliptical edge, and (b) its randomly discontinuous edge.

Fig. 3. Using circular structuring elements in five iterations with (a) r ¼ 3 and (b) r ¼ 5.

Fig. 4. An elliptical structuring element with a ¼ 5, b ¼ 3, and s ¼ 7.


same range of pixels counting from the endpoint in each iteration. The

parameter s denotes the range of pixels from endpoint.

Fig. 4 shows the result of using an elliptical structuring element a ¼ 5, b ¼ 3,

s ¼ 7. The result is not good because a ¼ 5 is too small for some big gaps. Fig.

5 shows the result of using a ¼ 7, b ¼ 3, s ¼ 9. Fig. 6 shows the result of using




a ¼ 7, b ¼ 3, s ¼ 11. There is not much difference between Figs. 5 and 6.

Compared with the original ellipse in Fig. 2(a), Figs. 5 and 6 have a few

shortcomings, but they are much better than Fig. 4. Fig. 7 shows the result of

using a ¼ 9, b ¼ 3, s ¼ 11. Therefore, using a ¼ 7 or a ¼ 9, we can obtain a

reasonable good result except for a few shifted edge pixels.


Fig. 8. Using an elliptical structuring element with a ¼ 5, b ¼ 3, and adjustable s.


3. Using a fixed sized elliptical structuring element for every endpoint but using

adjustable sized range of pixels to measure local slope in each iteration

Fig. 8 shows the result of using a ¼ 5, b ¼ 3, and adjustable s. Fig. 9 shows

the result of using a ¼ 7, b ¼ 3, and adjustable s. Compared with Figs. 4 and 6,

Figs. 8 and 9 are better in terms of elliptical smoothness. This is true because

Fig. 9. Using an elliptical structuring element with a ¼ 7, b ¼ 3, and adjustable s.


after each iteration, we increase the range of pixels used to measure local slope,

and more information from the original edge is taken into account.

4. Using adjustable sized elliptical structuring elements for every endpoint and

using adjustable sized range of pixels to measure local slope in each iteration (the

adaptive morphological edge-linking algorithm)

In this experiment we use adjustable sized elliptical structuring elements with

a changing from 5 to 7, b ¼ 3, and use adjustable s.

Fig. 10. Using the adaptive morphological edge-linking algorithm.


Fig. 10 shows the result of using the adaptive morphological edge-linking

algorithm. Compared with Figs. 6 and 9, Fig. 10 is not obviously better. The

advantage of the adaptive method is that we can adjust the parameters a and sautomatically, while the parameters for Figs. 6 and 9 are fixed, i.e., they workwell for a certain case, but may not work well for other cases.

Fig. 11(a) shows the elliptical shape with added uniform noise. Fig. 11(b)

shows the shape after removing the noise. Fig. 11(c) shows the result of using

the adaptive morphological edge-linking algorithm. Compared with Fig. 10,

Fig. 11(c) has several shortcomings. This is because after removing the added

noise, the shape is changed at some places.

Fig. 11. (a) The elliptical shape with added uniform noise, (b) the shape after removing noise, and

(c) using the adaptive morphological edge-linking algorithm.


5. Conclusions

In this paper we have presented an adaptive morphological edge-linking

algorithm using the elliptical structuring element. Adaptive dilation is appliedat each endpoint. The orientation and size of the elliptical structuring element

are adjusted according to local properties at each endpoint. The experimental

results show the success of our algorithm.

The adaptive morphological operations proposed in this paper are not re-

stricted to edge linking applications, and the shape of the structuring element is

not limited to ellipses. The shape and size of the structuring element and its

geometric transformations during processing can be carefully designed to fit

different application requirements.

References

[1] J.R. Parker, Algorithms for Image Processing and Computer Vision, Wiley Computer

Publishing, 1997.

[2] J. Basak, B. Chanda, On edge and line linking with connectionist model, IEEE Trans. Syst.,

Man, Cybernet. 24 (3) (1994) 413–428.

[3] A. Marteli, An application of heuristic search methods to edge and contour detection,

Graphics Image Process. 19 (2) (1976) 73–83.

[4] R.C. Gonzalez, R.E. Woods, Digital Image Processing, Prentice Hall, 2002.

[5] R. Nevatia, Locating objects boundaries in textured environments, IEEE Trans. Comput.

(1976) 1170–1175.

[6] V.S. Nalwa, E. Pauchon, Edgel aggregation and edge description, Comput. Vision, Graphics,

Image Process. 40 (1) (1987) 79–94.

[7] S.-M. Liu, W.-C. Lin, C.-C. Liang, An iterative edge linking algorithm with noise removal

capability, in: 9th International Conference on Pattern Recognition, vol. 2, 1988, pp. 1120–

1122.

[8] J.C. Russ, The Image Processing Handbook, CRC Press, 1992.

[9] J. Serra, Image Analysis and Mathematical Morphology, Academic, New York, 1982.

[10] R.M. Haralick, S.R. Sternberg, X. Zhuang, Image analysis using mathematical morphology,

IEEE Trans. Pattern Anal. Machine Intell. 9 (4) (1987) 532–550.

[11] F.Y. Shih, O.R. Mitchell, Threshold decomposition of grayscale morphology into binary

morphology, IEEE Trans. Pattern Anal. Machine Intell. 11 (1) (1989) 31–42.

[12] F.Y. Shih, P. Puttagunta, Recursive soft morphological filters, IEEE Trans. Image Process. 4

(7) (1995) 1027–1032.

[13] F.Y. Shih, S.-C. Pei, J.-H. Horng, Edge linking using adaptive morphology, Technical Report,

National Taiwan University (1997).

[14] C.S. Chen, J.L. Wu, Y.P. Hung, Theoretical aspects of vertically invariant gray-level

morphological operators and their application on adaptive signal and image filtering, IEEE

Trans. Signal Process. 47 (4) (1999) 1049–1060.

[15] B.-K. Jang, R.T. Chin, Analysis of thinning algorithms using mathematical morphology, IEEE

Trans. Pattern Anal. Machine Intell. 12 (6) (1990) 541–551.

adaptive mathematical morphology for edge linking

Documents