Multiscale image representation using scale-space theory

Leyza Baldo Dorini, Neucimar Jeronimo Leite,Instituto de Computacao, Unicamp,

Caixa Postal 6176, 13084-971, Campinas, SPE-mail: {ldorini, neucimar}@ic.unicamp.br

Abstract: Image simplification reduces the in-formation content of an image while preserv-ing as much as possible its essential features,being frequently used as a preprocessing stagein several algorithms to suppress undesired de-tails such as noise. In this paper, we ex-plore the scale-space properties of a toggle op-erator defined using mathematical morphologyoperations. The scale-space theory is a mul-tiscale representation approach where the in-herent multiscale nature of real-world imagesis represented by embedding the original signalinto a family of simplified signals, created bysuccessively removing image structures acrossscales. Some image segmentation examplesdemonstrate that the use of simplified imagessignificantly improves the results. Keywords:image processing, image segmentation.

1 Introduction

Multiscale approaches have been largely con-sidered, playing an important role when de-signing automatic methods to cope with real-world measurements where, in most of thecases, there is no prior information about whichwould be the appropriate scale. The basic ideabehind a multiscale analysis is to embed theoriginal signal into a family of derived signals,allowing the analysis of different representationlevels and, further, the choice of the ones ex-hibiting the interest features.

One of the basic problems that arises whenusing multiscale methods originates from thedifficulty to relate meaningful information ofthe signal across scales. In [19], Witkin pro-posed a novel multiscale approach, namedscale-space, where image structures at differ-ent scales can be handled consistently, giventhat the representation of an interest signal fea-ture describes a continuous path across scales.

Thus, it is possible to relate information ob-tained in different representation levels, andhave a precise localization of the interest fea-tures in the original signal. The scale param-eter depends on the filtering procedure usedto generate the scale-space, but usually corre-sponds to a spatial (or temporal) scale.

Another important characteristic is that thetransformation to a coarser level in the scale-space representation does not introduce arti-facts, that is, structures present at a specificscale are also present at all finer scales. In otherwords, the image is simplified in such a waythat new features are not created. This prop-erty is called monotonicity or causality, sincethe number of features must necessarily be amonotonic decreasing function of scale [19]. Itis not required that the feature positions at ahigher scale be a subset of those at lower scales,that is, features can drift spatially across scaleson continuous paths. If no drift occurs, thescale-space is said to possess strong causality.

Other required property for scale-space for-mulations is fidelity [3]. Let f be the originalimage (scale zero) and let T (σ) be the scale-space operator such that T (σ)f is the obser-vation at scale σ. The fidelity property re-quires that the scale-space signal converges tothe original signal as σ approaches zero, i.e.,

limσ→0

T (σ)f = f (1)

Euclidean invariance is also desired, that is,a translation and/or rotation of the originalimage should imply in an equally translatedand/or rotated scale-space.

In this paper, we present a brief introduc-tion to scale-space theory and define a scale-space based on morphological operations. Weshow some computational experiments to illus-trate how image simplification contributes toimprove segmentation results.

— 795 —

2 Scale-space approaches

Since the introduction of the scale-space theory,a large number of formulations have been pro-posed. In the linear approach, formalized byIjima [6] and Witkin [19], a family of imagesis generated by convolving the original imagewith a zero-mean gaussian kernel, where thestandard deviation is the scale parameter. Theresulting scale-space obviously satisfies the fi-delity requirement, since as the standard devi-ation goes to zero, the original image is ap-proached. The use of a symmetric gaussianguarantees Euclidean invariance, and causal-ity follows directly from the filtering process.It has been proved [1] that the gaussian is theonly convolution kernel that satisfies the mono-tonic property when using as features the zero-crossings of the Laplacian in 1D.

Since the gaussian is the Green’s function(or propagator) of the diffusion equation, PDEmethods can be also used to generate a gaus-sian scale-space. The solutions to the isotropicdiffusion or heat equation

∂F (x, t)∂t

= ∆F (x, t), (2)

where ∆ is the Laplacian, are samples of thegaussian scale space, for a fixed time t [7].

However, this formulation causes the result-ing images to be blurred, a problem for severalhigh level tasks that require the correct local-ization of the edges, which correspond to thephysical boundaries of objects. This fact can beseen as an example of the uncertainty principle,where the gaussian kernel (a low-pass filter) re-stricts the frequency spectrum, thus “blurring”the spatial domain [3].

Also, for dimensions higher than one, anyconvolution kernel used to obtain the scale-space introduces new features as the scale in-creases and, thus, the monotone requirementdoes not hold for linear filters and signal ex-trema [9]. This has motivated the developmentof non-linear approaches, such as the general-ization of the diffusion equation to other non-linear PDEs.

The anisotropic diffusion technique [13], forexample, decreases diffusion in the direction ofthe gradient with high magnitude, and pre-serves the monotone property for signal ex-trema when appropriate diffusion coefficientsare chosen. However, in a general way,

these techniques have a high computationalcost and depend on several parameters. Thetechniques based on mathematical morphologyovercome this problems, and have shown togenerate scale-spaces with interesting simpli-fication properties. In the next section, wepresent basic mathematical morphology defini-tions and some formulations based on morpho-logical operators.

3 Basic definitions and mor-phological scale-spaces

Mathematical morphology is a non-linear im-age analysis technique that extracts image ob-ject’s information by describing its geometricalstructures in a formal way. Since its formaliza-tion by Matheron e Serra [11], [15], mathemati-cal morphology has been largely used in severalpractical and theoretical image processing andanalysis problems.

Let f : D ⊂ Rn → R be an image functionand g : G ⊂ Rn → R be a structuring function.The two fundamental operations of gray-scalemorphology, erosion and dilation, are definedas follow:

Definition 3.1 [16] (Dilation) The dilationof the function f(x) by the structuring functiong(x), (f ⊕ g)(x), is given by:

(f ⊕ g)(x) = supt∈G∩D−x

{f(x− t) + g(t)} (3)

Definition 3.2 [16] (Erosion) The erosion ofthe function f(x) by the structuring functiong(x), (f g)(x), is given by:

(f g)(x) = inft∈G∩D−x

{f(x− t)− g(t)} (4)

where Dx is the translate of D, Dx = {x + t :t ∈ D}, and D is the reflection of D.

To introduce the notion of scale, we canmake the basic morphological operations of ero-sion and dilation scale-dependent by defining ascaled structuring function gσ : Gσ ⊂ R2 → R,such that [7]

gσ(x) = |σ|g(|σ|−1x) x ∈ Gσ, ∀σ 6= 0, (5)

where Gσ = {x : ‖x‖ < R} is the support re-gion of the function gσ. To ensure reasonablescaling behavior, some other conditions are

— 796 —

(a) (b) (c) (d)

Figure 1: (a) Original image and scale-space images (b) gaussian (σ = 5), (c) dilation (σ = 2),(d) erosion (σ = 2) and (e) SMMT (σ = 2).

necessary [7], requiring a monotonic decreas-ing structuring function along any radial direc-tion from the origin. To avoid level-shifting andhorizontal translation effects, respectively, onemust also consider that [7]

supt∈Gσ

{gσ(t)} = 0 and gσ(0) = 0. (6)

Jackway [7] and Boomgaard [17] showed thatthe parabolic erosions and dilations are themorphological analogue of the gaussian scale-space. In this paper, we use as structuringfunction g(x, y) = −max{|x|, |y|}, that in thescaled version is

gσ(x, y) = −σ−1 max{|x|, |y|} (7)

where σ represents the scale. Observe that, fora 3×3 structuring element (used in this work),gσ is zero at position 0 and −σ−1 otherwise.We drop the | | symbol on Equation 7 to em-phasize that we only work with positive scales.

3.1 Scale-spaces based on morpho-logical operators

In the mathematical morphology context,scale-spaces are generated by filtering gray-scale signals using specific combinations ofmorphological operators based on scaled struc-turing elements. Jackway [7] introduced ascale-space based on the Multiscale Morpholog-ical Dilation Erosion (MMDE) operator, whichperforms a scaled dilation for positive scalesand a scaled erosion for the negative ones.Since the interest features are different depend-ing on the scale signal (image maxima when thescale is positive and image minima when it isnegative), this approach can be seen as two dif-ferent scale-spaces. This leads the “negative”scales to be more a convenient notation whendisplaying a scale-space representation than anew definition. Also, due to the gray-level bias

of the dilation and erosion operations, this ap-proach is sensitive to noise.

Leite and Teixeira [8] defined a new operatorthat explores the idempotence of the MMDE,establishing a relation between the structur-ing function gσ and the extrema that persistat a given scale σ. Using the extrema set ob-tained during the filtering process as markers ina homotopic modification of the original image,they avoided the spatial shifting of the water-shed lines.

Scale-spaces based on openings or closingshave the nice properties of edge localizationand impulse noise suppression. However, themonotonicity property does not hold for 2D fil-tering of gray-level images [12] [4]. Another ap-proach generates scale-spaces using 2D sieves(area morphology), which preserve the scale-space monotone requirement for both extremaand edges as features [14].

In a previous work [5], we have defined a newscale-space based on the following toggle opera-tor, named Self-dual Multiscale MorphologicalToggle (SMMT).

(f � gσ)k(x) = ψk1 (x) if ψk

1 (x)− f(x) < f(x)− ψk2 (x),

f(x) if ψk1 (x)− f(x) = f(x)− ψk

2 (x),ψk

2 (x) otherwise,(8)

where ψk1 (x) = (f ⊕ gσ)k(x), that is, the dila-

tion of f(x) with the scaled structuring func-tion gσ k times. In the same way, ψk

2 (x) =(f gσ)k(x).

It can be proved that this operator obeys thecausality principle when using as features im-age extrema (there is no need to consider im-age maxima and minima separately as in pre-vious approaches). Figure 1 compares imagesfrom different scale-spaces: gaussian, dilation,erosion and SMMT. Note that our approachleads to an image simplification that does notdisplace the boundaries. Also, this new scale-

— 797 —

(a) (b) (c)

Figure 2: Simplification obtained by considering successive iterations (1, 3 and 5, respectively)of the operator at scale σ = 1.

space is self-dual, i.e., there is a symmetrictreatment of foreground and background, thusreducing the gray-level bias.

It follows from Equations 5 and 6 that

|σ| → 0 ⇒ gσ(x) →{

0, if x = 0;−∞, otherwise

(9)

which leads directly to the fidelity requirement.When using the structuring function defined inEquation 7, we also have the following equiva-lence for the SMMT operator:

(f � gσ3)k(x) == (f � gσ2k+1

)1(x) (10)

where the subscript on σ indicates the struc-turing element size. In a few words, to per-form k iterations of the primitives using a 3×3structuring element and then apply the toggleoperator is equivalent to performing one itera-tion using a structuring element of size 2k+ 1.This implies that, as more iterations are done,a wider neighborhood is being considered, andregions tend to be merged. The defined opera-tor can be seen as a quasi -connected operator,in the sense that it simplifies the image by cre-ating quasi-flat zones.

Definition 3.3 [10] (R-flat-zone) Two pixelsx, y belong to the same R-flat zone of a functionf if and only if there exists an n-tuple of pixels(p1, p2, . . . , pn) such that p1 = x and pn = y,and for all i, (pi, pi+1) are neighbors and satisfythe symmetrical relation fpiR fpi+1.

When R is the equality, we are dealing with flatzones, which consist on connected componentswhere the pixel value is constant. Here, weconsider quasi-flat zones,i.e., |fp−fq| ≤ λ. Notethat a flat zone can be formed by a single point,since there is no restriction on its size [10].

Figure 2 illustrates the creation of thesequasi-flat zones. We show the result of thegray levels of a small portion of an image whentransformed by successive iterations of the de-fined operator, k = 1 . . . 5, with σ = 1. Notethat quasi-flat zones are created.

4 Some Results

Image segmentation consists basically on par-titioning an image into a set of disjoint (non-overlapping) and homogeneous regions whichare supposed to correspond to image objectsthat are meaningful to a certain application.Accurate segmentation is a fundamental stagein a large number of image processing appli-cations, such as pattern recognition and repre-sentation. In a morphological framework, thisis typically done by first extracting markers ofthe significant structures, and then using thewatershed transform [2] to extract the contoursof these structures as accurately as possible.

However, image extrema (frequently used asmarkers) can correspond to insignificant struc-tures or noise, and the gradient image, oftenassociated to the watershed algorithm, natu-rally yields high responses in textured areas,for example. A common approach used to pre-vent over-segmentation is to select markers ac-cording to some criteria, such as contrast. Butsometimes this procedure is not enough, andit is necessary to simplify the image before ex-tracting the markers.

For the examples of Figures 3 and 4, we usethe h-maxima transform to suppress all im-age maxima whose contrast is lower than aspecified value h, and use the remaining ex-tended extrema as markers. The h-maximaconsists on a geodesic reconstruction by ero-

— 798 —

(a) (b) (c) (d)

Figure 3: Segmentation results for the skyg image with non-uniform illumination. (a) originalimage, and using transformed images (b) σ = 60 and k = 1, with h = 2; (c) σ = 60 and k = 1,with h = 10, and (d) based on the original image using h = 10.

(a) (b) (c) (d) (e)

Figure 4: Segmentation results for the peppers image. (a) original image, using original imagewith (b) h = 19, and using transformed image σ = 21 and k = 2, with (c) h = 22 and (d)h = 25, and (e) segmentation obtained when joining (c) and (d).

sion of the translated image (f + h) over f :δ∞(f, f + h) [16]

Figure 3 shows the segmentation results ob-tained for an ill-illuminated image (the param-eters are listed in the caption). In the last col-umn we show the best result obtained usingmarkers from the original image. Observe thatthe markers extracted from the transformedimage are less sensitive to illumination changes,yielding a segmentation that enhances the mostimportant structures of the image without in-troducing artifacts.

Figure 4 illustrates another result. Fig-ure 4(b) shows the segmentation when usingthe original image to extract the markers and tocalculate the watershed. Note that the resultsobtained using the image transformed by theoperator were highly improved (Figures 4(c)and (d)). Figure 4(e) joins the results of (c)and (d).

Figure 5 shows an example where the itera-tive application of the operator leads to a con-tour regularization. Note that the gradient ex-tracted from the original image is weak, whichcan cause a leaking of the watershed functionand, consequently, define a wrong segmenta-tion result. Also, this scheme can be applied tonoisy images, since the simplification performs

a filtering on the original signal, while preserv-ing and enhancing its boundaries.

(a) (b) (c)

Figure 5: Improvement of the gradient imageby applying a scale-space toggle operator. (a)Original and regularized images, (b) the weakand the well-defined contours, (c) the differentsegmentation results.

When applying the defined operator, in someneighborhood of an important minimum (maxi-mum), the pixels values will be eroded (dilated)in such a way that it is possible to identify thesignificant extrema of the image and their in-fluence zones.

In this sense, we define a new thresholdingoperation as follows:

— 799 —

(a) (b) (c) (d) (e)

Figure 6: Segmentation example for a historical document image. (a) Original image, (b/d) theabove operator result compared with (c/e) the moving averages approach.

(a) (b) (c) (d)

Figure 7: Image filtering results for an image corrupted by salt-and-pepper noise. (a) noisyimage, (b) using ψ1 = βα, (c) using ψ2 = αβ and (d) result for two iterations of the definedoperator using as primitives ψ1 = βα and ψ2 = αβ.

(f � gσ)k(x) ={255 if ψk

1 (x)− f(x) <= f(x)− ψk2 (x)

0 otherwise, (11)

and use as primitives the scaled erosion anddilation operations. Figure 6 shows the seg-mentation of a historical document in whichthe front side of the paper contains ink compo-nents from its verso side.

The results were compared against the mov-ing averages [18] algorithm, specially designedto segment text images. Note that our ap-proach is less sensitive to the presence of theverso components.

The operator defined in Equation 8 can bealso applied to image filtering. In order to ob-tain satisfactory results, one must apply the op-erator iteratively using appropriate primitives.In Figure 7, we show a filtering example of animage corrupted by salt-and-pepper noise. Weuse as primitives a combination of openings andclosings. An opening consists on an erosionfollowed by a dilation, α = δε, and a closingconsists on a dilation followed by an erosion,β = εδ.

Note that when using the primitives sepa-rately, the image is not adequately filtered. Agood filtering result of the original noisy imagecan be obtained by applying the toggle oper-ator twice, as illustrated in Figure 7(d). TheRoot Mean Square Error scores correspondingto the images of Figure 7 are: (a) 44.98, (b)

17.15, (c) 19.72 and (d) 11.21, respectively.

5 Conclusions

In this work, we have presented a brief intro-duction to scale-space theory, a multiscale ap-proach to construct a coarse-to-fine represen-tation of an image. The filtering operation re-duces the information content of the image bysuccessively removing image details. This sim-plification is important in several image analy-sis tasks, such as image segmentation, exploredin this paper.

Due to the limitations of the linear approach,we focus this work on non-linear approachesbased on mathematical morphology. We intro-duce a multiscale morphological simplificationalgorithm where we explored the variation onscale and number of iterations parameters tocreate simplified versions of an image. This ap-proach yields better segmentation results whencompared, for instance, with the definition ofmarkers or the gradient information obtaineddirectly from the original images.

References

[1] J. Babaud, Witkin A.P., M. Baudin, andR.O. Duda. Uniqueness of the gaus-sian kernel for scale-space filtering. IEEE

— 800 —

Transactions on Pattern Analysis and Ma-chine Intelligence, 8:15–25, 1986.

[2] S. Beucher and F. Meyer. The morpholog-ical approach to segmentation: the water-shed transformation. Mathematical Mor-phology in Image Processing, pages 433–451, 1993.

[3] J.H. Bosworth and S.T. Acton. Mor-phological scale-space in image process-ing. Digital Signal Processing, 13:338–367,2003.

[4] M.H. Chen and P.F. Yan. A multiscaleapproach based on morphological filter-ing. IEEE Transactions on Pattern Analy-sis and Machine Intelligence, 42(12):3377–3386, 1989.

[5] L. E. B. Dorini and N. J. Leite. A scale-space toggle operator for morphologicalsegmentation. In 8th International Sympo-sium on Mathematical Morphology, pages101–112, 2007.

[6] T. Iijima. Basic theory of pattern nor-malization for the case of a typical one-dimensional pattern. In Bull. Electrotech.Lab, pages 368–388, 1962.

[7] P. T. Jackway and M. Deriche. Scale-spaceproperties of the multiscale morphologi-cal dilation-erosion. IEEE Transactionson Pattern Analysis and Machine Intelli-gence, 18:38–51, 1996.

[8] N. J. Leite and M. D. Teixeira. An idem-potent scale-space approach for morpho-logical segmentation. In MathematicalMorphology and its Applications to Im-age and Signal Processing, pages 291–300.Kluwer Academic Publishers, 2000.

[9] L. M. Lifshitz and S. M. Pizer. A multires-olution hierarchical approach to imagesegmentation based on intensity extrema.IEEE Transactions on Pattern Analysisand Machine Intelligence, 12(4):529–540,1990.

[10] P. Maragos and F. Meyer. A pde approachto nonlinear image simplification via lev-elings andreconstruction filters. In Inter-national Conference on Image Processing,pages 938–941, 2000.

[11] G. Matheron. Random Sets and IntegralGeometry. John Wiley and Sons, 1975.

[12] K. Park and C Lee. Scale-space usingmathematical morphology. IEEE Trans-actions on Pattern Analysis and MachineIntelligence, 18(11):1121–1126, 1996.

[13] P. Perona and J. Malik. Scale-space andedge detection using anisotropic diffusion.IEEE Transactions on Pattern Analysisand Machine Intelligence, 42(12):629–639,1990.

[14] P. Salembier and J. Serra. Flat zones fil-tering, connected operators, and filters byreconstruction. IEEE Transactions on Im-age Processing, pages 1153–1160, 1995.

[15] J. Serra. Image Analysis and Mathemati-cal Morphology, volume 2: Theoretical Ad-vances. Academic Press, 1988.

[16] P. Soille. Morphological Image Analy-sis: Principles and Applications. Springer-Verlag, 2003.

[17] R. van den Boomgaard. Mathematicalmorphology: extensions towards computervision. PhD thesis, University of Amster-dam, 1992.

[18] P. Wellner. Adaptive thresholding for thedigital desk. Technical Report EPC1993-110, Xerox, 1993.

[19] A. P. Witkin. Scale-space filtering: a newapproach to multi-scale description. In Im-age Understanding, pages 79–95. Ablex,1984.

— 801 —

Acknowledgments

The authors are grateful to FAPESP(07/52015-0; 05/04462-2) and MCT/CNPq(472402/2007-2) for the financial support ofthis work.

— 802 —